Skip to content

test(convert): verify delta data survival and interrupted init recovery - #150

Merged
imlk0 merged 3 commits into
masterfrom
ci/persist-sentinel-recovery
Oct 13, 2026
Merged

imlk0 merged 3 commits into
masterfrom
ci/persist-sentinel-recovery

Conversation

@imlk0

@imlk0 imlk0 commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

Extends the disk-persist regression scenarios (issues #140 / #141) from "boots twice" to what issue #141 actually asked for: delta content survival across a forced power loss, and recovery from an interrupted delta initialization. No new CI jobs: the existing 4 test-convert-persist jobs each gain a third boot.

What each persist job now runs

  1. Boot Add github workflow for anolis8 rpm build #1 reaches login and a sentinel unit (injected pre-convert into the unencrypted image via qemu-nbd + mount) writes a random UUID to /var/lib and syncs. /var/lib lives on the dm-snapshot COW, i.e. the persistent delta.
  2. Forced VM kill (power loss), then boot integrity: fix udev env SYSTEMD_READY=0 when integrity is on #2 of the same overlay must reach login and report the identical sentinel UUID: the delta's content survived.
  3. Interrupted initialization: the COW header chunk (one snapshot chunk, 16 sectors) is blanked through the opened LUKS mapping (read-write, so the dm-integrity journal replays), simulating a crash between marking the delta volume initialized and committing the snapshot header. Boot KeyProvider: add stdin key provider Plugin #3 must recover to a working snapshot. The previous sentinel is gone by design in this case (the recovery semantics reinitialize the delta).

Implementation notes

  • Sentinel reporting goes to /dev/console (serial), so the harness only greps the boot log it already collects: no SSH, console input, or cloud-init (the guest FDE service reads cloud-init user data for its config, so a NoCloud seed would interfere).
  • The injection uses qemu-nbd + a plain ext4 mount rather than libguestfs: Alinux 3 ships virt-customize, Alinux 4's merged libguestfs package does not, and the pre-converted rootfs partition is unencrypted anyway.
  • test_qemu_boot_direct gained an optional required-marker argument (success = login AND marker), used for the sentinel checks.

Verification

Local end-to-end (disk-persist + integrity=true + stable key, UKI, TCG):

phase result
boot #1 login + CRYPTPILOT_SENTINEL_WRITTEN aaffcc3e-…
boot #2 after forced kill login + CRYPTPILOT_SENTINEL_OK aaffcc3e-… (identical UUID)
blanked COW header COW header chunk blanked (interrupted initialization simulated)
boot #3 login + fresh sentinel (recovered)

bash -n + shellcheck clean, YAML valid. CI will run both scenarios on both distros.

Refs: #140, #141.

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

imlk0 added 3 commits October 13, 2026 17:27
The persist scenarios only asserted that the same overlay boots twice,
not that the delta's content survives a reboot. Inject a oneshot unit
into the pre-converted image that writes a random sentinel to /var/lib
(living on the dm-snapshot COW) and reports to the serial console; the
second boot now must see the identical sentinel UUID after a forced VM
kill, which is the power-loss case issue #141 asked to cover.

The third boot covers the interrupted-initialization case: the COW
header chunk is blanked through the opened LUKS mapping (read-write, so
the dm-integrity journal replays), simulating a crash between marking
the delta volume initialized and committing the snapshot header; the
boot must recover by reinitializing the snapshot.

The sentinel injection uses qemu-nbd and a plain mount instead of
libguestfs: Alinux 3 ships virt-customize, Alinux 4's merged libguestfs
package does not, and the pre-converted rootfs partition is unencrypted
ext4 anyway.

Assisted-by: Claude:glm-5.3
Signed-off-by: Kun Lai <laikun@linux.alibaba.com>
lsblk's FSTYPE column comes from the udev database, and the CI test
container runs no udev, so the sentinel injection found no ext4 root
partition and failed within seconds on every persist job. blkid probes
the device directly and needs no udev; partition nodes exist either way
because devtmpfs creates them.

Assisted-by: Claude:glm-5.3
Signed-off-by: Kun Lai <laikun@linux.alibaba.com>
…manager

The alinux4 repos started shipping guestfs-tools (pulled in by
libguestfs), so virt-customize suddenly appears in the alinux4 CI test
containers. There it is unusable: supermin 5.3.5 has no driver for
alinux4's os-release (ID_LIKE=anolis only, while alinux3 also lists
rhel/fedora/centos), so every virt-customize invocation fails instantly
with "could not detect package manager" and takes every convert test
job down with it. This also turned master's CI red.

Extend the existing "virt-customize unavailable, skip hardening" guard
to cover the unusable case, with a loud warning that stays in the logs.
The proper fix is on the packaging side (an anolis driver in supermin,
or ID_LIKE coverage in alinux4's os-release).

Assisted-by: Claude:glm-5.3
Signed-off-by: Kun Lai <laikun@linux.alibaba.com>
@imlk0
imlk0 force-pushed the ci/persist-sentinel-recovery branch from c239c43 to 765751e Compare October 13, 2026 09:27
@imlk0
imlk0 merged commit 765751e into master Oct 13, 2026
@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,您的请求已接收,请耐心等待结果。

@ostest-bot

Copy link
Copy Markdown

@imlk0 ,您好,未检测到有镜像需要构建,如需重新检测请评论 /start 。

@imlk0
imlk0 deleted the ci/persist-sentinel-recovery branch October 13, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants