Repository navigation
fix(manager): the QEMU guest boots with network and its fw_cfg arguments #16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -10,12 +10,15 @@ | |
| # - cpio, gzip, e2fsprogs (mke2fs) | ||
| # | ||
| # The kernel (vmlinuz-lite-qemu) is NOT built here - it is reused from the | ||
| # upstream wrapper qemu-assets artifact (see guest/README.md). | ||
| # | ||
| # Kernel modules (e1000.ko / qemu_fw_cfg.ko) live in guest/modules/ and are | ||
| # copied verbatim from the upstream wrapper initramfs. | ||
| # build-lite qemu-assets artifact (see guest/README.md). KERNEL_ASSETS names | ||
| # the unzipped artifact: the kernel and the e1000 / qemu_fw_cfg modules both | ||
| # come from it, so they always match. A module built for another kernel fails | ||
| # to load, and the guest boots without network or fw_cfg arguments. | ||
| set -euo pipefail | ||
|
|
||
| KERNEL_ASSETS="${KERNEL_ASSETS:?set KERNEL_ASSETS to the unzipped build-lite qemu-assets artifact}" | ||
| KERNEL_ASSETS="$(cd "$KERNEL_ASSETS" && pwd)" | ||
|
|
||
| REPO_DIR="$(cd "$(dirname "$0")/.." && pwd)" | ||
| GUEST_DIR="$REPO_DIR/guest" | ||
| OVERLAY="$GUEST_DIR/overlay" | ||
|
|
@@ -70,9 +73,21 @@ if [ ! -f "$OVERLAY/etc/ssl/certs/ca-certificates.crt" ]; then | |
| fi | ||
| fi | ||
|
|
||
| # 5. Kernel modules. | ||
| echo "[build] staging kernel modules..." | ||
| cp "$GUEST_DIR"/modules/*.ko "$OVERLAY/" 2>/dev/null || true | ||
| # 5. Kernel and the modules built for it, from the same qemu-assets artifact. | ||
| echo "[build] staging the kernel and its modules..." | ||
| KERNEL_VER="$(file -bL "$KERNEL_ASSETS/vmlinuz-lite-qemu" | sed -n 's/.*version \([^ ]*\).*/\1/p')" | ||
| [ -n "$KERNEL_VER" ] || { echo "cannot read the version of $KERNEL_ASSETS/vmlinuz-lite-qemu"; exit 1; } | ||
| LITE_ROOT="$(mktemp -d)" | ||
| trap 'rm -rf "$LITE_ROOT"' EXIT | ||
| (cd "$LITE_ROOT" && gzip -dc "$KERNEL_ASSETS/lite-initramfs.cpio.gz" | cpio -id --quiet) | ||
| for mod in e1000 qemu_fw_cfg; do | ||
| [ -f "$LITE_ROOT/$mod.ko" ] || { echo "$mod.ko is missing from the qemu-assets initramfs"; exit 1; } | ||
| cp "$LITE_ROOT/$mod.ko" "$OVERLAY/$mod.ko" | ||
| magic="$(modinfo -F vermagic "$OVERLAY/$mod.ko" | cut -d' ' -f1)" | ||
| [ "$magic" = "$KERNEL_VER" ] || { echo "$mod.ko is built for $magic, the kernel is $KERNEL_VER"; exit 1; } | ||
|
Comment on lines
+86
to
+87
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Agreed, vermagic alone is a release check. The modules now come from the same qemu-assets initramfs that build-lite built against that kernel, so symbol versions match by construction; the vermagic check catches a mixed-up artifact, which is the failure that shipped. |
||
| done | ||
| cp "$KERNEL_ASSETS/vmlinuz-lite-qemu" "$OUT_DIR/vmlinuz-lite-qemu" | ||
| echo "[build] kernel and modules: $KERNEL_VER" | ||
|
|
||
| # 6. Init script at the initramfs root (required by the kernel). | ||
| cp "$GUEST_DIR/init" "$OVERLAY/init" | ||
|
|
@@ -96,5 +111,4 @@ if [ "${CREATE_DATA_IMG:-0}" = "1" ]; then | |
| echo "[build] data.img created: $DATA_IMG" | ||
| fi | ||
|
|
||
| echo "[build] done. Copy vmlinuz-lite-qemu next to the initramfs:" | ||
| echo " cp <wrapper qemu-assets>/vmlinuz-lite-qemu $OUT_DIR/" | ||
| echo "[build] done." | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,7 +12,7 @@ | |
| /bin/busybox hostname wrapper-manager | ||
|
|
||
| # --- network (QEMU user-mode NAT) --- | ||
| /bin/busybox insmod /e1000.ko 2>/dev/null || true | ||
| /bin/busybox insmod /e1000.ko || echo "[init] e1000.ko did not load: no network" | ||
| /bin/busybox ip link set lo up | ||
| /bin/busybox ip link set eth0 up 2>/dev/null || true | ||
|
|
||
|
|
@@ -45,17 +45,19 @@ fi | |
| # read it if present. The content is split on whitespace into arguments, so | ||
| # no argument may contain a space. | ||
| MANAGER_ARGS="-host 0.0.0.0 -port 8080" | ||
| /bin/busybox insmod /qemu_fw_cfg.ko 2>/dev/null || true | ||
| FROM_FW_CFG=0 | ||
| /bin/busybox insmod /qemu_fw_cfg.ko || echo "[init] qemu_fw_cfg.ko did not load" | ||
| if [ -f /sys/firmware/qemu_fw_cfg/by_name/manager_args/raw ]; then | ||
| echo "[init] reading manager args from fw_cfg" | ||
| ARGS_LINE=$(/bin/busybox cat /sys/firmware/qemu_fw_cfg/by_name/manager_args/raw 2>/dev/null) | ||
| if [ -n "$ARGS_LINE" ]; then | ||
| MANAGER_ARGS="$ARGS_LINE" | ||
| FROM_FW_CFG=1 | ||
| fi | ||
| fi | ||
|
|
||
| # Fallback: kernel cmdline base64 (works without qemu_fw_cfg driver) | ||
| if [ -z "$(echo "$MANAGER_ARGS" | /bin/busybox grep -c 'fw_cfg')" ]; then | ||
| if [ "$FROM_FW_CFG" = 0 ]; then | ||
| for w in $(/bin/busybox cat /proc/cmdline); do | ||
| case "$w" in | ||
| manager_args_b64=*) | ||
|
|
@@ -72,5 +74,9 @@ echo "[init] eth0 carrier: $(cat /sys/class/net/eth0/carrier 2>/dev/null)" | |
| echo "[init] dns test: $(/bin/busybox nslookup nightly.link 10.0.2.3 2>&1 | head -3 | tr '\n' '|')" | ||
| /bin/busybox chmod +x /wrapper-manager | ||
| cd /data | ||
| # Not exec: the kernel panics when init exits, and a panicked guest keeps | ||
| # QEMU running. Powering off ends QEMU, so the host sees the manager stop. | ||
| # shellcheck disable=SC2086 | ||
| exec /wrapper-manager $MANAGER_ARGS | ||
| /wrapper-manager $MANAGER_ARGS | ||
| echo "[init] wrapper-manager exited with status $?, powering off" | ||
| /bin/busybox poweroff -f | ||
|
Comment on lines
+80
to
+82
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Failed manager exits look successful to host When Learn moreThe guest init process now remains alive while wrapper-manager runs. Regardless of the manager's exit status, it calls Example: If the manager exits with status 2 after startup, init logs status 2 and powers off. QEMU exits normally and Recommended fix: Preserve the manager's exit status and communicate failures across the QEMU boundary, such as through an explicit QEMU guest-to-host exit channel handled by the launcher; keep normal poweroff for successful exits. Was this helpful? React with 👍 or 👎 to provide feedback.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not changing this here. amdl, the host that pins this guest, treats any QEMU exit before /status answers as a failed boot regardless of exit status, so the clean poweroff is what it needs. Carrying the manager's status to wrapper-manager-qemu would need a guest-to-host channel (e.g. isa-debug-exit); worth it only if a script depends on that launcher's exit code. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -252,12 +252,39 @@ func describeInstall(res installResult) { | |
| } | ||
| } | ||
|
|
||
| // PrepareWrapper downloads and extracts the wrapper-lite native package when | ||
| // prepareRetryFirst and prepareRetryMax bound the wait between attempts to | ||
| // install the payload. | ||
| var ( | ||
| prepareRetryFirst = 5 * time.Second | ||
| prepareRetryMax = time.Minute | ||
| ) | ||
|
|
||
| // PrepareWrapper installs the wrapper-lite payload, retrying a failed | ||
| // download with backoff until it succeeds. Nothing can be served without it, | ||
| // and exiting is no better: in the QEMU guest the manager is what init runs, | ||
| // so its exit stops the guest, and a host that sees a guest stop at once may | ||
| // blame its accelerator for a network that is merely down. A download that | ||
| // fails its pinned digest is returned instead: the same pin would fail again. | ||
| func PrepareWrapper(mirror bool) error { | ||
| delay := prepareRetryFirst | ||
| for { | ||
| err := prepareWrapper(mirror) | ||
| if err == nil || errors.Is(err, errPayloadSHA256) { | ||
| return err | ||
| } | ||
| log.Errorf("wrapper-lite is not ready: %v; trying again in %s", err, delay) | ||
| time.Sleep(delay) | ||
|
Comment on lines
+271
to
+276
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔴 Invalid payload pin blocks startup indefinitely When a pinned download fails its digest check, Learn moreThe payload digest check in downloadWrapperLite returns Example: Start with Recommended fix: Distinguish terminal validation failures such as Was this helpful? React with 👍 or 👎 to provide feedback.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in a8eff91: a digest mismatch now returns from PrepareWrapper and main exits 1; other download failures still retry with backoff. Test: TestPrepareWrapperGivesUpOnADigestMismatch. |
||
| delay = min(2*delay, prepareRetryMax) | ||
| } | ||
| } | ||
|
|
||
| // prepareWrapper downloads and extracts the wrapper-lite native package when | ||
| // missing or installed from another source. mirror routes the download through | ||
| // gh-proxy.com. Failing to replace an installed payload keeps it. | ||
| func PrepareWrapper(mirror bool) { | ||
| // gh-proxy.com. Failing to replace an installed payload keeps it, unless the | ||
| // download failed its pinned digest. | ||
| func prepareWrapper(mirror bool) error { | ||
| if wrapperPayloadReady() { | ||
| return | ||
| return nil | ||
| } | ||
| installed := wrapperPayloadInstalled() | ||
| if installed { | ||
|
|
@@ -271,12 +298,13 @@ func PrepareWrapper(mirror bool) { | |
| if err != nil { | ||
| if installed && !errors.Is(err, errPayloadSHA256) { | ||
| log.Warnf("keeping the installed wrapper-lite: %v", err) | ||
| return | ||
| return nil | ||
| } | ||
| panic(err) | ||
| return err | ||
| } | ||
| log.Info("wrapper-lite ready") | ||
| describeInstall(res) | ||
| return nil | ||
| } | ||
|
|
||
| // UpdateWrapper force-reinstalls the latest wrapper-lite payload from | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔍 Parent build instructions need updating
The documented guest build command omits mandatory
KERNEL_ASSETS. Readers following it stop before building the initramfs; the guest-specific README has the revised steps.Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in a8eff91: manager/README.md now fetches qemu-assets and passes KERNEL_ASSETS.