test: whole suite on the Rust supervisor with pyaleph 0.11.0, aleph-cli 0.17.0, scheduler 0.2.0, aleph-vm #1168

[odesenfans]

Why

Only the confidential TEE server ran ALEPH_VM_SUPERVISOR_IMPL=rust; the DigitalOcean CRNs took the launcher's python default, so every non-confidential test on this testnet (instances, programs, migration, backups, port forwards, IPv6) still validated the Python supervisor. crn-up.sh now writes ALEPH_VM_SUPERVISOR_IMPL (default rust, overridable from the environment) into supervisor.env on every CRN.

Pins

• pyaleph 0.11.0 (+ Kubo v0.43.0 as the release ships it)
• aleph-cli 0.17.0
• scheduler-rs / scheduler-api 0.2.0 (V-PROGRAM scheduling, aleph-vm-scheduler#193)
• aleph-vm from od/drop-attest-cli (aleph-vm#1168 deb). aleph-vm#1169 is stacked on it and targets that branch, so the deb workflow never ran for it; its only change is the nix guest kernel, which is not in the deb (guest images come from the pinned fixtures in scripts/vprogram-artifacts.sh).

What the run covers

• DO CRN 0/1 on the Rust supervisor + Rust controller: full regular suite.
• Static TEE server on Rust: test_confidential (SEV instance), test_vprograms (SEV-SNP V-PROGRAM exec + attested call), test_vprogram_compose (SEV-SNP aleph.compose/1).

Not covered here: test_vm_upgrade only runs from upgrade-check.yml (push to od/upgrade-check-1.15-to-dev), and the V-PROGRAM fixtures are still the 2026-08-20 build (aleph-vm ba690c65), so the #1168 init/agent changes and the #1169 kernel are not in the measured guest yet.

[foxpatch-aleph]

Well-structured PR that switches all CRNs to the Rust supervisor, adds V-PROGRAM end-to-end tests, and introduces an in-place upgrade test harness. The code is thorough with excellent diagnostic logging and well-documented race condition mitigations. The main concern is a file-read-during-write bug in the TEE network diagnostics step, plus a few minor observations.

.github/workflows/pr-tests.yml (line 403): The uplink and tap variables are obtained by grepping tee-network.txt from within the same { ... } > tee-network.txt 2>&1 block that is actively writing to that file. Since the shell opens the file for writing (truncation) before the block executes, and stdio output to the file may not be flushed by the time grep reads it, these variables will likely be empty, silently skipping the dual-capture diagnostic. Consider writing the static diagnostic output to a temporary variable or file first, then grep from that, before redirecting everything into tee-network.txt.

.github/workflows/pr-tests.yml (line 424): The DNAT-rule probing loop at the end (grep -E \"dnat.*:8443\" tee-network.txt | while ...) reads tee-network.txt after the { } block has closed, so the file is fully written at that point. However, the while read loop appends to tee-network.txt with >>, which is fine. One concern: if the earlier IPv6 section's { } block has already closed, the file content is complete, but the DNAT probe results are appended after the closing done of the outer block — this works but the appended content won't be visible if the file is later read by the upload step (it will be, since >> flushes). This is OK.

scripts/crn-up.sh (line 80): With the manifesto now using branch: instead of version: for aleph-vm, read_vm_version() will throw a KeyError if ever called without ALEPH_VM_VERSION set. Currently this path is unreachable because read_vm_branch() returns the manifesto branch first, but it's a latent fragility. Consider using m['components']['aleph-vm'].get('version') or adding a guard that errors with a clear message when neither version: nor ALEPH_VM_VERSION is available.

tests/test_vm_upgrade.py (line 150): The vm_hash is interpolated directly into a pgrep -f pattern: f\"pgrep -f '[q]emu-system-x86_64.*{vm_hash}' || true\". While vm_hash comes from the aleph CLI's JSON output (a hex hash), this pattern is injectable if the hash were ever malformed. Consider quoting or validating the hash format before interpolating it into remote shell commands.

tests/test_vm_upgrade.py (line 76): The _crn_run default timeout is 60 seconds. Some commands invoked through this helper (e.g., journalctl with large log volumes, or apt-get operations) may exceed 60s. The callers that need more time should pass an explicit timeout= — most do, but a few (like _supervisor_exe at line 126 calling readlink -f /proc/{pid}/exe) use the default, which is fine for that case. Just something to be aware of as new callers are added.

tests/test_vprograms.py (line 162): Neither test_vprogram_deploy_and_attested_call nor test_vprogram_compose_deploy_and_attested_call clean up (delete) the V-PROGRAM after the test. This is fine for CI (fresh testnet each run + tee-reset.sh wipes VM state), but adding a try/finally with aleph_cli(\"vprogram\", \"delete\", item_hash, ...) would be good hygiene for local dev runs and matches the pattern used in test_vm_upgrade.py.

tests/test_vm_upgrade.py (line 408): The confidential fixture has scope=\"module\" but catches pytest.skip.Exception. If the underlying fixtures (e.g., confidential_crn_host) are scope=\"session\", requesting them from a module-scoped fixture is fine, but the skip exception class check (pytest.skip.Exception) is an internal pytest API that may change between versions. Consider using pytest.skip.Exception via except Exception with a check for isinstance(e, pytest.skip.Exception) or simply catching pytest.skip.Exception — which is what's done here. This works but is fragile across pytest versions.

[foxpatch-aleph]

Well-structured PR that pins new component versions, forces the Rust supervisor on all CRNs, adds V-PROGRAM and in-place upgrade test suites, and fixes the Kubo config key for v0.43.0. The code is generally correct with thorough comments explaining non-obvious decisions (scheduler race conditions, IPv6 egress filtering, fixture provenance). The main observations are: a hardcoded IPv6 pool in the workflow (acceptable for testnet but fragile if infrastructure changes), docker pull/save in test_vprogram_compose.py using check=True which produces a raw traceback instead of a pytest failure message, and the very long inline TEE diagnostics block in pr-tests.yml that could be a script file for maintainability. None of these are blocking.

.github/workflows/pr-tests.yml (line 277): STATIC_CRN_IPV6_POOL=2001:bc8:702:32d::/64 is hardcoded. If the Scaleway flexible-IP /64 ever changes, this will silently break V-PROGRAM IPv6 reachability tests with no obvious error pointing here. Consider sourcing it from a secret or documenting the dependency more visibly.

tests/test_vprogram_compose.py (line 58): docker pull/save use check=True, which raises CalledProcessError on failure (a raw traceback) instead of pytest.fail (a clean message). The test still fails, but the diagnostics are worse. Consider check=False + an assert with the stderr tail, matching the pattern used for aleph_cli calls in the same test.

.github/workflows/pr-tests.yml (line 335): The TEE network diagnostics step is ~100 lines of inline bash. Consider extracting it to scripts/tee-diagnostics.sh for readability and reuse — the upgrade-check.yml workflow would benefit from the same diagnostics.

tests/test_vm_upgrade.py (line 154): The nested python3 -c command with escaped double quotes inside the SSH command string is correct but fragile to modify. A comment with an example of the expanded command would help future maintainers.

tests/test_vprogram_compose.py (line 108): The retry loop references root after the loop exits (line 131 asserts on root.stdout). If the loop were to exit without the break (e.g., due to an unexpected exception path), root would be the last assigned value, which might be a non-successful result. The current logic is correct since the only break is on success, but the coupling between loop control and the variable is implicit.

[foxpatch-aleph]

Well-structured PR that switches the full test suite to the Rust supervisor, pins pyaleph 0.11.0 / aleph-cli 0.17.0 / scheduler 0.2.0, adds V-PROGRAM and V-PROGRAM compose end-to-end tests, and introduces a thorough in-place upgrade validation workflow. The test coverage is excellent (controller snapshots, QEMU PID tracking, rootfs inode checks, adoption error detection, supervisor impl swap tests). Comments are detailed and explain non-obvious decisions well. The concerns noted (unsanitized variables in remote shell commands, deb cache reuse, missing V-PROGRAM cleanup) are minor and non-blocking — the controlled CI environment and existing cleanup mechanisms (tee-reset.sh) mitigate them in practice.

tests/test_vm_upgrade.py (line 194): The pattern and since_ts are interpolated into a remote shell grep -iE command. since_ts comes from a controlled date format so this is safe today, but consider using printf '%q' or adding a comment noting the format constraint, to prevent future format changes from silently breaking the remote command.

scripts/crn-up.sh (line 793): The cached upgrade deb (aleph-vm-upgrade.${deb_variant}.deb) persists across calls. If --upgrade is invoked twice with different targets (branch A then branch B), the second call reuses the stale cached deb from the first because the filename doesn't encode the branch/version. Consider removing the cached file at the end of upgrade_crn or including the branch in the filename.

tests/test_vprogram_compose.py (line 51): The V-PROGRAM testnet-whoami is never deleted after the test. While tee-reset.sh wipes execution state, the on-chain message persists. If the test is re-run against the same testnet, vprogram create testnet-whoami may fail on the duplicate name. Consider adding a try/finally with vprogram delete or using a unique name.

tests/test_vprograms.py (line 36): Same as the compose test — the V-PROGRAM testnet-fib is never deleted. Consider adding cleanup in a try/finally block.

.github/workflows/pr-tests.yml (line 403): The diagnostic step greps tee-network.txt while the enclosing redirect is still writing to it. This works because earlier output is flushed sequentially, but it is fragile — if output ordering changes, the grep patterns may miss. A comment noting this dependency on sequential flush would help future maintainers.

deploy/001-update-ipfs-config.sh (line 6): The config key rename from Reprovider.Strategy to Provide.Strategy has no version guard. If this script ever runs against a pre-0.42 Kubo, it will fail. Since the manifesto pins v0.43.0 this is safe, but a comment noting the minimum Kubo version would be helpful.