Repository navigation
VFIO (experimental): berth vfio status and berth vfio run - #130
Merged
Merged
Conversation
…dupe nodes; status stays readable with IOMMU off
A script in its own process group that reads the terminal is stopped by SIGTTIN, so sudo's password prompt would hang (reproduced with a pty: the child sat in state T). The script now stays in berth's process group, and a Ctrl-C reaches it from the terminal. Only ExitCode stays exported from the runner. The docs no longer claim a real run, and the example script moves each function with driver_override, which the card's audio function needs.
# Conflicts: # CHANGELOG.md
berthd's probe is one long-running nvidia-smi -lms, not a call per tick, so skipping the read during a hold still left berthd holding the card open, and the driver unbind would block on berthd itself. Found by running berth vfio status on aphotic. Stream.Pause kills the loop and keeps it stopped until the hold ends; the broker calls it every tick. berth vfio run now refuses a probe still open after the settle time, from a berthd that does not pause. Kernel threads (a readable, empty cmdline) are skipped in the openers scan: on aphotic they were 400 of the 430 unreadable entries.
They passed on a machine with nvidia-smi and failed on CI without one.
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second half of VFIO passthrough support, after #129. Berth coordinates; the operator's own script switches drivers (with sudo) and runs the VM. Berth never binds a driver, writes to sysfs, asks for a password or stops a process.
berth vfio statusshows IOMMU on or off, the card's IOMMU group with each function's driver (anything besides the card's own functions blocks passthrough), the processes holding/dev/nvidia*or the card's DRM nodes, and what berthd reports. It changes nothing. It takes an address, becausenvidia-smicannot see a card bound tovfio-pci.berth vfio run [--reason NAME] -- CMD:BERTH_VFIO_DEVICESset;nvidia-smi.Found in review and fixed here:
nvidia-smi -lms, so VFIO (experimental): berthd shows a card passed through to a VM and stays off it during a switch #129's skipped reads still left the device open, and the unbind would block on berthd.Stream.Pausekills the loop for the whole hold, the broker calls it every tick, andvfio runrefuses a probe still open after the settle time.berth rundoes for engines. A process outside the terminal's foreground group that reads the terminal is stopped (SIGTTIN), so sudo's prompt would freeze. I reproduced it with a pty, where the child sat in state T. The script now runs in berth's own group.driver_override, which the card's audio function needs (it is not bound tonvidia). Checked withsh -nand shellcheck.Proof:
/procand a stand-innvidia-smi: each refusal, the happy path, the env var, the hold kept while the card is out, an unreadable process, kernel threads skipped.TestStreamPauseStopsTheLoop(a real child process: killed on pause, not restarted, restarted on resume).TestVFIOHoldPausesTheProbe.TestVfioRunRefusesALingeringProbe.berth vfio statuson aphotic lists exactly the four processes I had found by hand (llama-swap's and aether'snvidia-smiloops, Quickshell, berthd's probe).berth vfio runthere refuses because IOMMU is off.go test ./...,-raceon gpu, broker, cli and vfio, Windows build and vet, golangci-lint (0 issues).mkdocs build --strictpasses.