Repository navigation
Cgroup attribution closes the escaped-child stowaway window (chaos scenario 3) - #112
Merged
Merged
Conversation
Scope(pid) parses the cgroup v2 unified line of /proc/<pid>/cgroup and returns the path of the nearest berth-*.scope ancestor, so a lease that owns its own scope can later name every process in it. Non-Linux platforms read nothing.
The gateway and berth run now share one systemd-run scope wrapper: ScopeArgv names the unit <prefix>-<name>-<rand>, and a run can ask whether a user manager is there at all.
A berth run that is not already inside a berth scope (the gateway scopes the runs it starts) and finds a systemd user manager execs its child through systemd-run --scope, unit berth-run-<as>-<rand>. The scope survives the process tree: a child that forks, leaves the group and reparents before any tick stays inside a cgroup the lease can name. Without a user manager the child runs as always, one debug line later.
A lease records the berth-*.scope of the engine attached to it (attach and re-attach). In owner, after the descendant record and before the container match, a GPU process whose scope equals a live lease scope is that lease's, and it joins the descendant set so reclaim and the stuck report name it exactly like a recorded descendant. An empty scope never matches an empty scope.
Chaos scenario 3 as fakes: the engine's child exits before any tick and the grandchild, already reparented, holds the card. In the scope the lease recorded, the grandchild is the lease's from its first sighting: the lease turns stuck naming it, reclaim lists it, and a whole-card requester never gets a contested grant. Outside every scope it is a stowaway as before, and the whole-card requester is granted contested. A process in one lease's scope is never counted under a neighbour's, and an empty scope matches no lease, scopeless or not.
…ager Scenario 3's expected result now says the window is closed on Linux with a user manager (the run's engine runs in its own scope, which every child inherits on fork) and stands without one, and on Windows. The capabilities bullet says the same, the roadmap bullet moves out of Next, and the changelog gets the line. The tested-hardware row keeps its live-run result until a run says otherwise.
… scope names no lease A berth run inside another lease's engine (llama-swap's per-model runs, when the gateway started llama-swap) inherited that lease's scope, so two live leases recorded one scope and owner() picked one at random. The run now always scopes its engine, and owner() ignores a scope more than one live lease recorded.
systemd-run joins its scope only after a round trip to the user manager. Measured on aphotic: /proc/<pid>/cgroup 1 ms after start still named the wrapper's cgroup 10/10 times, and the new scope 10/10 times at 50 ms. The attach-time read therefore never recorded the berth-run scope.
Owner
Author
|
Review fix pushed: the scope was read at attach, before systemd-run had joined it (measured: 0/10 in the new scope 1 ms after start, 10/10 at 50 ms), so a real lease never recorded its berth-run scope. The broker now reads each attached engine's scope on every tick. TestScopeReadAfterAttach fails without the change and passes with it. Trade-off to keep in mind: an engine started by a |
# Conflicts: # internal/runner/runner.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the one known stowaway window: chaos scenario 3, a child that forks and whose parent exits inside one 2 s tick. OME Q69Q3RR.
On Linux with a systemd user manager,
berth runstarts its engine throughsystemd-run --user --scope(unitberth-run-<name>-<rand>). The broker records that scope when the lease attaches, and a GPU process in it belongs to the lease even after it leaves the process tree. Reclaim and the stuck report name it like a recorded descendant. Without a user manager, and on Windows, nothing changes.internal/procs:Scope(pid)reads the cgroup v2 line and returns the nearestberth-*.scope.internal/sysd: the gateway's scope helpers, moved so the runner can share them.internal/runner: every run scopes its engine, including one that already sits in a scope (a gateway start, or llama-swap's per-modelberth runs), so two leases never share a scope.internal/broker: scope match after descendants and before containers. A scope that more than one live lease recorded names neither.Proof:
systemd-run --scopekeeps the PID: run from bash viaexec, the command's PPID equals the calling shell's parent, and its cgroup line ends in the newberth-run-*.scope. So the PID the lease records is the engine's own.TestSharedScopeNamesNeitherfails 5/5 without the uniqueness check and passes with it.go test ./..., Windows build and vet, golangci-lint (0 issues).Not done here: the live chaos scenario 3 run on aphotic, which decides the Tested-hardware row.