Repository navigation
Measure the whole launch frame, and run an oversized continuation cold - #2013
Merged
Merged
Conversation
backend/deploy/e2e/fake-codex is the static twin of FakeCodexCli, mounted into the real worker image for the deploy compose E2E. It still took the goal from the last argv argument after the goal moved to stdin, so the worker handed it "-" and it answered "DONE: -". The E2E never noticed, because it only waited for a Success status — which a CLI handed no prompt at all can still report. The fake reads stdin now, and run.sh asserts the folded summary from agent_run.result_jsonb is "DONE: Deploy E2E smoke task", so the lane checks that the prompt crossed the real images rather than that a process exited 0. The fake is a mirror of production behaviour with no drift detector, which is how it drifted. SubtaskAwareFakeCliDriftTests now runs the real file through the real CodexHarness with the goal on stdin and asserts the summary run.sh expects — red on the old fake, green on the new one.
The stdin preflight only measured the standard input, and its comment claimed the rest of the invocation frame always fits. It does not: a continue carries the restored session transcript in ConfigHomeFiles, captured up to 32 MiB, and the frame write enforced the 16 MiB bound only after transmission was marked started. That path deliberately skips the netns/cgroup teardown, because a failed write may be a lost ACK, and it surfaced as "Native invocation exceeds its pipe bound." — a generic executor error the node retries identically. Encode the frame first, refuse it while transmission has not started, then send the bytes already encoded. The refusal is the same terminal SandboxArgumentTooLongException an oversized argument gets, it names the two carriers that can grow that large without quoting either, the catch tears the isolation down, and the broker reads EOF and releases its slot as rejected rather than indeterminate. The stdin check stays as the early cut that costs nothing before a spool exists; its comment now says what it covers.
The whole-frame check refuses an invocation the pipe cannot carry as a terminal failure. That is right for a goal that will never fit and wrong for a retry whose restored transcript is the oversized part: the same work can go on in a fresh conversation, and before the check a node retry did exactly that one attempt later. Measuring the frame had turned a recoverable retry into a finished task. A restored transcript past its share of the frame is now dropped before launch, on both paths that bring one: a node retry or continue, where the bytes are first known just after the executor resolves the reference, and the executor's own revise loop, which stays warm only when the transcript fits. The degrade mirrors the unreadable-checkpoint one — the session id and every "resumed from" stamp go with the bytes, and the goal is told the conversation it was promised is not there. The frame check stays as the backstop, now for a goal no attempt can carry. The share is one number, NativeLaunchProtocol.LargeCarrierBudgetBytes, used by the stdin preflight too. Also: run.sh reads the succeeded attempt's summary, since a run won on a retry has one row per attempt, and the frame refusal labels its counts and its config-home files as what they are.
The cold degrade dropped a restored transcript whenever it alone encoded past half the launch frame. That share had nothing to do with the room the frame actually had left: a revise goal is a short delta, so nearly all 16 MiB is the transcript's, and on main continuations with an 8-16 MiB transcript resumed warm - after the half-frame rule they silently lost their conversation. The opposite case was refused outright: a goal and a transcript each under half, together over the whole, failed terminally although a cold attempt fits. The executor now builds the spec, and only when it carries a restored transcript and does not fit the frame (NativeLaunchProtocol.FitsTheFrame, with a fixed allowance for what wraps a spec) does it rebuild cold - on the launch path and for a revise round, which keeps any model escalation already applied. The stdin preflight keeps its half share. A cold attempt now leaves a durable trace. The persisted envelope drops its continuity claims, so the Room no longer marks a cold attempt as "resumed", and the timeline says it ran cold; the persisted goal is left alone because the contract hash covers it. Both call sites were tested by nothing - removing either left every suite green. Executor-level tests now drive each through the real runner's frame check with the real Claude adapter's spec, and go red when the degrade is disabled.
A respawn can be checked out at the branch its earlier attempt pushed, with no other sentence saying that work is there. "You are starting this task from the beginning" had the agent redo or overwrite its own half-finished edits. The hint now says the conversation is gone and that whatever the workspace already holds beyond the base is its own earlier work - true on every path, including a fresh clone.
FitsTheFrame measured the spec and held a flat 1 MiB for everything else. The invocation carries the spec and, a second time, the child's argv - which appends the spec's arguments, a persona on --append-system-prompt included - and its environment, which copies the spec's. With a large escaped persona that second copy approached the allowance on a 4 KiB-page host and passed it where the argv ceiling is higher, so a spec the fit admitted was a frame the runner refused. With a plain persona the allowance gave away ~1 MiB, so continuations whose frame fit ran cold for nothing. Both copies are counted now, with 64 KiB for the parts that do not grow with the spec. Pinned against the invocation the runner really encodes, at the boundary, for an ASCII and a worst-case persona.
The cold degrade put its hint into the run's own goal. Local acceptance hashes that goal against the persisted envelope before launch, so a locally graded continuation read as a different contract and failed as local-context-mismatch, a grader fault, before any process started - the refusal the degrade exists to replace. The unreadable-checkpoint degrade on main amends the goal the same way and failed a locally graded retry identically. The hint now rides only the dispatched spec, and verification is graded on the goal the envelope persisted, which fixes both. The cold trace is recorded once the attempt is sure to launch, and says what happened per path: only a revise round keeps its workspace, so only its note says so. The launch test now records the real argv, so its "no --resume" assertion can fail.
The cold note and the cleared envelope were written after local acceptance but before spend admission. A continuation refused for spend - a lost host whose dead attempt still holds the run's cap - never started a process, yet its timeline said it had started a fresh conversation, and its envelope no longer said it was a resume. Both now follow admission, on the launch path and for a revise round.
Moving the revise round's cold note below its spend admission had no test: putting it back above admission left every suite green. A cold round refused for spend under a capped quick lane now asserts no "continued as a fresh conversation" note, and goes red with the note moved back.
9 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes three gaps in how an agent launch carries large text after the goal moved to stdin (#2008). Each fix is its own commit.
backend/deploy/e2e/fake-codexis the static twin ofFakeCodexCli, mounted into the real worker image. After the move to stdin the worker handed it-, it answeredDONE: -, and the lane stayed green becauserun.shonly waited forSuccess. The fake reads stdin now, andrun.shasserts that the succeeded attempt's folded summary inagent_run.result_jsonbisDONE: Deploy E2E smoke task.ConfigHomeFiles(captured up to 32 MiB).WriteFrameAsyncenforced the 16 MiB bound only aftertransmissionStarted = true, which deliberately skips the netns/cgroup teardown, so the failure surfaced as a generic, retriedexecutor-error. The frame is now encoded first and refused before transmission as the terminalSandboxArgumentTooLongException. The catch tears the isolation down, and the broker releases its slot asrejected.NativeLaunchProtocol.FitsTheFrame). This covers the launch path and each revise round, and a model escalation already applied to the round stands.--append-system-promptis in both copies), plus a 64 KiB allowance for the rest. It is pinned against the invocation the runner really encodes, at the boundary.local-context-mismatchbefore it launched. The same fix covers a bug already on main: the unreadable-checkpoint degrade amends the goal the same way and failed a locally graded retry identically.Test plan
SubtaskAwareFakeCliDriftTestsruns the realbackend/deploy/e2e/fake-codexthrough the realCodexHarnessinvocation, with the goal on stdin, and asserts the summaryrun.shexpects. It was red on the old fake.sandbox_argument_too_long. The message names both carriers' sizes and never their contents, and the broker receipt settles onrejected, notindeterminate. It was red first (InvalidDataExceptionfrom the late write).AgentRunExecutorTests). Setup: the real executor, runner and frame check, and the real Claude adapter's spec (recorded with its real argv) with only the executable swapped. A transcript of 16 MiB + 1, restored by artifact reference, runs cold and succeeds. The warm spec carried the transcript and--resume(fixture checks); the launched one has neither. Stdin ends with the hint, the persisted envelope has no resume, and the timeline note is present. Red with the degrade disabled.AgentRunReviseLoopFlowTests). Round 0 writes a session file past the whole frame at the path Claude keeps it, and the real capture reads it. Round 1 is rebuilt cold with the whole contract and passes. Red with the degrade disabled.NativeLaunchFrameFitTests): with a 130,000-character persona, ASCII and<(six bytes each, in both copies), a spec exactly at the fit boundary encodes to a frame within 16 MiB, and within two allowances of it. Red when the second copy of argv and environment is not counted.deploy-e2e.yml): green on 639bf2f (the frame and cold-degrade code it exercises is unchanged since), including the newrun.shsummary assertion on a real Linux worker.Notes
LargeCarrierBudgetBytes). It is the early cut for a goal no attempt can carry, taken before a spool exists.agent_run.resumed_from_agent_run_idcolumn still records that the attempt was created to resume its predecessor. That is creation provenance and remains true; the Room's mark reads the envelope.