fix(resident): bound drain waits by the fallback deadline - #2113
Conversation
There was a problem hiding this comment.
Changes requested: The fallback deadline resets across wake/drain hand-offs, and one stop path leaves the new wake span open.
Warning
Changes requested · head 4c420e7 · 2 findings: 1 major, 1 minor
| Severity | Finding | Where |
|---|---|---|
| major | F1 Preserve one fallback deadline across wake/drain hand-offs | src/execution/resident.ts:1771 |
| minor | F2 Close the wake span when the initial probe is stopped | src/execution/resident.ts:1701 |
Full review
F1: awaitWake starts a fresh awaitDrainEnd with the full drainBoundMs. If the first drain wait spends 2½ minutes, hands off through a transient refusal, and the wake re-attach encounters the drain again, the second drain wait receives another three minutes. Repeated deploy resets can keep renewing the bound toward the lease limit. Carry one absolute fallback deadline—or only the remaining allowance—across every drain/wake hand-off.
F2: When the run stops during the initial /status probe, probeStatus returns an unreachable result and the abort check throws before reaching the span-closing try/catch. The dispatch.workspace.attach.wake-wait span therefore never ends. Wrap the initial probe and abort check in the same failure/finalization path, with a regression test for stopping during that probe.
4c420e7 to
1ec9053
Compare
A run refused by the fleet drain waited at its attach for the whole deploy (issue #2101 defect 1: 18 minutes, DRAIN.waitMaxMs or the lease less the reserve), then did the job in a seeded sandbox that stands up in about two. The factory's first attach now passes drainBoundMs = DRAIN.fallbackWaitMs (3 min), so the wait is bounded by the fallback's own cost, never the deploy's; a resumed run's re-attach keeps the lease's bound (its worktree lives on the resident or nowhere). The wait is also visible (defect 3): the drain wait and the attach's wake wait are each one child span under the attach span (dispatch.workspace.attach.drain-wait / .wake-wait, display names in ATTACH_WAIT_NAMES), failing with the typed classification; the typed draining error carries the drain's share (drainWaitOf), the fallback selection carries it as ExecutorSelection.drainWaitMs, and the dispatcher publishes the drain_wait run note from either source, so runs friction counts the wait the incident counted as zero. Spec: resident-repos.md item 69 (prose + a new proof row), execution.md item 26. Co-Authored-By: coreplane-switchboard[bot] <318072483+coreplane-switchboard[bot]@users.noreply.github.com>
1ec9053 to
2e1891f
Compare
There was a problem hiding this comment.
LGTM: The rebase preserves the reviewed source patch and correctly combines both sides of the spec conflicts.
Note
Approved · head 2e1891f · no findings
Full review
The rebase preserves the reviewed implementation and both sides of the spec updates, with no new issues.
|
Live receipt 2026-09-21 (1.260.2 deploy, the first deploy with #2113 in the bot): review child |
A drained first attach now falls back to the seeded sandbox on one three-minute deadline, even across wake/drain hand-offs. Drain and wake waits remain visible through spans and a drain_wait run finding without new card or health vocabulary.
Why: Issue #2101 exposed an 18-minute drain hold before the same run used a two-minute seed. This keeps record 0059's drain semantics while bounding the first resident attach by its cheaper fallback.
Where to look
Feedback wanted: Please judge the absolute-deadline propagation across repeated drain/wake cycles and the span finalization for every initial-probe exit.
Risk: 444 changed lines across one resident attach path. Splitting visibility from the bound was rejected because friction derives from the same timing data; rollback is the single commit.
Verified: 145 resident tests, scoped TypeScript, Prettier, hygiene, spec integrity, and spec coverage passed. CI and live deploy timing remain human-gated.
Decisions (2)
Validation (7 criteria)
For agents
Review fixes F1 and F2 are folded into the original coherent commit. Branch was rebased onto origin/main at db6003c and force-pushed at 1ec9053. The two new regressions first failed on the reviewed head: deadline renewal timed out and the wake span had no end record. No generated files changed; no new plane health word or card string was introduced.
Requested by @justinhelmer in slack:C0BRRHKFLCB
🤖 Generated with Claude Code