Skip to content

Fix cross-thread Codex identity binding and rollout writer handoffs - #3460

Open
Willhong wants to merge 5 commits into
get-bb:mainfrom
Willhong:fix/codex-thread-identity-writer-lock
Open

Fix cross-thread Codex identity binding and rollout writer handoffs#3460
Willhong wants to merge 5 commits into
get-bb:mainfrom
Willhong:fix/codex-thread-identity-writer-lock

Conversation

@Willhong

@Willhong Willhong commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Human comments

What was wrong

Two bb threads ended up bound to one Codex rollout, so the second thread's resume was refused with thread <id> already has an active writer (#2327). The binding went wrong inside the host daemon, not in Codex: the Codex bridge announces a thread's identity twice, once through the direct thread/identity notification (stamped with the bb thread id) and once through the thread.identity delta that the assembler emits for Codex's thread/started (stamped with an empty thread id). emitTranslatedEvents ignored the notification's source thread for the second event and popped a FIFO of "pending identity" slots instead. Those slots were never consumed when the identity arrived through the first path, and a staged rewind's slot was never consumed because staging events are suppressed. After one message edit, the next thread/start on the same bridge recorded its Codex thread id onto the rewound thread. Every later event of that thread was stamped with the foreign id, the server resumed it into the other thread's rollout, and the rollout's owner then failed with the active-writer error.

Investigation on a live machine: thr_fh9dq7fvsu (edited at 08:00, provider thread 01a08d8d) started stamping 01a08ddd two seconds after thr_r6xinwrds3 started with 01a08ddd on the same environment bridge; the rollout of 01a08ddd then received a turn from thr_fh9dq7fvsu at 10:12 and thr_r6xinwrds3 was refused at 10:19.

What changed

  • packages/agent-runtime/src/runtime.ts: a thread/identity event is bound to the notification's source thread first, then the event's own thread id, and only then a pending slot. An identity that would bind a provider thread another hosted thread already owns is dropped with a stderr line instead of being recorded. resumeThread refuses, before touching the bridge, to resume a provider thread that another hosted thread owns, so the daemon no longer collides on Codex's writer lock in that case.
  • packages/agent-runtime/src/runtime-thread-identity.ts: recording a thread's identity consumes that thread's pending slot, so slots no longer accumulate.
  • plugins/provider-codex/src/bridge: carries the three commits of Fix Codex rollout writer handoff races #2578 (author preserved) rebased onto current main. The writer-handoff serialization those commits added to constructThreadSession and releaseSession already landed in Fix Codex archive undo synchronization #3025, so the bridge keeps main's implementation; what this brings over is the bounded active-writer retry ladder with stderr diagnostics, the actionable guidance on a persistent writer, the writer.lock fixture in fake-codex-app-server.mjs, bridge-process.test-support.ts, and the writer-lock regressions.
  • fake-codex-app-server.mjs: the scripted SIGTERM delay now applies only to children that served a thread/start, thread/resume, or thread/fork. Since Fix Codex archive undo synchronization #3025 the bridge awaits a maintenance child's exit before settling a maintenance request, and a fake that delayed every child let an in-flight rebuild finish before a concurrent discard could mark the replacement closing. A real maintenance child holds no writer and exits promptly.

No host-daemon wire contract changed, so HOST_DAEMON_PROTOCOL_VERSION is unchanged. #2328 (recovery storms on a dead session) is not addressed here. The server-side gap that let the corrupted stamp become the resume target is tracked in #3461.

How you verified

New regressions, proven red before the fix:

  • runtime.codex-topology.test.ts "keeps a rewound thread's provider identity when another thread starts on the same bridge": with the old runtime, t1's provider thread id became t2's (expected 'codex-fx-…-1' to be 'codex-fx-…-fork-1').
  • runtime.codex-topology.test.ts "refuses to resume a provider thread that another hosted thread already owns".
  • runtime-thread-identity.test.ts "consumes a thread's pending identity slot when its identity is recorded".
  • bridge.writer-lock.test.ts from Fix Codex rollout writer handoff races #2578 (8 tests), including the concurrent-discard case that fails against the unrestricted fake delay.

Commands run on this branch (based on current main):

  • pnpm exec turbo run test --filter=bb-plugin-provider-codex --filter=@bb/agent-runtime --filter=@bb/host-daemon --force (28/274, 22/321, 46/572 passing)
  • pnpm exec turbo run typecheck --filter=bb-plugin-provider-codex --filter=@bb/agent-runtime --filter=@bb/host-daemon
  • pnpm exec turbo run lint --filter=bb-plugin-provider-codex --filter=@bb/agent-runtime
  • pnpm exec oxfmt --check plugins/provider-codex/src/bridge packages/agent-runtime/src (the only report is the pre-existing bridge.resume-hydration.test.ts, untouched here)

Fixes #2327
Supersedes #2578

AGENT GENERATED

Willhong and others added 5 commits September 11, 2026 11:08
…thread

Delta-assembled thread/identity events carry an unstamped thread id, so the
runtime ignored the notification's source thread and popped a FIFO of pending
identity slots instead. Those slots were never consumed when the identity
arrived through the direct bridge notification, and a staged rewind's slot was
never consumed because staging events are suppressed. After one rewind, the
next thread/start on the same bridge recorded its Codex thread id onto the
rewound thread; both bb threads then resumed one rollout and the second resume
failed with "already has an active writer".

Resolve the identity target from the source thread first, consume a thread's
pending slot whenever its identity is recorded, refuse to bind a provider thread
that another hosted thread already owns, and refuse to resume a provider thread
that another hosted thread already owns before the bridge touches the rollout.
…-serving children

Since get-bb#3025 the bridge waits for a maintenance child to exit before it settles
a maintenance request, so a fake that delays every child on SIGTERM let an
in-flight rebuild finish before a concurrent discard could mark the replacement
closing. A real maintenance child holds no rollout writer and exits promptly;
apply the scripted delay only to children that started, resumed, or forked a
thread.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

turn/completed can precede command completion and wedge later turns between active-writer and no-session errors

2 participants