[bk-agent-01]

Problem

bkt3 production freezes for 50–95 s about every two minutes. During a freeze the desktop app shows "Connecting". I measured two causes on 2026-10-19:

1. Pull-request sweep. Every minute the server checks all 1,104 unsettled threads (200 branch groups) for a new pull request. Each check starts 6–7 git processes, also when the answer is in the cache. Each start forks the 1.4 GB server on the event loop (uv_spawn → copy_present_ptes).
   • perf on the main thread: 55% of samples in uv_spawn. The main thread was at 92% CPU.
   • Each sweep started 1,144–1,543 git processes at about 30 per second and ran for 50–95 s. Sweeps covered 57% of the time.
   • A 250 ms probe of /.well-known/t3/environment: median reply 504 ms during a sweep, 2 ms outside a sweep, and up to 4.3 s.
   • Desktop connects (client spans): 9–15 s during sweeps. Two of them hit the 15 s setup limit (09:38:17 and 09:39:15 UTC). Outside a sweep, a connect took 1.7 s.
2. Plan-flag reads. agentToolkit.correctPlanFlags (MCP t3_whoami and the session lists) reads the turn items of one run for up to 40 sessions. The filter run_id IN (...) OR (? = 1 AND run_id IS NULL) stops SQLite from using the (thread_id, run_id) index. SQLite then walks every item of the thread in ordinal order.
   • The server logged a 2.5 s event-loop stall at 10:32:12 UTC. The trace shows one such statement that took 2,586 ms. The same query read-only on the live database took 2,596 ms on that cold 30,580-item thread.

Fix

• Sweep (threadPullRequestSweep.expbkt3.ts, marked hooks in ThreadPullRequestService.ts): a thread with no run, no message and no live run for 24 h is checked on a rotation, once in 30 all-thread passes. The slot comes from a hash of the thread id, so the idle threads spread over the passes. Active threads are checked on every pass, as before. A thread's own events (run end, checkpoint, unsettle, metadata change) still queue an immediate one-thread check, and backfill is not rotated.
  • Expected on bkt3 today: about 90 active plus about 34 idle threads on each pass, not 1,104.
• Run filter (ProjectionStore.ts, marked): when the filter has no null run, the query uses run_id IN (...) with INDEXED BY orchestration_v2_projection_turn_items_thread_run_idx. The IN ... OR ... IS NULL form stays for filters that include null.
  • Measured read-only on the live database, 9 cold large threads each: current median 343 ms (max 2,066 ms), new median 4.3 ms (max 7.4 ms). Every new query shape prepares and shows SEARCH ... USING INDEX ..._thread_run_idx (thread_id=? AND run_id=?).

How I tested

• New tests in threadPullRequestSweep.expbkt3.test.ts:
  • The idle rule: a day with no run or message, and no live run.
  • Each idle thread is checked exactly once per rotation, and an active thread on every pass.
  • A service-level test over 30 passes: the idle thread's lookup runs once, the active thread's lookup runs 30 times.
• The existing TurnStartReads.test.ts covers the results of turnItemRunId and turnItemRunIds (including []) on SQLite.
• The tests run in CI only. This host does not run tests.
• node scripts/check-fork-markers.ts passes (with FORK_UPSTREAM_REF=upstream-ro/main).
• No UI changes, so there are no screenshots.
• After the merge to stage, I will run the same 250 ms probe and the perf stat -e sched:sched_process_fork count against stagebkt3. After a promotion to bkmain, I will run them against bkt3.

Not in this PR

• The Linear bridge reads the 3.2 MB shell about 52 times per hour and re-reads about 240 archived threads (404) after each shell read. That fix belongs in t3-linear-bridge-next (TEC-1502).
• A Linux spawn helper, so that short-lived git/gh starts no longer fork the server. This is a separate, bigger change (size M).

🤖 Generated with Claude Code — Claude Opus 5.5 in T3 Code.

[bk-agent-01]

Merged at head 5afbacbb on the go of tushar in T3 session "Debug BKT3 Connection Drops" ("quickly deploy it to stage and than bkmain"). All checks passed; Test Server 6 passed on re-run (flaky ACP test, unrelated). "Transfer report artifact" stays queued: its blacksmith runner takes no jobs in this repository. Codex is not connected here; a fresh-context agent review found no bugs.

[bk-agent-01]

Production check: this change went live through #326. Its check is in this #326 comment. Server git forks are now 191 to 320 per minute, against about 1,300 per minute before the fix.