[Bug]: Account pool routing destroys prompt cache and triggers 10x-50x token burn death-spiral above 80% usage threshold

[Bug]: Account pool routing destroys prompt cache and triggers 10x–50x token burn death-spiral above 80% usage threshold

Repository: lidge-jun/opencodex (Package: @bitkyc08/opencodex)
Labels: bug, critical, routing, performance, cost-anomaly

1. Summary

When multiple accounts are registered in OpenCodex, the current routing algorithm in src/codex/routing.ts creates two major compounding failure modes under heavy autonomous / agentic workloads:

Subagent Workload Cascading: When a large volume of one-off, single-turn subagent tasks (e.g. IDE background checks or repo inspections, ~150k tokens each with 0% cache) hits the proxy, OpenCodex routes them sequentially to fresh accounts as each hits the autoSwitchThreshold = 80% (default: config.autoSwitchThreshold ?? 80). This drains every account in the pool sequentially.
The 80% Multi-Turn Ping-Pong Death Spiral: Once accounts reach autoSwitchThreshold, reevaluateAffinityQuota() bypasses the 60-second session affinity lock on every turn and calls pickLowerUsageAccount(). Active, long-running multi-turn conversations (which previously enjoyed a 95%+ cache hit rate) are suddenly bounced between Account 1 <-> Account 2 <-> Account 3 on literally every turn. Because OpenAI Codex prompt caches (GPT-6 Astra / GPT-5.6 Luna / Sol) are strictly tenant/account-isolated, this reduces the multi-turn cache hit rate to 0%, turning a 7k-token turn into a 150k-token raw billing catastrophe.

Real-world telemetry from our audit: Over 1.90 Billion tokens (1,902,323,250 tokens across 15,607 requests) were burned in ~13 hours across 5 Pro accounts.

2. Environment & Version

OpenCodex Version: @bitkyc08/opencodex (latest)
Upstream Provider: OpenAI Codex (chatgpt.com/backend-api/codex / responses API)
Models: gpt-6-astra, gpt-5.6-luna, gpt-5.6-sol, gpt-5.6-terra
Account Configuration: Multi-account pool (5 Pro subscriptions)
Client: Orca / Codex CLI connected to http://127.0.0.1:10100

3. Root Cause Analysis (Code & Empirical Telemetry)

3.1 Phase 1: How Account 2 & 3 Reached 80% (Subagent Workload Cascading)

Our detailed investigation of usage.jsonl revealed:

Account 2 (orca-d***0) processed 741 total conversations.
732 of these were one-off single-turn subagent calls dispatched during autonomous workflows.
Each single-turn call submitted ~148k tokens of repository context with 0% to 4% cache hit rate, consuming 104,188,859 uncached raw tokens in just 2 hours (16:00–18:00 KST).
In contrast, the 9 long-running multi-turn conversations transferred to Account 2 only consumed 14.8M uncached tokens (maintaining a 95.6% cache hit rate after turn 1).
OpenCodex Flaw: Rather than throttling or warning that the workload was an un-cacheable runaway subagent storm, OpenCodex blindly cascaded the storm: Account 1 reached 80% -> poured into Account 2 (104M uncached) -> Account 2 reached 80% -> poured into Account 3 (108M uncached) -> Account 3 reached 80%.

3.2 Phase 2: The 80% Multi-Turn Ping-Pong Death Spiral (`src/codex/routing.ts`)

// Conceptual bug pattern in routing logic (src/codex/routing.ts: reevaluateAffinityQuota)
const threshold = config.autoSwitchThreshold ?? 80;

if (usage >= threshold) {
    // BUG: Discards the 60s session affinity re-eval interval entirely!
    // ("// Over-threshold pins re-eval immediately so a depleted primary does not keep serving for up to 60s")
    // Instead of sticking with the account until a hard 429 is reached,
    // it selects whichever account has 1% lower usage on every single request.
    selectedAccount = pickLowerUsageAccount(config, entry.accountId, usage, now);
}

Once accounts 1, 2, and 3 were all >= 80%, the 60-second affinity lock was permanently bypassed on every request.
The main multi-turn sessions (e.g. b36bfd009... with 2,788 turns) were subjected to extreme ping-pong:
- Turn N (Account 1): 150k context
- Turn N+1 (Account 2): 150k context (0% cache on Account 2!)
- Turn N+2 (Account 3): 150k context (0% cache on Account 3!)
- Turn N+3 (Account 1): 150k context (Cache evicted or mismatched!)
The multi-turn sessions, which should have cost ~7k tokens/turn, suddenly cost 150k raw tokens/turn, multiplying token burn by 20x–25x until every account returned HTTP 429.

4. Production Telemetry & Empirical Evidence

Data collected from full log audits of usage.jsonl (15,607 requests) and local Orca transaction databases (logs_2.sqlite):

4.1 Telemetry by Account (Masked)

Account Hash	Account Identifier	Quota Reset Window	Requests	Tokens Burned	Uncached Tokens	Primary Workload Pattern
`p***e`	`orca-3*7` (`h*d@gmail.com`)	Sep 12 18:00 KST	8,696	956,521,557	64,870,299	Primary session host (93.2% cache) until 80% threshold
`p***f`	`orca-d*0` (`s*t@agentryx-ai.com`)	Sep 13 15:48 KST	3,317	462,569,476	119,039,343	Hit by 732 subagent calls (104M uncached) -> 80%
`p***a`	`orca-b*e` (`w*g@gmail.com`)	Sep 13 18:20 KST	2,644	382,378,090	112,431,200	Hit by 773 subagent calls (108M uncached) -> 80%
`p***d`	`orca-6*d` (`r*2@gmail.com`)	Sep 12 17:14 KST	650	91,859,644	18,240,110	Swept into 4-way ping-pong death spiral
`p***7`	`orca-f*e` (`m*y@gmail.com`)	Sep 13 09:09 KST	4	0	0	Failed immediately with HTTP 429 (pre-exhausted)
Unassigned	Direct / Pre-routing	-	296	9,004,482	8,912,450	Initial direct requests / zero-attempt drops
Total	5 Accounts + Direct	-	15,607	1,902,333,249	323,493,402	Entire pool drained within ~13 hours

4.2 Cache Hit Rate Dynamics

Single-Account Stable Sessions: ~95.6% – 96.0% cache hit rate (~7,800 uncached tokens/turn).
Subagent One-Off Requests: ~4.0% cache hit rate (~142,000 uncached tokens/turn).
Cross-Account Ping-Pong (Above 80%): 0.0% handover cache hit rate (~150,000 uncached tokens/turn).

5. Proposed Remediation

1. Mandatory Session Sticky Routing

Active multi-turn sessions MUST NEVER switch accounts mid-conversation, regardless of whether the account is at 80%, 90%, or 95%.
An account should only be abandoned for a session if OpenAI explicitly returns HTTP 429: The usage limit has been reached.

2. Subagent / One-Off Task Circuit Breaker

OpenCodex should recognize high-volume, uncached single-turn bursts and avoid cascading them to fresh accounts automatically without user consent.
Allow users to designate a specific "worker/agent account" rather than letting subagent bursts wipe out the entire pool.

3. Sequential Account Draining Instead of Load-Balancing

For LLM caching architectures, load-balancing across accounts above 80% is an anti-pattern.
Implement Sequential Drain: Dedicate 100% of traffic to Account 1 until 429 is reached, then switch to Account 2.

리뷰 · 우선순위 78 / 80

이 이슈는 계정 풀이 autoSwitchThreshold(기본 80%)를 넘기면 다른 계정으로 옮겨 가는데, 그 순간 이미 따뜻해진 프롬프트 캐시를 버리고 새 계정에서 처음부터 토큰을 다시 태운다는 비용 사고 보고입니다. 한 번에 끝나는 서브에이전트성 큰 작업이 계정을 차례로 소진하고, 그 위에 멀티턴 대화까지 임계치를 넘기면 캐시 미스가 겹쳐 체감 토큰이 10~50배로 뛴다는 설명입니다. 라벨에 bug/account-pool이 붙어 있고, 본문이 src/codex/routing.ts를 지목합니다.

현재 dev의 src/codex/routing.ts를 보면 이 긴장이 코드 주석에도 이미 적혀 있습니다. 스레드 어피니티는 프롬프트 캐시 최적화이고, autoSwitchThreshold는 “바빠지기 시작” 힌트이며, 임계치를 넘는 순간 옮기면 그 대화의 캐시를 버린다는 설명이 있습니다. 즉 “버그가 전혀 문서화되지 않은 구멍”이라기보다, 쿼타 보호와 캐시 유지 중 무엇을 우선할지가 제품 판단으로 남아 있는 상태입니다. src/types/config.ts에도 풀·임계치·라이브 세션 이동에 대한 설정 서술이 있습니다.

보고자가 말하는 death-spiral은 자율/에이전틱 부하에서 특히 아픕니다. 서브에이전트가 캐시 0%인 큰 원샷을 연달아 보내면 계정마다 80%까지 채운 뒤 다음 계정으로 넘어가고, 본 대화 스레드까지 같이 옮겨 가면 캐시가 통째로 리셋됩니다. 사용자는 “풀이 똑똑해서 막아 준다”고 기대했는데 실제로는 “풀이 캐시를 부순다”로 느낍니다. 비용 이상이라 우선순위는 매우 높습니다.

다만 수정 방향은 한 가지가 아닙니다. (1) 임계치 도달 시에도 같은 스레드면 sticky 유지, (2) 원샷/서브에이전트만 다른 계정으로 돌리고 멀티턴은 고정, (3) 임계치를 높이거나 기본값을 바꾸기, (4) UI/문서에 “옮기면 캐시 버린다”를 분명히 쓰기. 코드에 이미 힌트가 있으므로, “주석만 있고 동작이 기대와 다르다”인지 “동작은 의도인데 기본값이 위험하다”인지를 먼저 갈라야 합니다.

types/config 분할 캠페인과 직접 충돌하지는 않습니다. 설정 키가 types/config.ts에 있는 만큼, 동작 변경 PR이 그 파일을 만지면 분할 PR과 겹칠 수 있으니 착수 전에 열린 설정 분할 PR이 있는지만 보면 됩니다. 미리보기 배포는 계획에 없습니다.

경로 src/codex/routing.ts (autoSwitchThreshold / sticky / cacheAffinity) - 보고와 같은 트레이드오프가 주석·로직에 이미 있다. “미구현”이 아니라 “우선순위 선택” 이슈에 가깝다.
경로 src/types/config.ts (autoSwitchThreshold 설명) - 설정 의미와 실제 라이브 세션 이동 동작이 사용자 기대와 맞는지 검토 지점이다.
심볼 기본값 80 - 에이전틱 부하에서 너무 공격적인지 기본값/문서 재검토가 필요하다.
재현 조건 - 서브에이전트 원샷 연쇄 + 멀티턴 본스레드가 같은 풀 정책을 쓰는지 분리해서 재현하면 수정 범위가 작아진다.

메인테이너의 판단이 필요한 지점

쿼타 보호 vs 프롬프트 캐시 유지 중 기본 우선순위를 무엇으로 둘지.
멀티턴 sticky와 원샷 부하를 설정으로 나눌지, 코드 기본 동작만 바꿀지.
critical 비용 이슈로 2.54.0 직후 핫픽스 후보에 넣을지, 설계 이슈로 남겨 다음 마이너에 넣을지.

너의 추천

수용(유효한 제품 버그/트레이드오프). 먼저 sticky-on-threshold 옵션 또는 멀티턴 비이동 기본값을 짧게 설계한 뒤 PR을 받으세요. 임계치만 숫자로 올리는 패치는 근본 해결이 아닙니다.

이 댓글은 grok-bot이 작성했습니다

test(gui): update pool copy assertions to the cache-first wording (#4546

fix(codex): keep a bound thread's prompt cache and tell configured ro…

Partly landed on dev by 1de9184. Leaving this open, because closing it against that merge would overstate what shipped.

What landed: src/codex/routing.ts routes live quota rebinds through a cache-safe replacement check at both the affinity re-evaluation and the reusable-account preview, so a bound thread can only move to an account that has real quota headroom and is also strictly cooler. That removes the all-hot ping-pong — the 95/90/97 case where every candidate was over threshold and the thread bounced between them, discarding the account-isolated prompt-cache prefix each time. New cases in tests/codex-integration/codex-pool-rotation.test.ts pin stay-put, a single move onto real headroom, release on 429, and exhaustion with a sibling.

What did not land, and is what this issue stays open for:

a below-threshold sibling still takes the thread once, so the cache is lost once before affinity becomes sticky
the Phase 1 drain, and the "do not switch until 429" behavior this issue asks for
unbound subagent cascading
cache affinity is still opt-in rather than the default

That remainder is being worked in a separate session against current dev.

Fixed on dev via #4581 at 1de9184.

docs(devlog): roadmap the #4546 cost-guard stabilization program

fix(codex): hold a bound thread's account for its prompt cache (#4546)

docs: align pool routing docs with cache-first bound-thread policy (#…

test(gui): update pool copy assertions to the cache-first wording (#4546

fix(codex): hold a bound thread's account for its prompt cache (#4546) (

Partially fixed on dev by #4580 (squash 8c7f014), on top of #4581 (1de9184). Keeping this open, because the amplifiers behind the measured burn are not addressed yet.

What landed

pool.cacheAffinity now defaults on. A bound thread keeps its account until that account genuinely cannot serve. An explicit false restores capacity-first routing.
A bound thread may only move to a destination with genuine quota headroom and strictly lower usage (fix(codex): keep a bound thread's prompt cache and tell configured routing apart from adopted routing #4581's rule), under either setting. With the whole pool in the 80-99% band nothing moves, which is the reported ping-pong.
A transient failure streak no longer deletes a live binding. The request detours to a remembered alternate while the thread keeps its home and returns as soon as that account serves again; the hold is bounded at 10 minutes. Independent of cacheAffinity. The model-detour lane gets the same hold, and the account-wide affinity clear on a transient streak is removed.

Quota refusals, pauses, credential invalidation, generation bumps and TTL expiry still release a binding.

Why this issue stays open

Routing decided where the cold prefix went; retry decided how many times it was sent, and neither the fan-out nor the reporting is bounded yet. Measured on the current code:

One logical request can reach upstream 4 times on a default Codex 5xx, 7 on a 401 followed by a 5xx, and 12 across a three-target combo. feat(providers): add opt-in transient-5xx retry with a shared total-send budget #2981's shared send budget covers only opted-in key-auth openai-chat; the Codex passthrough path gets a fresh allowance per recovery leg.
There is no per-root cap on concurrent children, no cold-input ceiling, and no cumulative spend limit per workflow. The pool affinity key derives from x-codex-parent-thread-id, so a fan-out draws on the same account the interactive session is bound to.
Bridged Responses, Chat and Anthropic paths emit cached_tokens: 0 unconditionally, so "the provider reported nothing" is indistinguishable from "nothing was cached" — which is why this had to be diagnosed from a token bill rather than a dashboard.

The remaining phases are planned in devlog/_plan/260914_cost_guard_stabilization/: one send budget per logical request, worker/interactive isolation with a root-workflow reservation, quota and cache domain grouping for pooled credentials, and Codex prompt-cache reinforcement with honest cache reporting.

fix(kiro): keep unreported cache counters unknown instead of zero (#4546

Merge pull request lidge-jun#4772 from lidge-jun/codex/bl7-4546-integ…

Merge pull request lidge-jun#4771 from lidge-jun/codex/bl5-recovery-l…

fix(codex): key pool affinity on the conversation cohort, not the thr…

Assessed against dev@61ee64747b by static source reading, in answer to one question: how much of this does the current tip close, and what remains. The full write-up with every citation is in #4969 as devlog/_plan/260914_cost_guard_stabilization/100_dev_tip_closure_assessment.md; this is the summary.

The cache-destruction half is closed. The token-burn half is not, and the reason is not routing. This issue should stay open, but the reason it stays open has changed and is worth restating, because it is no longer the defect it was filed about.

What closes it

mayRebindAffinityForQuota (src/codex/routing/cache-affinity.ts:57) is now the bar a voluntary move must clear, and by default that bar is genuine exhaustion rather than a threshold crossing: it returns !isCodexAccountUsable(...) || (!isUnknownUsage(usage) && usage >= 100) whenever something retains, and isCacheAffinityEnabled is config.pool?.cacheAffinity !== false (src/codex/routing/selection.ts:275). An install that has never heard of the flag keeps its binding through the entire 80-99% band -- the band this report measured. That is remediations 1 and 3 from the original text as the default behaviour.

The re-score short-circuit, which was the sharpest observation in the report, is fixed by keying it off that same predicate (src/codex/routing.ts:680). Since mayRebind is false throughout the band, a bound thread is re-scored once a minute rather than once a request. With cacheAffinity: false the historical rule returns but the destination must still have real headroom and be strictly cooler (pickCacheSafeQuotaReplacement, :649), and since CODEX_UNKNOWN_USAGE_SCORE is 101 an unmeasured account can never win that compare. Preview carries the same rule, so subagent fallback cannot disagree with resolve. Cohort keying (#4935) then derives one key for a whole conversation tree (src/codex/lineage.ts:168), so a fan-out cannot split its family across accounts.

What remains

Phase 1 of the original telemetry, and it is cold for a reason no affinity rule can fix. The 732 one-off subagent calls on account 2 carried ~142k uncached tokens each because each was a distinct prompt, not because it landed on a different account. Account affinity cannot warm a prefix that was never warm.

The instrument built to bound it cannot be switched on. DEFAULT_SPEND_RESERVATION_POLICY leaves every token ceiling undefined by design and says so -- "Real numbers belong behind explicit operator configuration" (src/lib/spend-reservation-ledger.ts:124). That configuration is the part that did not ship: sharedSpendLedger() constructs the process-wide ledger with no policy argument at all (:944), there is no configureSharedSpendLedger, and src/types/config.ts declares no spend key. A SpendReservationPolicy carrying a real maxTokens is built in exactly two places in the tree, both of them tests. limitFor() therefore returns undefined for every scope and the refusal branch is unreachable. The admission path agrees: src/server/index.ts:512 calls admitWorkflowTurn(workflowRootId, workflowLane, undefined, workflowThreadId) with no spend argument.

So what is actually enforced is counts (src/lib/workflow-budget.ts:72), and against the reported rate they do not fire:

ceiling	value	Phase 1 rate	fires?
`maxPhysicalSends`	256 / 10 min	~61 / 10 min	no
`maxDistinctChildren`	64 / 10 min	~61 / 10 min	no, by three
`maxConcurrentChildren`	8 (7 for workers)	shape, not volume	n/a
token ceiling	unset and unsettable	~8.7M uncached input / 10 min	cannot

090_remaining_stack.md predicted exactly this -- "the count caps from #4614 are already live and permissive" -- and chose observational-first deliberately. That choice was right; the configuration surface that was supposed to follow it is the gap.

Three things that are true and easy to misread as defects

Cohort keying now deliberately concentrates a fan-out onto the interactive account, and cacheAffinity then holds the conversation there until usage >= 100. That is correct for cache locality and is what was asked for, but the workers drive the interactive account to hard exhaustion and interactiveReserve: 1 does not help -- it reserves a concurrency slot, not quota. Remediation 2 from the original report, designating a worker account, has no implementation: src/types/config.ts has no worker-lane key and the cohort key currently prevents one from being honoured. Separately, a request carrying no x-codex-parent-thread-id has no root, and admitWorkflowTurn returns early on if (!rootId) return undefined (src/lib/workflow-budget.ts:425), so unparented one-off traffic sits outside every ceiling.

The wpc domain classifier landed and is unwired. src/routing/identity-domains.ts exports classifyCredential, relateQuotaDomain, relateCacheDomain, assessQuotaRotation, countQuotaCapacity and canPortConversationState, and its only production consumer is src/config/schema/leaf-validators.ts importing credentialGroupIssues for config validation. Nothing in the pool selection path consults it; src/server/responses/account-change-state.ts:76 still defers to it "once that module lands". Pool routing continues to treat credential string identity as cache identity.

The synthetic cached_tokens: 0 on the bridged wire is a required compatibility behaviour, not dishonest accounting: grok-build's pinned async-openai fork deserializes the detail objects as required fields, so omitting them turns a completed turn into a hard exit (src/bridge/internal.ts:36). Internal provenance is honest -- usageFromBridge protects raw adapter values and src/server/request-log.ts:170 records that a synthetic zero is not a measured cache read. OpenCodex can tell unreported from measured-zero; a strict client reading the wire still cannot.

Recommended next unit

Not a routing change. A configuration surface for SpendReservationPolicy -- an operator-set token ceiling at root scope, defaulting to unset so no existing install is newly refused -- plus the worker-lane account designation that cohort keying currently makes impossible.

One correction for the record: of the merges adjacent to this work, only #4935 moves this issue. #4071 is GUI static-asset HTTP caching, #4676 is Antigravity failover ranking, and #4921 is flagship-model availability by roster.

docs(devlog): assess what the dev tip closes of the pooled-routing co…

Re-verified against `dev` at `11bc4f708c`

The assessment posted here against 61ee64747b still holds: the cache-destruction half is closed,
the token-burn half is not, and the reason it stays open is still not routing. Static source
reading only; no local verification was run.

Nothing in the mechanism moved in between. Diffing the eleven files that assessment rests on between
61ee64747b and 11bc4f708c touches three, none of them behaviourally relevant here:
src/codex/routing.ts (pin-release classification moved into codexAccountPinDrainReason),
src/server/index.ts (an applyProxyEnv argument), and src/types/config.ts
(codexProviderDisplayName plus proxy documentation). cache-affinity.ts, selection.ts,
lineage.ts, spend-reservation-ledger.ts, workflow-budget.ts, identity-domains.ts,
account-change-state.ts and request-spend.ts are unchanged.

Confirmed at this tip

mayRebindAffinityForQuota (src/codex/routing/cache-affinity.ts:57) still returns
!isCodexAccountUsable(...) || usage >= 100 whenever something retains, so a threshold crossing
alone cannot move a binding.
isCacheAffinityEnabled is still config.pool?.cacheAffinity !== false
(src/codex/routing/selection.ts:275), so unset, absent pool, and true all enable it.
The re-score short-circuit is still keyed off that predicate (src/codex/routing.ts:693) against
CODEX_THREAD_AFFINITY_REEVAL_INTERVAL_MS of 60s.
pickCacheSafeQuotaReplacement still requires headroom and bestUsage < boundUsage
(src/codex/routing.ts:640), and CODEX_UNKNOWN_USAGE_SCORE is still 101
(src/codex/quota.ts:78), so an unmeasured account can never win the compare.
Cohort keying still derives one key per conversation tree (src/codex/lineage.ts:168, :189,
:238), with a bare thread-id carrying no session or parent left unbound.
The spend gap is intact in all five parts: no default token ceiling (:119), sharedSpendLedger()
constructed with no policy (:944), no configureSharedSpendLedger anywhere in the tree, no
spend key in src/types/config.ts or src/config/schema/config-schema.ts:211, and therefore no
reachable refusal.
src/server/index.ts:505 still passes no spend to admitWorkflowTurn, the count ceilings are
still 256 sends / 64 distinct children / 8 concurrent with interactiveReserve: 1
(src/lib/workflow-budget.ts:72), and if (!rootId) return undefined (:425) still puts
unparented traffic outside every one of them.
src/routing/identity-domains.ts is still unwired: the only production importer is
src/config/schema/leaf-validators.ts:15, and it takes credentialGroupIssues for config
validation only.

Three corrections to the earlier write-up

1. The admitWorkflowTurn argument was named wrong. The explicit undefined in
admitWorkflowTurn(workflowRootId, workflowLane, undefined, workflowThreadId) is policy, the third
parameter, not spend. spend is the sixth and is simply omitted (src/lib/workflow-budget.ts:416).
The conclusion is unaffected — no spend request reaches admission — but the position was misdescribed.

2. "limitFor() returns undefined for every scope" is overbroad. limitFor reads
policy[scope].maxTokens (src/lib/spend-reservation-ledger.ts:530), so an injected policy does
produce real limits; that is how the tests reach the refusal branch. The accurate statement is that
it returns undefined for every production scope, because the shared ledger is always built with
the observe-only default.

3. Dispatch-time accounting is live, which the write-up did not say.
src/server/responses/request-spend.ts:73 reserves the ledger on every physical send and
refuses only on spend-limit-exceeded, failing open on any other denial. So production already
measures and journals spend per send; what is missing is exclusively the operator-set ceiling that
would let a reservation refuse. That is a materially better starting position than "the instrument
does not exist" — the wiring is done and the configuration surface is the gap.

One thing the write-up missed

src/server/responses/account-change-state.ts:73 still says src/routing/identity-domains.ts "owns
this decision once that module lands", and asks that the check stay in one function so it can be
swapped for the shared export. That module has landed, and it exports an equivalent
canPortConversationState at src/routing/identity-domains.ts:440. So the response path now carries
a second implementation of the same decision behind a comment that describes it as temporary. That is
a small, self-contained follow-up, and it is the kind of duplicate that diverges quietly.

Remediation status, corrected

The earlier write-up overstated remediations 1 and 3 by treating default affinity as equivalent to
both.

1, mandatory session sticky routing — partial. One conversation tree shares one key and
affinity is on by default, but it is not mandatory: pool.cacheAffinity: false restores
threshold-driven movement (documented as "restores capacity-first routing" at
src/types/config.ts:890), and quota refusal, unusability, transient detours and exhaustion still
move serving traffic.
2, subagent circuit breaker — partial. The count circuit is live but permissive against the
measured Phase 1 rate; the token circuit cannot be configured; unrooted traffic returns before
every ceiling; and there is still no worker-lane account key, which cohort keying currently
prevents from being honoured anyway.
3, sequential drain — partial, and conversation-local only. A bound conversation stays on one
account through the 80-99 band and moves at exhaustion. There is no global "drain A before placing
any new session on B": new unbound sessions still use DEFAULT_STRATEGY = "quota"
(src/oauth/pool-kernel.ts:25).

Smallest next unit

Still not a routing change, and now smaller than the earlier write-up implied, because the dispatch
reservation is already wired. Add an opt-in root-scope token ceiling and a startup seam to install
it: a narrow spendReservation?: { rootMaxTokens?: number } in src/types/config.ts and
config-schema.ts accepting only a bounded positive integer, a startup-only
configureSharedSpendLedger(policy) in src/lib/spend-reservation-ledger.ts called before the lazy
ledger is created, and the call in src/server/index.ts after validated config load. Default unset,
so no existing install is newly refused. admitWorkflowTurn stays unchanged — HTTP admission runs
before input tokens are known, while the physical-send tracker already knows them.

The worker-lane account designation is the unit after that, and it needs a cohort-key decision
first: concentrating a fan-out onto the interactive account is currently correct for cache locality
and is exactly what prevents remediation 2 from being honoured.

What static source still cannot establish: whether the incident's one-off children carried
session-id, thread-id and x-codex-parent-thread-id, whether they would share one cohort key
today, and how much of the measured multiplier came from account movement rather than distinct cold
prompts. The source proves the rules, not the shape of the historical traffic.

Assessment against the current dev tip, so this issue is narrowed to what actually remains rather than left as a whole open report. Full record in devlog/_plan/260914_cost_guard_stabilization/100_dev_tip_closure_assessment.md.

The cache-destruction half is closed. The token-burn half is not, and the reason is not routing.

What closed it. mayRebindAffinityForQuota now requires genuine exhaustion rather than a threshold crossing, and cacheAffinity defaults on, so an install that has never heard of the flag keeps its binding through the entire 80-99% band — the exact band you measured, and your own remediations 1 and 3 as the default. The re-evaluation short circuit keys off the same predicate as the rebind decision, so a bound thread in that band is re-scored once a minute rather than once a request, which was your sharpest observation. Cohort keying (#4935) derives one key for a whole conversation tree, so a fan-out cannot split its family across accounts. With cacheAffinity: false the old rule returns, but the destination must still be strictly cooler and CODEX_UNKNOWN_USAGE_SCORE is 101, so an unmeasured account can never win and the all-hot case selects nothing under either setting.

What remains is not an affinity problem and no affinity rule can fix it. Your Phase 1 telemetry is a high-volume fan-out of one-off requests, each carrying a large cold prefix. That prefix is cold because each request is a distinct prompt, not because it landed on a different account.

The instrument that would bound it exists, journals to disk, and cannot be switched on. DEFAULT_SPEND_RESERVATION_POLICY sets every limit undefined, the process-wide ledger is constructed with no policy argument, there is no configureSharedSpendLedger, and src/types/config.ts declares no spend key. A SpendReservationPolicy with a real maxTokens is constructed in exactly two places in the tree and both are tests. So limitFor() returns undefined for every scope and the refusal branch is unreachable. The admission path agrees: admitWorkflowTurn is called with no spend argument.

Measured against your figures — 732 one-off calls in two hours, about 142k uncached tokens each, 104,188,859 uncached total:

ceiling	value	your Phase 1 rate	fires?
`maxPhysicalSends`	256 / 10 min	~61 / 10 min	no
`maxDistinctChildren`	64 / 10 min	~61 / 10 min	no, by three
token ceiling	unset and unsettable	~8.7M uncached input / 10 min	cannot

The incident passes every live ceiling. Observational-first was a deliberate choice and the right one; the configuration surface that was supposed to follow it is the gap.

One consequence worth stating plainly, because it is a cost of the fix rather than a regression. Before cohort keying a fan-out drew on whichever account each child resolved to. Now the tree shares one key by design, so a large cold fan-out draws on precisely the account the interactive conversation is bound to, and cacheAffinity holds it there until usage reaches 100.

This issue stays open for the token ceiling specifically. A lane is implementing the operator-facing configuration so the existing enforcement can actually be turned on.

fix(spend): record a send that already left, so the ceiling can arm (#…

feat(spend): make the durable token ceiling configurable and its refu…

Closed on dev by #5032, squashed as a13a908193799b6aaccb2cfaa5ac30e37a86aeb2.

Both halves of your report are now addressed, and they turned out to need different fixes.

The cache-destruction half closed earlier in this round. mayRebindAffinityForQuota now requires genuine exhaustion rather than a threshold crossing and cacheAffinity defaults on, so a binding survives the entire 80-99% band — your remediations 1 and 3 as the default. The re-evaluation short circuit keys off the same predicate, so a bound thread is re-scored once a minute rather than once a request, which was your sharpest observation. Cohort keying in #4935 makes a conversation tree resolve to one account.

The token-burn half is what this change closes, and the diagnosis was that it was never a routing problem. The spend ledger already supported per-scope token ceilings and already journalled faithfully — but DEFAULT_SPEND_RESERVATION_POLICY left every limit undefined, the shared ledger was constructed with no policy argument, there was no configureSharedSpendLedger, and the config declared no spend key. A policy with a real maxTokens existed in exactly two places in the tree and both were tests. So limitFor() returned undefined for every scope and the refusal branch was unreachable.

Measured against your own figures — 732 one-off calls in two hours, ~142k uncached tokens each, 104,188,859 total — every live ceiling passed: maxPhysicalSends at 256 per ten minutes against ~61, maxDistinctChildren at 64 against ~61, and a token ceiling that could not be set at all.

There is now a validated spend section with per-scope maxTokens and retentionDays. A denial carries scope, limit and projected into the 429 message, the refusal header, the log row and the workflow-budget event ring, so a refusal names which ceiling refused it rather than arriving as a generic budget error. Counts are checked before tokens and neither refusal is relabelled as the other.

No default ceiling ships. An install that has never written the key is byte-identical: observe-only accounting, the existing count caps, no ledger resolved at admission, no journal opened. Real numbers belong to you, not to us.

Two limitations, stated rather than hidden. A send that crosses a ceiling mid-request still reports request_send_budget_exhausted on the wire, because the request execution budget flattens every refusal before the dispatch renderers; its log row and event carry workflow_spend_exhausted with scope and ceiling, and the next request is refused legibly at admission. And the in-flight output reservation reads only an explicit max_output_tokens, so a caller sending none under-reserves the guard — settlement uses real usage, so the durable total is unaffected.

One consequence of the earlier fix worth knowing, since it is a cost rather than a regression. Before cohort keying, a fan-out drew on whichever account each child resolved to. Now the tree shares one key by design, so a large cold fan-out draws on precisely the account your interactive conversation is bound to. With a ceiling configured, that is now bounded; without one, it concentrates.

Shipping in the next release off dev. Thank you for the telemetry — the per-phase token figures are what made it possible to prove which ceiling would and would not have fired.

One finding from the implementation of #5032 deserves to be recorded here rather than only in the pull request, because it is the kind of defect that would have made this issue look fixed while nothing changed.

The first cut of the ceiling would have been inert on the path that matters. The canonical passthrough ladder does not reserve its physical sends; it reports them after the fetch through onSendsConsumed, charging the observer post hoc. reserve() refused anything over the limit, and a refused reservation books nothing — so the very send that would have crossed the ceiling was dropped from the total.

That is a fixpoint, not a rounding error. The total sits one send short of the limit forever, the scope never reads as exhausted, and nothing is ever refused. Against a 20M root ceiling with your ~142k-token requests, accounting would have stalled near 19.9M and refused nothing at all — indistinguishable from the current behaviour, with a config key that looked like it was working.

It was found by adversarial review rather than by a failing test, which is worth saying because no test would have caught it: every individual piece behaved as written. src/lib/request-execution-budget.ts already documented the intended behaviour in a comment; the ledger simply could not honour it.

The fix is that a reservation for a send which has already left skips the limit check and the durability refusal and is marked dispatched immediately. Taking the scope over its ceiling is what arms the next refusal, rather than the crossing send being the one that is refused. The same review also found the tracker treating "reserve not durable" as permission to send, which now refuses — and because the ledger raises that only under a configured limit, unconfigured installs are untouched.

Mentioning it so that if you configure a ceiling and see the first over-limit request succeed, that is the designed behaviour and not a regression: the crossing send completes, and the next one is refused with the scope and ceiling named.

release: opencodex 2.55.0

fix(kimi): K2.8 registry refresh — adjustable thinking, 1M context, k…

fix: demote developer to user for OrcaSAQ-2 leading-system template (#…

Navigation Menu

Repository navigation

[Bug]: Account pool routing destroys prompt cache and triggers 10x-50x token burn death-spiral above 80% usage threshold #4546

[Bug]: Account pool routing destroys prompt cache and triggers 10x–50x token burn death-spiral above 80% usage threshold

1. Summary

2. Environment & Version

3. Root Cause Analysis (Code & Empirical Telemetry)

3.1 Phase 1: How Account 2 & 3 Reached 80% (Subagent Workload Cascading)

3.2 Phase 2: The 80% Multi-Turn Ping-Pong Death Spiral (`src/codex/routing.ts`)

4. Production Telemetry & Empirical Evidence

4.1 Telemetry by Account (Masked)

4.2 Cache Hit Rate Dynamics

5. Proposed Remediation

1. Mandatory Session Sticky Routing

2. Subagent / One-Off Task Circuit Breaker

3. Sequential Account Draining Instead of Load-Balancing

리뷰 · 우선순위 78 / 80

193 remaining items

What closes it

What remains

Three things that are true and easy to misread as defects

Recommended next unit

Re-verified against `dev` at `11bc4f708c`

Confirmed at this tip

Three corrections to the earlier write-up

One thing the write-up missed

Remediation status, corrected

Smallest next unit

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[Bug]: Account pool routing destroys prompt cache and triggers 10x-50x token burn death-spiral above 80% usage threshold #4546

Description

[Bug]: Account pool routing destroys prompt cache and triggers 10x–50x token burn death-spiral above 80% usage threshold

1. Summary

2. Environment & Version

3. Root Cause Analysis (Code & Empirical Telemetry)

3.1 Phase 1: How Account 2 & 3 Reached 80% (Subagent Workload Cascading)

3.2 Phase 2: The 80% Multi-Turn Ping-Pong Death Spiral (src/codex/routing.ts)

4. Production Telemetry & Empirical Evidence

4.1 Telemetry by Account (Masked)

4.2 Cache Hit Rate Dynamics

5. Proposed Remediation

1. Mandatory Session Sticky Routing

2. Subagent / One-Off Task Circuit Breaker

3. Sequential Account Draining Instead of Load-Balancing

Activity

lidge-jun commented on Sep 13, 2026

리뷰 · 우선순위 78 / 80

lidge-jun commented on Sep 14, 2026

lidge-jun commented on Sep 14, 2026

lidge-jun commented on Sep 14, 2026

193 remaining items

lidge-jun commented on Sep 17, 2026

What closes it

What remains

Three things that are true and easy to misread as defects

Recommended next unit

lidge-jun commented on Sep 18, 2026

Re-verified against dev at 11bc4f708c

Confirmed at this tip

Three corrections to the earlier write-up

One thing the write-up missed

Remediation status, corrected

Smallest next unit

lidge-jun commented on Sep 18, 2026

lidge-jun commented on Sep 18, 2026

lidge-jun commented on Sep 18, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

3.2 Phase 2: The 80% Multi-Turn Ping-Pong Death Spiral (`src/codex/routing.ts`)

Re-verified against `dev` at `11bc4f708c`