[Bug]: routed first-turn tool catalog is 3–5x native Sol input tokens #1587

Activity

1. lidge-jun

   added

   bugSomething isn't workingSomething isn't working

   proxyHTTP proxy, routing, reverse-proxy / management authHTTP proxy, routing, reverse-proxy / management auth

   on Aug 13, 2026
2. coderabbitai commented on Aug 13, 2026

   

   coderabbitaibot

   on Aug 13, 2026 – with coderabbitai

   Contributor

   More actions

   🔗 Related PRs

   #302 - feat(kiro): harden completion and transport integration [merged]
   #304 - fix(kiro): restore hardening and complete text turns [open]
   #426 - feat(codex): add account-qualified model namespaces [open]
   #439 - fix(kiro): report context pressure for compaction [open]
   #757 - feat: route GPT-5.6 Pro through standard ChatGPT [open]

   ---

   📝 Issue Planner

   _{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

   • Create Plan

   ---

   🧪 Issue enrichment is currently in open beta.

   To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

   issue_enrichment:
     auto_enrich:
       enabled: false

   💬 Have feedback or questions? Drop into our discord!
3. github-actions

   added

   catalogModel catalog, slugs, visibility, routed entriesModel catalog, slugs, visibility, routed entries

   toolstool_calls, MCP, web-search / sidecar toolstool_calls, MCP, web-search / sidecar tools

   on Aug 13, 2026
4. lidge-jun

   added

   roadmapAccepted long-term work item; needs its own project cycleAccepted long-term work item; needs its own project cycle

   and removed

   roadmapAccepted long-term work item; needs its own project cycleAccepted long-term work item; needs its own project cycle

   on Aug 15, 2026
5. lidge-jun commented on Aug 19, 2026

   

   lidge-jun

   on Aug 19, 2026

   OwnerAuthor

   More actions

   리뷰 · 우선순위 52 / 80

   같은 첫 인사인데 네이티브 gpt-5.6-sol은 약 21k input이고, 라우트된 Kimi/Anthropic은 62k에서 99k까지 올라간다는 보고입니다. 보이는 채팅 텍스트는 비슷하고, 차이는 Codex Desktop의 hosted/namespace 도구를 라우트 어댑터가 전체 function JSON으로 펼치는 데 있습니다. 네이티브 경로가 깨진 것은 아니지만, 매일 토큰 비용이 3-5배가 됩니다.

   현재 dev의 src/responses/parser.ts buildTools()는 type namespace 그룹을 개별 함수로 펼칩니다. functions 네임스페이스는 네임스페이스 없이 평탄화하고, 다른 MCP 그룹은 라운드트립을 위해 네임스페이스를 유지합니다. src/adapters/openai-chat.ts의 toolsToChatFormat()과 src/adapters/anthropic.ts의 toolsToAnthropicFormat()이 그 함수마다 parameters/input_schema를 실어 보냅니다. 네이티브 openai-responses는 이 확장을 하지 않습니다.

   원하는 결과는 라우트된 첫 턴도 네이티브 21k 근처에 가깝거나, 쓰이지 않는 커넥터 스키마를 뒤로 미루는 것입니다. 채팅/Anthropic 와이어는 호스티드 네임스페이스를 그대로 못 받으므로, 스키마를 안 보내면 도구 호출이 깨질 수 있습니다. 그래서 단순히 카탈로그를 줄이는 패치는 위험합니다.

   해결방안: 첫 턴에서 네임스페이스만 광고하고 선택 시에만 스키마를 넣는 deferred-tool 설계가 이슈 방향과 맞습니다. 다만 모델이 이름을 추측하지 못하게 parser가 wire name을 정확히 나열하는 계약이 이미 있습니다. 그 계약을 깨지 않는 설계가 필요합니다. 지금은 어떤 커넥터를 늦출지 근거가 없어 패치는 추측하지 않음.

   이 댓글은 grok-bot이 작성했습니다
6. alexph-dev commented on Aug 21, 2026

   

   alexph-dev

   on Aug 21, 2026 – with ChatGPT Codex Connector

   Contributor

   More actions

   Additional evidence: the excess catalog also remains uncached on continued Claude Desktop turns

   I audited local OCX usage for 2026-08-20 on macOS with OpenCodex 2.27.0, separating cold first turns from follow-ups and comparing all four client/provider combinations used that day.

   The strongest reproduction is one Claude Desktop session routed alternately to native OpenAI and native Anthropic. Request/account identifiers are omitted; conversation continuity was verified from the same stable OCX conversation id per model.

   Client → provider/model	Consecutive turn	input	cache read	cache write	cache-read share
   Claude Desktop → GPT-5.6 Sol	1	57,063	29,184	0	51.1%
   Claude Desktop → GPT-5.6 Sol	2	57,178	29,184	0	51.0%
   Claude Desktop → GPT-5.6 Sol	3	57,765	29,184	0	50.5%
   Claude Desktop → Claude Opus 5	1	84,246	45,478	38,766	54.0% read
   Claude Desktop → Claude Opus 5	2	84,438	45,478	38,958	53.9% read

   The OpenAI cache key appears stable: the cache-read count is exactly 29,184 on every continued turn. But roughly 28k tokens remain outside the reusable prefix on every request. On Anthropic, the same session repeatedly reads the stable 45,478-token prefix while recreating almost the entire 39k-token tail on every turn. This is quota/cost-significant, not only a dashboard presentation issue.

   Controls from Codex Desktop on the same install/day were much healthier for the main sessions:

   • GPT-5.6 Sol: 78.9–93.1% aggregate cache-read across two multi-turn conversations.
   • Claude Fable 5: 93.2% aggregate cache-read across 18 requests.
   • There were separate low-hit helper/Luna and third-party-provider conversations, but those are not used as the primary reproduction because many were cold one-shot calls.

   This extends the issue beyond first-turn inflation: the expanded/deferred tool and instruction shape seems to create a large stable-but-nonreusable tail specifically on Claude Desktop routing. The exact fixed read counts make random eviction an unlikely explanation.

   Suggested acceptance check for a fix: use one Claude Desktop conversation, send 3 no-tool follow-ups through GPT and Claude, and assert that cache-read grows with the shared conversation prefix rather than remaining fixed while a 28–39k tail is reprocessed/recreated.
7. Ingwannu commented on Aug 21, 2026

   

   Ingwannu

   on Aug 21, 2026

   Owner

   More actions

   This confirms the operational impact beyond the cold first turn: the fixed cache-read counts and repeated 28k to 39k uncached/cache-created tail are too large to dismiss as dashboard presentation noise.

   It does not yet prove that the tool schemas themselves change between turns. The same numbers are also consistent with a stable oversized tool/instruction tail being serialized after the providers reusable cache boundary. Those two causes need different fixes: canonicalization for an unstable tail, or deferred schemas/cache-boundary placement for a stable tail. Dropping schemas before distinguishing them could break tool availability.

   The next acceptance capture should stay on one provider/model for three no-tool follow-ups on current 2.28.x or current dev, without alternating the same conversation across OpenAI and Anthropic. We need only bounded structural facts per turn: input/cache-read/cache-create counters, tool count, serialized tool-catalog byte count, and a privacy-safe equality indicator for the canonical tool catalog and instruction envelope. No tool schemas, prompts, identifiers, or credentials should be logged.

   If the catalog digest changes, fix ordering/canonicalization first. If it stays equal while the large tail is repeatedly uncached, the deferred-tool design is the correct production direction. I am keeping this open as a confirmed bug and will not reduce the catalog speculatively.
8. lidge-jun commented on Aug 23, 2026

   

   lidge-jun

   on Aug 23, 2026

   OwnerAuthor

   More actions

   Already fixed by fcbef381e, confirmed an ancestor of origin/dev via git merge-base --is-ancestor.

   That commit restored deferred discovery after measuring the regression precisely: first-turn payload down from 258,929 to 96,699 characters, recorded at structure/03_catalog-and-subagents.md:231, with MCP reachability preserved. normalizeRoutedCatalogEntry() at src/codex/catalog/parsing.ts:526 pairs tool_mode: "code_mode_only" with supports_search_tool: true, and src/codex/catalog/sync.ts:371 gives template-less routed entries the same contract. bun test tests/catalog-cursor-search.test.ts re-run just now: 5 pass / 0 fail.

   On the later Claude Desktop cache-tail comment: that is a different client and a smaller, separate question, not a reproduction of the original Codex App defect. Closing this on the original report — happy to look at the Claude Desktop caching behavior under its own issue rather than keeping a fixed one open as a placeholder.
9. lidge-jun

   closed this as completedon Aug 23, 2026
10. vadymhimself commented on Aug 31, 2026

    

    vadymhimself

    on Aug 31, 2026

    Contributor

    More actions

    We can reproduce the unresolved routed-Claude cache-tail problem described above on OpenCodex v2.37.0.

    Recent 4-hour sample

    • 32 routed claude-opus-5 logical requests
    • 56 physical attempts/sends and 24 recoveries
    • 10.40M inclusive input tokens
    • 2.46M cache-read, 7.94M cache-created, only 38 raw
    • 28.37% post-warm cache-read share
    • all 15 measured post-warm requests wrote over 100k tokens; maximum 553,558

    Cleaner one-send sequence

    Same observed provider/model, one attempt/send, no recovery, and 7–18 second gaps:

    • cache-read stayed fixed at 122,957
    • cache creation grew 195,008 → 196,000 → 210,133 → 212,493

    Across the durable ledger, 4,147/4,948 cache-creation transitions followed the same observed provider/model within 5 minutes; 4,060 simultaneously read an earlier prefix. This bucket represented 74.9% of cache-write volume.

    Native control was materially healthier: two warmed native Anthropic sessions reached 98.12% and 96.96% post-warm cache-read share with no post-warm write above 10k.

    This proves stable-prefix reuse plus repeated large-tail creation. It does not identify the exact invalidator because current telemetry lacks account identity, branch lineage, replay-path status, and cache-breakpoint prefix fingerprints.

    We also tested a local canonical-Messages source-wire preservation patch. It removed the previous large raw-input billing but did not stop tail rewrites, so we are not presenting it as a completed fix:
    https://github.com/vadymhimself/opencodex/tree/fix/routed-anthropic-source-wire-870a2ad

    Since the original tool-catalog problem is fixed but this routed Anthropic cache regression remains, should this issue be reopened or split into a dedicated routed-cache issue?