Skip to content

feat(codex): append tools loaded by Claude Code's tool search instead of rewriting the tool prefix - #184

Draft
Butanium wants to merge 2 commits into
raine:mainfrom
Butanium:incremental-deferred-tools
Draft

Butanium wants to merge 2 commits into
raine:mainfrom
Butanium:incremental-deferred-tools

Conversation

@Butanium

@Butanium Butanium commented Oct 15, 2026 •

Copy link
Copy Markdown

⚠️ Drafted by Claude (Opus 5.5) in Claude Code at @Butanium's request; not reviewed by a human. We hit this in real use, and he asked for it to be fixed and sent upstream. Whether to AI-review, merge, adapt or close it is up to you.

Closes #182. With Claude Code's tool search on (ENABLE_TOOL_SEARCH=true), a tool loaded through ToolSearch is now appended where it was loaded, instead of rewriting the leading tool list. Requests stay append-only across loads and keep their prompt cache. The load result now reads Tool loaded: <name>, and Claude Code's DeferredToolPlaceholder stub is no longer forwarded.

Problem. Without tool search, Claude Code sends every MCP tool schema with every request. In our case that was 281 tools, about 150k of a ~270k-token request. With tool search on, the Lite lane put every loaded deferred tool into the single leading additional_tools item. So each load changed the first input item, and the next request missed the cache for the whole conversation.

Change (Responses Lite lane)

  • A deferred tool loaded by a tool_reference in the history leaves the leading item. It is appended as its own additional_tools item right after the function_call_output of the result that loaded it. This is what Codex's IncrementalTools does (test).
  • Deferred tools with no load in the visible history, such as tools loaded before a compaction, stay in the leading item so they remain callable.
  • tool_reference renders as Tool loaded: <name>.
  • DeferredToolPlaceholder is dropped on both lanes.

The full Responses lane keeps every loaded tool in tools, as before.

Result. Replaying a real Claude Code 2.1.293 session (load a tool, run a command, finish) through the proxy into a stub upstream, each translated request's input is an exact prefix of the next one's, including across the load. Against real Codex, the request right after a load read 99% of its input from the cache.

Docs. "Configure Claude Code" gains an "MCP tool search" section recommending ENABLE_TOOL_SEARCH=true. The Codex page describes the mapping.

Details for reviewers and agents: implementation, edge cases, tests, live check

Implementation (src/providers/codex/translate/request.rs)

  • tool_reference_loads maps each tool_result's tool_use_id to the tool names its tool_reference children load.
  • split_loaded_deferred_tools keeps loaded deferred tools out of the prefix and drops the placeholder.
  • append_loaded_tools inserts one additional_tools item after each matching function_call_output. The order follows the tools array and the message order, so output is deterministic. A tool loaded twice is appended once, at its first load.
  • A debug_assert! checks that every loaded tool was placed.

Edge cases checked

  • One result loading several tools.
  • Several results in one message.
  • A tool_reference naming a tool absent from tools: no item is emitted.
  • Error results.
  • String content.
  • Server compaction: mid-history additional_tools items fall in the conversation slice of split_input_envelope. After Claude Code compacts, the loads are no longer visible, so those tools return to the prefix.
  • Continuation: the Lite lane keeps tools: None, and the new items land in the input delta.

Tests

  • New:
    • loaded_deferred_tool_is_appended_after_its_tool_search_result
    • tool_load_keeps_the_earlier_request_as_a_prefix
    • deferred_tool_without_a_visible_load_stays_in_the_prefix
    • full_lane_keeps_loaded_tools_and_drops_the_placeholder
  • cargo fmt --all -- --check: clean.
  • cargo clippy --all-targets --locked: no diagnostics.
  • cargo test --all-targets --locked: 1121 passed, 0 failed. One earlier run failed bridge_resume_continues_after_tool_use_pause in tests/cursor_native.rs, which this change doesn't touch. That test file passed 15 of 15 times both on this branch and on main, and the next full run passed.
  • Rebased onto main at fe10718 (2026-11-11), which adds the opt-in full Responses lane (codex: add fullLane config to escape Responses Lite for sol/terra (parallel tool calls) #77): cargo clippy --all-targets --locked -- -D warnings clean, cargo test --all --locked 1175 passed, 0 failed. The full lane keeps loaded tools in tools as before; only the Lite lane appends them mid-history.

Live check. This was a real Claude Code 2.1.293 session with ENABLE_TOOL_SEARCH=true and gpt-6-astra, through our running build (this change applied on v0.1.44). ToolSearch loaded the deferred CronList, the model called it, then it ran a Bash command. Codex accepted the appended additional_tools item. Cache reads per request:

Request Input tokens Read from cache
1, first 35,626 0
2, right after the load 35,793 35,456
3 35,824 35,584
4 35,872 35,584

Review. One automated review pass with a separate Claude agent found no correctness bug. It suggested the debug_assert!. It could not determine from the source whether Codex accepts an id-less additional_tools item mid-input; the live check above covers that.


Drafted by Claude Opus 5.5 in Claude Code on behalf of @Butanium; not reviewed by a human.

https://claude.ai/code/session_01Xmx9RE7u4ZLBzSUjpjqyBM

Butanium and others added 2 commits November 11, 2026 12:47
…incremental tools

With tool search on (ENABLE_TOOL_SEARCH), Claude Code sends deferred MCP tools
with defer_loading: true once ToolSearch has loaded them, and marks the load
with a tool_reference block in the tool_result. On the Responses Lite lane the
proxy used to put every tool, deferred or not, in the leading additional_tools
item, so each load rewrote the prefix and the next request missed the prompt
cache. tool_reference blocks also reached the model as
'[unsupported content block omitted: tool_reference]'.

Now, on the lite lane, a deferred tool that a tool_reference loaded is appended
as its own additional_tools item right after that tool_result, the way Codex's
IncrementalTools feature appends catalog changes without rewriting history
(core/tests/suite/scenarios_incremental_tools.rs). Deferred tools with no
visible load (e.g. loaded before a compaction) stay in the prefix. The
Anthropic-only DeferredToolPlaceholder stub is dropped on both lanes, and
tool_reference renders as 'Tool loaded: <name>'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Xmx9RE7u4ZLBzSUjpjqyBM
…odex

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Butanium
Butanium force-pushed the incremental-deferred-tools branch from 4aed636 to 40515bd Compare November 11, 2026 21:02
gongx030 added a commit to gongx030/claude-code-proxy that referenced this pull request Nov 12, 2026
)

* feat(codex): append loaded deferred tools incrementally on the Lite lane

With tool search enabled in Claude Code, a ToolSearch result carries
tool_reference blocks and the loaded tools arrive with defer_loading.
The Lite lane previously put every tool in the leading additional_tools
item, so each load rewrote the request prefix, and tool_reference
reached the model as an unsupported-block placeholder.

Now a deferred tool loaded by a tool_reference is appended as its own
additional_tools item right after the matching function_call_output, so
each request's input is an exact prefix of the next. The Anthropic-only
DeferredToolPlaceholder is dropped (only when also defer_loading) on
both lanes, and tool_reference renders as "Tool loaded: <name>". Full
lane tools and parallel_tool_calls are unchanged.

Adds a 3-request sequence test on both lanes (fails on the prior code:
the second request rewrote the prefix) and a byte-identity test whose
goldens were captured before this change.

Based on raine#184 (commit 8f326c0, request.rs only).
Refs raine#182

Co-Authored-By: Clement Dumas <dumasclement2002@gmail.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

* docs: document ENABLE_TOOL_SEARCH and incremental deferred tools on Codex

Documents the opt-in ENABLE_TOOL_SEARCH setting and how the Codex
Responses Lite lane now appends loaded deferred tools where they were
loaded. Describes the mechanism only; no cost or cache-hit claim, and
the proxy does not enable the setting itself.

Adapted from raine#184 (commit 40515bd).
Refs raine#182

Co-Authored-By: Clement Dumas <dumasclement2002@gmail.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Wuming Gong <gongw13@gene.com>
Co-authored-by: Clement Dumas <dumasclement2002@gmail.com>
Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex: with ENABLE_TOOL_SEARCH, each ToolSearch load rewrites the cached prefix and tool_reference reaches the model as an unsupported block

1 participant