[REFERENCE ONLY — DO NOT MERGE] Preserve Token Miser dev.8 known-good source - #10
Draft
Harold Hunt (huntharo) wants to merge 24 commits into
Draft
[REFERENCE ONLY — DO NOT MERGE] Preserve Token Miser dev.8 known-good source#10Harold Hunt (huntharo) wants to merge 24 commits into
Harold Hunt (huntharo) wants to merge 24 commits into
Conversation
) Three fixes from the first end-to-end run. Authenticode signing failed before signing anything: Cannot process argument transformation on parameter 'Files'. Cannot convert value to type System.String. `Invoke-TrustedSigning -Files` is typed [string], not [string[]]. grok-build passes a single path, which is the shape the module actually accepts; batching five was my assumption and it was wrong. Now one call per binary, signing and verifying in the same pass so a failure names the file it belongs to. Five signing round-trips instead of one, which costs seconds. Both macOS builds were killed at the 120 minute timeout. Measured on standard hosted runners: linux-aarch64 43m, linux-x86_64 55m, windows-prepare 101m, macos-aarch64 114m, macos-x86_64 killed at 120m. Standard hosted macOS is a 3-core M1 and roughly 2x slower than ubuntu for this workspace. Raised to 240, well inside GitHub's 6 hour per-job ceiling. The cache was also not being kept. The combined `actions/cache` skips its post-step save when a job fails, so the timed-out macOS jobs discarded everything they had compiled. Split into restore/save with `if: always()`, and extended to cover codex-rs/target so a killed build resumes rather than restarting. Upstream uses the same split in bazel.yml with a `!cancelled()` guard; that guard is deliberately omitted here, since a timeout is precisely the case whose output is worth keeping. The save key carries run_id and run_attempt because cache entries are immutable once written.
`handle_runtime_response` is the script-to-model boundary in code mode: it
converts all three `RuntimeResponse` arms into content items and passes them
through `truncate_code_mode_result`, which is a fixed token truncation. That
funnel already destroys output, so replacing the fixed truncation with a
pluggable reduction changes only what enters model context.
This deliberately does not touch `call_nested_tool`. That `JsonValue` is
returned *into the running script*, which may grep, parse, or count it;
substituting a summary there would silently break scripts doing programmatic
post-processing. `PostToolUseFeedbackOutput::code_mode_result` returning
`self.original` is consistent with that and is left alone.
What this adds:
- A reduction seam wrapping `truncate_code_mode_result`, covering the Yielded,
Terminated, and Result arms and both the `exec` and `wait` callers. With
nothing configured it is the existing truncation call, byte for byte.
- An optional external reducer over loopback HTTP, off by default. Codex reads
a host-written descriptor (`{"version":1,"url":...,"token":...}`, mode 0600)
on every reduction, POSTs the full content items with a bearer token, and
uses the bounded replacement it gets back. The call originates in the Codex
process, which is the parent of the `sandbox-exec`/landlock children and is
not itself sandboxed, so there is no sandbox transport problem here.
- Fail-open on every path: missing descriptor, unreachable host, timeout,
non-2xx, malformed body, oversized body, and empty replacement all fall back
to truncation. A code-mode turn never fails because the reducer is down.
- A size gate (16 KiB default) so ordinary results stay zero-latency, and a
20s total round-trip budget. The reducer is on the synchronous path to model
output and there is no `DEFAULT_EXEC_COMMAND_TIMEOUT_MS` bound to inherit, so
the number is chosen: model-backed reduction over a 16 KiB..32 MiB payload
lands in single-digit seconds, and because the failure mode is a fallback,
hitting the bound costs latency only.
- `features.code_mode.max_output_tokens_ceiling`, a host clamp on the budget.
`max_output_tokens` / `max_tokens` arrive from the model with no operator
lever today, so a model can request a budget large enough to make any
reduction pointless. The ceiling binds the built-in default too.
Replacement content is fenced as untrusted data before it enters context. The
reducer's own isolation protects the reducer, not the parent model, and the
parent has tools; hostile file content can propagate through a summarizer as
text aimed at the parent. The fence is not a security boundary, but the seam's
contract is that replacement content is data, and it says so rather than
leaving each host to remember.
Not solved: streaming. This is a terminal reduction at the script-to-model
boundary and does not detect a hung or runaway command mid-stream.
`RuntimeResponse::Yielded` means long-running scripts pass intermediate
results through this seam, which partially covers the case, but no more than
that is claimed.
Tests drive a real loopback server rather than a stubbed trait, covering:
reducer absent (identical to today), healthy, timing out, malformed body,
invalid content items, error status, auth mismatch, below-threshold (never
contacted), missing descriptor, request shape, and the budget clamp.
Three gaps found by reviewing whether this seam actually meets the need. The reducer was summarizing blind. It received the content items and some ids but not the script that produced them, so gpt-5.6-luna had no way to know it was looking at a repeated stack frame versus three thousand `rg` matches. `ReductionContext` now carries the script source. `wait` resumes a cell it did not start, so `CodeModeService` remembers the source per live cell and drops it once the cell can produce no more output. The map is only populated when a reducer is configured, so the default path allocates nothing. The config -> CodeModeService -> live reducer path had no coverage at all, and it fails silently: if the HTTP client cannot be built, the service stores `None`, logs a warning, and the feature is inert while every unit test still passes. `core/tests/suite/code_mode_output_reducer.rs` runs real code-mode turns against a mocked Responses API and a stub bridge, asserting on the payload the model is actually sent: unreduced without a reducer, replaced and fenced with one, and unreduced again when the bridge is unreachable. It also checks the bridge received the script and a cell id. The CI gate only runs `--lib`, so that test would never have executed here. It now runs as a fourth job scoped to this one integration test by name, rather than pulling in the whole suite, which wants sandbox helpers and exec servers. Also documented what `min_trigger_bytes` actually gates. Reduction is per increment, not per turn -- `wait` returns only output produced since the last yield -- so a chatty script never reaches the reducer and one large burst costs one reduction. 16 KiB is ~40% of the default token budget, so the reducer engages well before truncation would, which is the point.
response_cell_id returned String while the per-cell script map is keyed by CellId, so the lookup failed to compile: expected `&CellId`, found `&String` Return the borrowed CellId and scope the borrow so it ends before the match consumes the response.
Two independent failures from the last run. `integration` failed with `no such command: nextest`. The install and cache steps were gated on `matrix.name == 'test'`, so the new job never got the binary. Gated on the command instead, which cannot drift out of sync when a job is renamed or another nextest job is added. `clippy` failed on `clippy::redundant_clone`, which this workspace denies. `TestCodexBuilder::with_config` takes `FnOnce`, so the reducer config can move into the closure rather than being cloned.
The turn was coming back "unsupported custom tool call: exec", so the script never ran and the reducer never fired. Enabling Feature::CodeMode is not sufficient -- effective_tool_mode reads the model's own tool_mode, so the test now overrides model info the way the working code-mode tests do. Worth noting the test earned its place before it ever passed: it is the only thing in the tree that would have caught the seam being wired up but never reached in a real session.
Two of the three tests were passing while `exec` was not even registered. They asserted on the whole serialized request, which also carries the `custom_tool_call` input -- the script source -- and the script contains the exact text the assertions look for. So "the original reached the model" was satisfied by the model's own outgoing call, not by any output. I chose that shape deliberately for robustness; it was the opposite. Now extracts the `custom_tool_call_output` for call-1 and fails loudly on "unsupported custom tool call", so a session where code mode never activated can no longer masquerade as a pass. Also switched to the model recipe the working code-mode tests use. `with_model_info_override` patches a catalog entry but does not select the model, so the previous attempt left the default model in place.
All three tests failed the new guard with "unsupported custom tool call: exec",
which is the guard doing its job: code mode was never active.
CodeModeSessionProvider::availability checks that the host executable exists on
disk, and test_codex resolves it through cargo_bin("codex-code-mode-host").
`cargo nextest run -p codex-core` does not build workspace binaries, so the file
was absent, availability failed, and code mode fell back to direct tools --
which is exactly the silent degradation the seam is supposed to survive, just
in the wrong place.
The integration job now builds the host first, and the guard names it as the
likely cause so the next person does not have to rediscover this.
Code mode is now active -- the host builds and `exec` registers -- but the extracted output came back empty, so the content assertions failed. `custom_tool_call_output.output` is a bare string only on the error path. When code mode actually runs it is an array of content items, and `as_str()` on that yields None. Now flattens both shapes, and asserts the extraction is non-empty so a future shape change fails as a shape change rather than as a misleading content mismatch.
Run the same barrier-backed Promise.all cell with reduction disabled and with protocol v2 enabled. Assert that both nested operations finish, that the reducer receives their full completed output and configured token budget, and that Codex acknowledges the selected replacement with the matching reduction identity.
Tell reducer-enabled models that reduction happens after a Code Mode cell, reinforce parallel batching after every selected replacement, and expose stable cell/member grouping to PostToolUse hosts. Advertise grouping and continuation guidance versions through app-server capabilities so Token Miser can build grouped selective retrieval and account for Codex-owned context overhead.
Harold Hunt (huntharo)
force-pushed
the
pwragent
branch
from
August 28, 2026 19:15
66ae2cd to
fe71949
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reference snapshot only. Do not merge.
This draft preserves the exact source commit used for the known-working Token Miser dev.8 evaluation so it cannot be lost while polling-cost behavior is investigated.
Exact build source: 2998e36
Feature tip/tree: fbd49ec
Base at build time: f1ed3d5
The merge commit tree is identical to the feature-tip tree. This PR is not proposed product work; it is an immutable forensic comparison point for dev.8 versus dev.12.