Let a large-PR review read its diff and fail honestly past a limit by ppXD · Pull Request #2014 · ppXD/CodeSpace

ppXD · 2026-09-23T12:35:27Z

Summary

Makes a large-PR review read its diff intact, and makes it fail once, with a cause the author can act on, when the diff is past a limit. Each fix is its own commit.

The diff reached the model as escape codes. VariableResolver.cs:137 wrote an object into text with the default HTML-safe encoder, so every CJK character and every + < > & ' became a six-character \uXXXX: 1.07x the bytes for an ASCII C# diff, up to ~2x for a CJK-heavy one. WorkflowJson.InterpolatedText (UnsafeRelaxedJsonEscaping) leaves characters as themselves and still escapes what JSON requires. MapResultsPrompt uses the same options, because its under-budget text must stay byte-identical to the resolver's. An object now behaves like the string branch beside it, which always inserted these characters raw. In a URL query, a header or a single-quoted sh -c string the old escape codes did keep an object's & or ' from reading as syntax. A string there never had that protection, so no template could rely on it; the doc now says so.
A context-window overflow was respawned. It failed as an ordinary non-zero-exit, and a respawn warm-resumes, re-sending the goal inside a longer request. A task-launched node (three default attempts) ended on three identical refusals.
- Each harness folder now types the overflow from fields its CLI writes. It stamps AgentTerminalOutcomeReader.ContextWindowExceededExitReason (context-window-exceeded), and AgentRetryCauses.Classify reads it off the exit reason only. Text never classifies: a text scan also matched a fail-closed rubric about context windows, which switched model escalation off for its failed grade.
- Claude: terminal_reason prompt_too_long (the provider refused), or blocking_limit (the CLI's own estimate is past the window, so it sent nothing — the pinned 2.1.263 does this from ~760 KB on a 200K model), or a 400 api_error relaying a gateway overflow body.
- Codex: its terminal turn.failed carries OpenAI's context_length_exceeded, a gateway overflow message (vLLM's wording, LiteLLM's ContextWindowExceededError, or an Anthropic refusal behind a proxy — one shared marker list, so the two folds agree), or Codex's own streamed-refusal sentence.
- What stays retryable: a 5xx, and a connection failure (api_error_status: null).
- A fold never throws. A throw there lands the run as executor-error and drops its session, diff and transcript.
  - A null status reads as "not an overflow".
  - A relayed body that is not valid text (an unpaired surrogate escape parses, then throws on read) is read through its escaped text. One bad character therefore neither throws nor hides a refusal the rest of the body states.
- AgentCodeNode treats the overflow as deterministic. Its message says the request is larger than the window the CLI or the model allows (the CLI raises blocking_limit before sending anything), and names the author's remedy.
Codex refuses a goal past 1,048,576 characters, and we launched it anyway. CodexHarness.BuildInvocation now refuses first, as the terminal SandboxArgumentTooLongException, counting Unicode scalar values the way Rust does. The cap is harness-owned (Rule 7) and pinned. When the CLI does refuse (input_too_large on stderr), the fold reports the same code rather than advice to pick a larger-window model. A cold revise goal restates the whole contract, so it can cross the cap the original goal fit under. That refusal stops the revision, not the run: the last graded round's result stands, and the timeline says why, the same way a spend refusal stops a revision.
A cost-capped node hid a failed run's cause. Any unpriced attempt failed as "cannot be priced … cumulative spend is missing", including a launch refused for its size, which never had a spend. A failed run now keeps its own message and is never retried. It notes the cap only when the cap is what stopped a retry the node's own policy would have bought. A succeeded run whose cost cannot be determined still fails closed.
Two misreadings on the way out. In the Room, a size refusal counting a 1401234-character goal rendered as "Authentication failed / Fix credentials"; 401 now has to stand alone. On the LLM plane, LlmApiException.Classify labelled both Anthropic overflow bodies BadRequest, so the supervisor brain's reactive compaction never ran for them; they are ContextLengthExceeded now.

Test plan

Resolver: an object with CJK and + < > & ' interpolated into prose keeps them readable (red first). A value embedded in a JSON body still parses and round-trips. The resolver/MapResultsPrompt byte-identity pin gains a CJK branch; mutation-checked.
Overflow typing, through the real ParseEvents + BuildResult. The lines come from real CLIs (Claude Code 2.1.226 and the pinned 2.1.263, Codex 0.147.0 and the pinned 0.142.2), with a dummy key against a local stub, unless noted:
- Anthropic's two prompt_too_long shapes, blocking_limit, and a gateway's 400.
- OpenAI's context_length_exceeded, Codex's streamed rewording, and vLLM (code: 400) and LiteLLM (code: "400") bodies relayed by 0.142.2.
- A LiteLLM refusal in front of an Anthropic model. This one is representative of LiteLLM's exception mapping, not captured from a live proxy.
- Negatives:
  - Codex's real rewrapped 503, 413 and 422 lines. The pin prints these as prose, not a relayed body.
  - A JSON-bodied 5xx on either harness. The Codex one is synthesized, because the pin rewraps a 5xx as prose; it guards only the 5xx check.
  - A crash whose last message quotes an overflow.
  - A fail-closed rubric that talks about context windows; escalation is still offered.
  - Plain text that says "Prompt is too long".
- Mutation-checked: reverting each fold fix reds its cases.
Regressions the typed folds briefly introduced, pinned:
- Claude's real connection-failure lines (ConnectionRefused, ECONNRESET; api_error_status: null) fold to non-zero-exit with the CLI's own text and session id.
- Codex's relayed 400 bodies with an unpaired surrogate escape fold without throwing and keep the thread id. The overflow ones are still typed, including OpenAI's typed code beside an unreadable message; the one that is not an overflow stays non-zero-exit.
- Both threw before their guards; red without them.
Codex cap: pinned at 1,048,576. Exactly at the cap builds, and one past is refused without the goal in the message. A goal over the cap in UTF-16 units but under it in scalar values builds. The CLI's own input_too_large folds to sandbox_argument_too_long.
Revise at the cap, executor level. A goal 500 characters under the cap runs round 0; its cold revision crosses the cap. Round 0's verdict (acceptance-failed), changed files and branch stand, and the timeline says the revision stopped. It was red before: the throw replaced the whole result with a bare size refusal.
Node: an overflow is not retryable and names its cause. On a $5-capped node a size refusal keeps its message (no "cannot be priced"). An unpriced crash keeps its cause, and names the cap only on a node that retries.
Room: a size refusal counting 1401234 characters is not an auth failure (built through the real Codex refusal). Real 401 shapes still are.
LLM plane: both Anthropic overflow bodies classify as ContextLengthExceeded.
Rebased onto main after Measure the whole launch frame, and run an oversized continuation cold #2013 merged, conflicting only in the revise loop: the size refusal now covers both the warm build and the cold rebuild Measure the whole launch frame, and run an oversized continuation cold #2013 added. Full unit suite: 11161 passed, 0 failed. Integration (every AgentRun* suite, agent node, real harness, task launch, Room projector, map, supervisor cost cap / retry escalation, supervisor decider, native launch, local acceptance, session resume, continue-parked, operator cancel): 863/863. Full solution build clean.

Notes

Characters outside the Basic Multilingual Plane (emoji) are still written as surrogate-pair escapes; every built-in encoder does that.
Left as follow-ups, all pre-existing on main:
- Claude's own line parser throws on an unpaired surrogate in a relayed body, before the fold is reached.
- On a Custom gateway, the Claude CLI assumes a 200K window for any non-claude model unless CLAUDE_CODE_MAX_CONTEXT_TOKENS is set, so blocking_limit fires at 200K there. The harness does not pass the credential model's declared window yet.
- A task-launched judge rubric inlines the whole goal.
- Gemini/Vertex overflow wording is not typed.
- The reconciler's recovered-from-spool and marker-only re-attach paths do not re-fold events.

A PR-review template binds {{nodes.fetch_diff.outputs.files}} into prose. VariableResolver wrote an object or array into surrounding text with the default HTML-safe encoder, so every CJK character and every + < > & ' of the diff reached the model as a six-character \uXXXX: "修复" arrived as 修复 and every added line began +, at up to twice the bytes. The model was reviewing escape codes. The string branch two lines above already inserted all of those characters raw, so the escaping never protected anything; it only made objects and strings disagree. WorkflowJson.InterpolatedText leaves characters as themselves and still escapes what JSON requires — the quote, the backslash, control characters — so a value embedded in a JSON body stays parseable. MapResultsPrompt, whose under-budget text must stay byte-identical to the resolver's, uses the same options; its identity pin gains a CJK branch, since plain ASCII could not tell two encoders apart. Characters outside the Basic Multilingual Plane are still escaped: no built-in encoder lifts that.

When the model refused a request as larger than its context window, the run failed as an ordinary non-zero exit and the agent.run node respawned it. A respawn warm-resumes the conversation, so its request carries the goal again and overflows harder; a task-launched agent node, which defaults to three attempts, ended on three identical billed refusals. The text the author saw was the CLI's advice to its interactive user — trim your tools, start a new thread — which a workflow author cannot act on. AgentRetryCauses gains ContextWindowExceeded, matched against the phrases the CLIs themselves print, each observed from the real binary answered with the provider's own error body: Claude's "Prompt is too long", a gateway's "maximum context length is", OpenAI's context_length_exceeded and its message, Codex's rewording of a streaming failure, and Codex's own input_too_large. The tests run those real lines through ParseEvents and BuildResult before classifying, so the markers are pinned to what a failed run actually carries. The node treats the cause as deterministic unless a stronger model is on offer, and names it with the author's remedy. The cause carries no mitigation. The escalation trigger only escalates after a failed acceptance grade, which an overflowing run never reaches, and every other consumer keys on the format-fault cause specifically.

codex-rs refuses a turn/start input longer than 1,048,576 characters with input_too_large before any model request, and says so on stderr only — verified against 0.142.2, the worker's pin, and 0.147.0. The platform provisioned a sandbox and cloned the repository for a goal Codex would reject in the first second, and the stdin preflight allows up to 8 MiB encoded, well past that cap. The limit belongs to this CLI, so CodexHarness owns it (Rule 7): BuildInvocation refuses a longer goal with the same terminal SandboxArgumentTooLongException an argument the kernel cannot take gets, naming the goal's size and the cap and never the goal. Counted as Unicode scalar values, the way Rust counts a string's characters, so an emoji-heavy goal Codex accepts is not refused on its UTF-16 length. The cap is pinned by a test and moves by PR with the CLI; the context-window classifier also recognises the CLI's own refusal as the backstop.

The overflow cause was a substring scan over the run's error text. A fail-closed acceptance verdict overwrites that text with the rubric's own requirement and the judge's evidence, so a review whose rubric is about context windows classified as an overflow - and the escalation trigger, which stands down for any classified cause, stopped offering the stronger model the failed grade was evidence for. A crash whose last agent message quoted an overflow read the same way. Each harness folder now decides from what its CLI wrote: Claude's terminal_reason (or a 400 api_error relaying a gateway overflow body), Codex's terminal turn.failed (the provider's typed error code, or its own streamed-refusal sentence). The fold stamps a dedicated exit reason, and the classifier reads it off the exit reason only; text never classifies an overflow. A 5xx whose body mentions context length stays an ordinary, retryable failure. Codex refusing an input past its own character cap is not the model's window, so it folds to sandbox_argument_too_long - the code the harness preflight already throws - instead of advice to pick a model with a larger window, which could not help.

On a node with maxCostUsd, any attempt without a cumulative spend failed as "cannot be priced ... cumulative spend is missing", whatever killed it. A launch refused for its size never starts a process, so it never has a spend to read, and that sentence replaced the one telling the author to shorten the goal. Only a succeeded run still fails closed on its price, since its output would otherwise escape the cap unaccounted. A failed one keeps its own message and is never retried - which the retry's own prior-spend check would decide one attempt later anyway - and says so only when that is what stopped a failure a respawn could otherwise change.

The auth heuristic matched "401" as a substring, so a size refusal reporting a 1401234-character goal rendered as "Authentication failed" with a Fix credentials action, sending the author to rotate a working key. The digits now have to stand alone.

The doc claimed the HTML-safe escaping protected nothing. In a URL query, a header or a single-quoted sh -c string it did keep an object's & or ' from reading as syntax - protection a string value there never had, so no template could rely on it. State that instead.

The overflow predicate read api_error_status with TryGetInt32, which throws on a non-number. Claude writes "api_error_status": null on every connection-level failure - ConnectionRefused, ECONNRESET, a dead gateway, a broker listener that went away - so BuildResult threw on exactly those runs. The run landed as executor-error with a .NET message, and its diff, transcript, usage and session id were never captured; the real-model gates read that code as a code fault. Guard the kind first. Pinned with the pinned 2.1.263's own lines.

Claude refuses locally, sending nothing, once its own token estimate is past the window: terminal_reason "blocking_limit", result "Prompt is too long" (the pinned 2.1.263 does this from ~760 KB on a 200K model). It folded to non-zero-exit and was respawned with no cause named. Codex relays a vLLM or LiteLLM refusal verbatim, with the code as 400 or "400" and the reason only in the message, so it matched neither OpenAI's typed code nor Codex's own sentence - while the Claude fold typed the same gateway body. The gateway marker list now lives beside the exit reason and both folds read it, only off a relayed provider body. A 5xx stays retryable on both.

An unpriced failure is never retried under a cap, and the message said so - on every node, including one whose own policy allows a single attempt, where the cap decided nothing and sent the operator after pricing for a retry that could not have happened. The note now needs the node's retry policy too; the verdict is unchanged.

Neither "prompt is too long: N tokens > M maximum" nor "input length and max_tokens exceed context limit" matched a needle, so both were BadRequest. The supervisor brain's reactive compaction keys on ContextLengthExceeded and never ran for an Anthropic model whose window the operator had not declared, and every llm.complete overflow was labelled a malformed request.

A gateway's error text can be well-formed JSON that is not valid text: an unpaired surrogate escape (a preview cut in the middle of an emoji) parses, then throws on read. Codex relays such a 400 body verbatim, and the fold read its message to look for an overflow, so BuildResult threw on every such line, overflow or not. The run landed as executor-error and its session, diff and transcript were dropped - the same shape as the null-status throw, one field over. An unreadable body is now simply not a refusal the fold can type. The Claude fold's string reads get the same guard. Pinned with the pinned 0.142.2's own relayed lines.

The shared gateway markers matched a LiteLLM refusal only when the upstream was OpenAI-shaped. In front of a Claude or Bedrock model LiteLLM carries its own class name and the upstream's words, so a Codex agent on such a proxy was respawned into the same refusal. The list gains LiteLLM's ContextWindowExceededError and Anthropic's two refusals, still read only off a relayed 400 body. The Codex negative that stood for a 5xx was a line the pinned 0.142.2 never prints: it rewraps a 503, 413 or 422 as prose. The real lines are pinned now, and stay retryable whatever they mention.

blocking_limit shares the exit reason with a provider's refusal, but the CLI raises it before sending anything, measured against the window it believes. The node's cause suffix said the model had refused the request. It now says the request is larger than the window the CLI or the model allows, and the classifier's and exit reason's docs say the same.

Guarding the unreadable-message throw dropped the whole relayed body, including a code already read. A gateway refusal carrying OpenAI's typed context_length_exceeded beside a message with an unpaired surrogate escape was typed as an ordinary, retryable failure, and so was a vLLM overflow whose ASCII wording was intact. An unreadable string now stands in as its escaped text. Every word the folds look for is ASCII, which an escape cannot split, so one bad character neither throws out of the fold nor hides what the rest of the body states. A lookup that cannot unescape a gateway-authored key is still "not a refusal", never a throw.

A cold revise goal restates the whole contract, so a goal just under Codex's 1,048,576-character cap produces a revision past it. The harness refused that revision while the executor built it, and the throw left the revise loop: the generic catch replaced round 0's graded result - its diff, summary and verdict - with a bare size refusal, as if nothing had run. The refusal is about the round, so it now ends the revise loop the way a spend refusal does: a timeline note says why, and the last graded round's result stands. Nothing needs settling, because the round was never admitted.

ppXD added 16 commits September 25, 2026 22:37

ppXD force-pushed the fix/let-a-large-pr-review-read-and-fail-honestly branch from 544d562 to 6d49c8c Compare September 25, 2026 14:55

ppXD merged commit ee58293 into main Sep 25, 2026
6 of 7 checks passed

Navigation Menu

Repository navigation

Let a large-PR review read its diff and fail honestly past a limit - #2014

Let a large-PR review read its diff and fail honestly past a limit#2014
ppXD merged 16 commits into
mainfrom
fix/let-a-large-pr-review-read-and-fail-honestly

ppXD commented Sep 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ppXD commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Notes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ppXD commented Sep 23, 2026 •

edited

Loading