fix(aider): stream Anthropic requests to avoid the 10-min non-streaming cancel - #8
Merged
Merged
Conversation
…ng cancel aider configured the Anthropic coder with stream=False + max_tokens=64000. Anthropic cancels non-streaming requests whose generation can exceed its ~10-min server cap; the large floor-lib generations (chardet, voluptuous, marshmallow, jinja) hit it and failed with httpcore.RemoteProtocolError 'Server disconnected without sending a response' (elapsed_s~600 = the cap), producing empty patches. Small/fast libs were unaffected -- which is why the 7 cells that ran on 2026-06-04 succeeded and these 4 did not. Root-caused via a $0 httpx request-capture diagnostic: direct litellm calls of every size (up to 200k tok) + caching + streaming all succeeded; only aider's max_tokens=64000 + stream=False combo failed. Fix: stream=True. Validated 2026-06-08 -- re-ran the affected Sonnet cells full-suite via score_branch: voluptuous 133/149 (89.3%), marshmallow 1073/1229 (87.3%), chardet 250/376 (66.5%); jinja remains n/a (its ~170k-tok context + 64k max_tokens exceeds Sonnet's 200k window -- a separate, fundamental limit). Also adds a KAIZEN_AIDER_CACHE env toggle (default on; tested off, not the cause), and documents the diagnosis + corrected 10/16 Aider-Sonnet count in RERUN_CHECKLIST.md / CORRECTIONS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses deterministic failures when running the Aider baseline against Anthropic Sonnet for long generations by enabling request streaming, and updates the re-validation documentation to reflect the corrected rerun status and root-cause diagnosis.
Changes:
- Enable streaming in the Aider runner to avoid Anthropic’s ~10-minute non-streaming server cap for long generations.
- Add an env toggle (
KAIZEN_AIDER_CACHE) to disable Aider prompt caching while keeping it enabled by default. - Update
CORRECTIONS.mdandcommit0/RERUN_CHECKLIST.mdto document the diagnosis, fix, and updated full-suite re-validation status.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
commit0/baselines/aider/_aider_runner.py |
Switches Aider to streaming requests and adds an env-controlled prompt-cache toggle. |
commit0/RERUN_CHECKLIST.md |
Documents the Anthropic streaming root cause/fix and rerun outcomes. |
CORRECTIONS.md |
Updates the corrected status/numbers for Aider × Sonnet and references the streaming fix. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+325
to
+326
| cache_prompts=(os.environ.get("KAIZEN_AIDER_CACHE", "1") != "0"), # native Anthropic prompt cache; KAIZEN_AIDER_CACHE=0 disables | ||
| stream=True, # MUST stream on Anthropic: non-streaming requests are cancelled at the ~10-min server cap, and large-generation cells (chardet/voluptuous/marshmallow/jinja) exceed it -> httpcore.RemoteProtocolError "Server disconnected" (elapsed_s~600 = the 10-min cap). Root cause confirmed via $0 request-capture diagnostic 2026-06-08; see RERUN_CHECKLIST.md. |
Comment on lines
+27
to
+28
| > Tier-A re-run of the 4 aider-Sonnet cells (`chardet, voluptuous, marshmallow, jinja`; | ||
| > `.git/run_tier_a_sonnet.sh`) failed 100% with **`httpcore.RemoteProtocolError: Server |
…pic; drop untracked-script ref - stream: enable only for Anthropic (provider == 'anthropic') rather than unconditionally, so OpenAI keeps its validated stream=False path. The 10-min non-streaming cancel is Anthropic-specific; no need to change OpenAI behavior. - RERUN_CHECKLIST: reference the tracked, reproducible repeat_runner command instead of the untracked .git/ scratch script. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
|
Thanks @copilot — both addressed in 9c6af15:
|
Confirmed — both changes in 9c6af15 look correct. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The aider runner failed 100% deterministically on Anthropic for large-generation libraries, with:
aider spun ~601 s (
elapsed_s ≈ 600), committed an empty patch (sha e3b0c442…), and the valid-rep gate correctly discarded it. This blocked the Sonnet re-validation of the floor-lib Aider cells (chardet, voluptuous, marshmallow, jinja).Root cause (found via a $0 request-capture diagnostic)
An
httpx-layer shim captured aider's outgoing request and diffed it against a working directlitellm.completion. The only meaningful difference: aider sendsmax_tokens=64000withstream=False.Anthropic cancels non-streaming requests whose generation can exceed its ~10-minute server cap — and the large floor-lib generations hit it (the
elapsed_s ≈ 600is exactly that 10-min timeout). Everything else was ruled out as a red herring: key, model (claude-sonnet-4-6),litellm 1.83.14, prompt caching, WSL MTU, and direct litellm calls up to 200k tokens / 683 KB (stream and non-stream) all succeeded. This also explains the pattern — the 7 small/fast Aider×Sonnet cells that ran on 2026-06-04 succeeded; only the 4 long-generation cells failed.Fix
stream=(provider == "anthropic")incommit0/baselines/aider/_aider_runner.py— streaming is enabled for Anthropic only, since it cancels non-streaming requests at its ~10-min server cap. OpenAI keeps its validatedstream=Falsepath (no such cap observed). Also adds aKAIZEN_AIDER_CACHEenv toggle (default on; disabling caching was tested and is not the cause).Validation (2026-06-08)
Re-ran the affected Sonnet cells full-suite via
score_branch, with streaming:jinjaremainsn/a— its ~170k-token repo context + the 64kmax_tokensexceeds Sonnet's 200k window (a separate, fundamental limit; it ran on OpenAI only because of the larger window). Voluptuous Sonnet 89.3% ≈ the OpenAI sampled 88.7% — nice cross-provider agreement.Docs
commit0/RERUN_CHECKLIST.md— full diagnosis + the fix-then-rerun recipe (references the trackedbaselines/sampling/repeat_runner.pycommand for reproducibility).CORRECTIONS.md— Aider×Sonnet updated to 10/16 full-suite-scored (7 + the 3 re-validated; jinja n/a).🤖 Generated with Claude Code