[chethanuk]

What

• Limit the messages sent to the judge to a fixed 18000 characters when recent_turn_window is set. Long text is shortened in the middle. Tool arguments, images and other blocks that cannot be cut are replaced with a short note. If the messages still do not fit, the oldest window messages are dropped. The opening message stays, though its content can be cut.
• Window start widening for tool pairs now shares one helper (track_tool_pairs), and the drop-oldest step keeps a tool call and its results together.
• MAX_REQUEST_CHARS and truncate_middle in the escalation code become pub(crate) so the windowed path reuses the same 18000 value and the same truncation.
• One row of the recent_turn_window docs table describes the cap and what it can drop or cut.

This is a fixed character cap on the windowed path only. It is not a route-level knob and not a token budget, which others have asked for. Unwindowed requests are unchanged. Routes that set recent_turn_window do change when they go over the cap. For example, a task with a 200000-character base64 image reaches the judge as its text and the note.

Why

With recent_turn_window set, one large tool result or message can push the judge request past what the judge model accepts, and the judge call fails. By default (fail_open = true) the route then picks the capable target without a judge decision. With fail_open = false the request gets the error. The escalation judge already bounds its input at 18000 characters, but the windowed path for capability, custom and decision judges had no bound.

Which issue does this address?

None linked in this fork PR.

Notes for reviewers

Start at cap_window in crates/libsy/src/algorithms/llm_class.rs. Tests cover a single message larger than the cap, a tool call and result at the window start staying paired, one character over the cap, content under the cap staying unchanged, a follow-up surviving four 40000-character results sent in one message, an opening message of only tool results becoming empty text, and the structured judge request.

Test run at 9f94343:

• cargo fmt --all --check and cargo clippy --workspace --all-targets -- -D warnings exit 0.
• cargo test --workspace exits 0, with 1168 passed, 0 failed and 1 ignored. cargo test -p switchyard-libsy --lib algorithms::llm_class gives 34 passed, 0 failed.
• uv run pytest tests/ -v exits 1 at the first failure, because addopts in pyproject.toml sets -x. uv run pytest tests/ -v -o addopts= exits 1, with 177 passed and 2 failed. Both are proxy tests in tests/e2e that got permission denied on the Docker socket, and both also fail on the clean base. A third test, test_interrupt_stops_only_owned_children[2-130] in tests/test_nemo_gym_run.py, hit a 20-second timeout on the clean base and on the previous head, and passed in this run.
• No real provider was called.

Limits. The numbers come from throwaway tests, which are not in this PR. They ran against the same non-test code as this commit, since the last commit changes only tests and docs.

• The 18000 count covers message content only. It leaves out JSON envelopes and the tool ids on results, and the judge system prompt and response format sit outside the cap. In one test, 17710 counted characters were 20300 characters as serialized messages. Characters are not tokens, so the cap is only a proxy for the judge model's context limit.
• About 16 parallel results sent as separate messages leave only the opening task, and the number falls as ids and tool names get longer. The first count that did this was 20 with 6-character ids and search, 18 with 29-character ids and read_file, and 16 with 40-character ids and mcp__filesystem__read_file. Result size from 1000 to 40000 characters made no difference. With all results in one message, 96 results kept the window in all three shapes, and 128 kept it only with the 6-character ids.
• Dropping is oldest first, so a later user follow-up can be dropped. With a task, a reply, a follow-up and then call and result pairs, window 50, the follow-up stays at 3 pairs and is gone at 4 with 40000-character results. With 3000-character results it is gone at 6 pairs. A window of 6 already loses it at 3 pairs by message count, before the cap runs.
• A tool result whose call sits before the opening task, sent as the newest message, leaves only the opening task once the messages are over the cap.
• An opening message of only tool results, too many to fit (400 of 100 characters in a throwaway test, 2000 short results in the test row in this PR), becomes an empty text block and the results are lost. Some providers may reject that. This was not tested against a provider.
• An opening message of many blocks can be cut to the start of its text. A task of 300 text blocks of 100 characters lost its final text, and the cut adds no marker at the end.
• An oversized provider block with no normalized form is replaced with the note. If it is one half of a provider server tool pair, the other half stays. In one test a server_tool_use block stayed and its web_search_tool_result became the note. Whether a provider rejects this was not tested. A rejection fails the judge call, with the outcome described under Why.
• docs/reference/toml_schema.md and docs/routing_algorithms/stage_router_routing.md do not mention the cap. docs/routing_algorithms/llm_classifier_routing.md does. Its row now says a later user message can be dropped, a very large tool group or a newest tool result whose call came before the opening message can leave only the opening message, and the opening message can be cut to the start of its text, or to empty text if it holds only tool results.
• Open PRs that edit the same functions may need a rebase.

[codeant-ai]

  CodeAnt Review Status

✅  Review completed

Status	Commit	Started (UTC)	Finished (UTC)
✅  Reviewed your PR	6a7c3f7	Oct 30, 2026 · 18:09	18:20

[codeant-ai]

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[codeant-ai]

CodeAnt-AI Description

Keep windowed routing requests within the judge’s size limit

What Changed

• Windowed routing requests now stay within a fixed 18,000-character limit, preventing long conversations or tool output from exceeding the judge’s input size.
• Long text is shortened, oversized tool arguments and content that cannot be shortened are replaced with a note, and older messages are dropped if needed. The opening task and complete tool call/result pairs are retained.
• Requests without a conversation window are unchanged.

Impact

✅ Fewer routing failures on long conversations
✅ Opening task retained when older history is trimmed
✅ Oversized images and tool arguments no longer block routing

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[github-actions]

PR Preview Action v1.8.2
🚀 View preview at https://chethanuk.github.io/Switchyard/pr-preview/pr-13/
Built to branch gh-pages at 2026-10-30 18:52 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[chethanuk]

Independent review of 6a7c3f7 against base 3fbfdff. This is a comment review because GitHub does not allow approving your own PR.

Verdict: REVISE, PR body only. I found no code defect that must block this fork PR. The cap and tool pairing held in everything I ran. Two limits in the body read smaller than what I measured, and three known limits are missing from it.

Fix in the PR body

1. "Twenty or more parallel tool results" only holds for very short ids. The test fixtures use 6-character ids. With results sent as separate messages, a follow-up before the group, and results of 1000 to 40000 characters, I measured:

   Call shape	Last N that kept the window	First N that left only the task
   6-char id, search	19	20
   29-char id, read_file	16	18 (17 not run)
   40-char id, mcp__filesystem__read_file	15	16

   The result size made no difference. The body should say about 16, and that the number falls as ids and tool names get longer. With all results in one message, N up to 64 kept the window.

2. "A long run of large tool turns" understates the follow-up loss. With [task, "ok", follow-up, P call and result pairs of 40000 chars] and a window of 50, the follow-up stays at P = 3 and is gone at P = 4. Four large reads after a user message is ordinary for a coding agent. A small window such as 6 already loses the follow-up at 3 pairs before the cap runs, so the cap adds this loss only for larger windows.

3. Add these limits. They are missing from the body:

   • The count leaves out JSON envelopes and the tool ids on results. In one probe, 15551 counted characters were 17840 characters as serialized messages. The judge system prompt is also outside the cap, so "judge payload" means the messages only.
   • A result whose call sits before the opening task, sent as the newest message, leaves only the task, because start never moves at llm_class.rs:230. I read this from the code and did not run it.
   • An opening message with only tool results, too many to fit, becomes an empty text block (llm_class.rs:211-212, unwrap_or_default). Some providers may reject that. I read this from the code, did not run it, and did not call a provider. It is also the one case where "the opening task is always kept" does not hold. The input is unusual.

Non-blocking

• Replacing an oversized unknown block with a note could split a provider's server tool pair, and no provider was tested. I did not test it either.
• Any window message over room / (group + 1).max(4) is shortened even when dropping one old message would have been enough. The newest message can be cut to about 4470 characters this way. This follows from the design, and it is a tradeoff to know about.
• A task just over the cap is cut to about half even when little else needs the room (llm_class.rs:206).
• Results in one message get far less than the room: 10 results of 5000 characters came out at 5120 characters in total.
• docs/reference/toml_schema.md describes recent_turn_window in several rows and does not mention the cap. The routing doc does. Adding it there would pass the 100-line limit.
• 506 test lines for 96 non-test lines in llm_class.rs may draw a request to trim. The rows do test distinct behavior through TaskInput, so I would keep them. The table has no row for a realistic id length, an orphan result as the newest message, or the empty text fallback.

What I checked

• Correctness: cap_window runs only when a window is set. start moves only when no result is waiting for a call, so no kept result loses its call. used stops at room, and the opening message is held to half the cap, so the content total stays within WINDOW_CHARS. share cannot divide by zero, and drain cannot invert. The window walk and track_tool_pairs keep the old block order.
• All 4 uses of TaskInput go through messages, so the capability, custom and decision judges share the cap. trim_messages has no other caller.
• Scope: 3 files. Non-test additions are 96 in llm_class.rs, 2 in escalation.rs and 1 in the docs, 99 in total. It stays under the limit by one line, and any fix to the first item above would need a matching removal.
• Title, body and commits: I searched them, and the added lines, for issue numbers, close keywords, upstream links and mentions. I found none. Both commits are one line with a DCO sign-off and no other trailers. The body keeps the What, Why and Notes template sections and says no issue is linked.

Commands run

• cargo test -p switchyard-libsy --lib algorithms::llm_class: 34 passed, exit 0. This matches the body.
• cargo test -p switchyard-libsy --lib: 443 passed, exit 0.
• cargo clippy -p switchyard-libsy --all-targets -- -D warnings: exit 0.
• cargo fmt --all --check: exit 0.
• git merge-tree against the latest upstream main, 5 commits ahead of this base: clean, exit 0.
• PR checks: all passed, and the Pages deploy was skipped.
• Probes: 4 throwaway print tests in a scratch copy with its own build directory, exit 0. They are not in this PR.

Not run locally: cargo test --workspace, pytest, or any real provider. CI ran its own Rust and Python jobs.

[codeant-ai]

Suggestion: Tool result IDs and error flags are uncounted, so many empty results can make the forwarded judge messages exceed the intended character cap.

Assessment: 🟠 Major · 🔁 Occurrence: Rarely · 🏷️ Incomplete implementation

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/libsy/src/algorithms/llm_class.rs
**Line:** 172:172
**Comment:**
	*Incomplete Implementation: Tool result IDs and error flags are uncounted, so many empty results can make the forwarded judge messages exceed the intended character cap.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

Suggestion: A long Refusal contains text that could be shortened, but this branch replaces it entirely and loses that context from the windowed judge.

Assessment: 🟠 Major · 🔁 Occurrence: Rarely · 🏷️ Logic error

Prompt for AI Agent 🤖

This is a comment left during a code review.

**Path:** crates/libsy/src/algorithms/llm_class.rs
**Line:** 187:192
**Comment:**
	*Logic Error: A long `Refusal` contains text that could be shortened, but this branch replaces it entirely and loses that context from the windowed judge.

Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix

👍 | 👎

[codeant-ai]

  CodeAnt PR Risk

🟡  Medium Risk

• The PR needs attention before merging because the character cap does not count tool-result IDs or error metadata, so many empty results can exceed it.
• Long refusal content is replaced with a generic note instead of shortened, losing context the judge may need.

Assessed commit: 6a7c3f7b4647