[chethanuk]

What

• The LLM classifier judge now drops complete <system-reminder>, <hindsight_memories>, <hindsight_recall_query_rewrite> and <hindsight_chunks> blocks from user messages in its own copy of the request. A user message left with no other text is skipped when picking the opening task, the latest follow-up and the recent_turn_window.
• If no user message has other text, the judge sees the first such message unchanged.
• The selected model still gets the full request. The copy is built in TaskInput::messages from &Request, so the request itself is not touched.
• Documented in docs/routing_algorithms/llm_classifier_routing.md.

Why

Agent clients inject these blocks into user turns. The judge read them as the user's question, and a follow-up holding only a block was picked as the latest message and displaced the real one. On the base, open, real q, then a lone <system-reminder> turn gave the judge open plus the reminder. With this change it gets open and real q.

None linked in this fork PR.

Notes for reviewers

• Start at task_messages and TaskInput::messages in crates/libsy/src/algorithms/llm_class.rs. The one new test, judge_reads_the_question_without_injected_context, is a 27-row table run through the classifier route.
• The design has no maintainer sign-off. A maintainer comment on the issue said the current behaviour may be correct, since recalled memory is also context for the router. The tag list is fixed and not configurable. Other tags are not covered, and a complete block quoted by a user is removed too.
• Limits: a request whose only user message is a tool result plus a reminder gives the judge an empty or instruction-only payload, where the base would send the reminder. With fail_open = false that empty payload is sent as is; I read in the code that a provider rejecting it would stop the run, but did not run it. Decision judges read the same messages, but no test covers them. The escalation judge path is not changed.
• A turn holding only hook blocks still reruns the judge on classify_trigger = "user_turn" routes, because is_user_turn is unchanged. A lone <system-reminder> follow-up calls the judge again; a lone <environment_context> follow-up does not. This is not a regression.
• Checks run: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace all exit 0.
• Gate substitutions. This machine cannot reach Docker, so the Python suite does not pass cleanly on either side:
  • uv run pytest tests/ -v: branch exit 1, clean base exit 1. It stops at the first docker failure, test_closed_book_proxy_allows_switchyard_service_and_blocks_cheat_source, the same id on both.
  • uv run pytest tests/ -v -o addopts="": branch exit 1 (2 failed, 157 passed), clean base exit 1 (2 failed, 157 passed). The failing ids are identical on both: test_closed_book_proxy_allows_switchyard_service_and_blocks_cheat_source and test_open_book_proxy_allows_raw_github_hostname. In gate.round-4.log both failures show "permission denied while trying to connect to the docker API" on the docker socket, on the branch and on the base.
  • Not run: mkdocs, live providers.

[codeant-ai]

  CodeAnt Review Status

✅  Review completed

Status	Commit	Started (UTC)	Finished (UTC)
✅  #bc8fe Reviewed your PR	6f81b68	Oct 31, 2026 · 05:32	05:38

[codeant-ai]

---

Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

[codeant-ai]

CodeAnt-AI Description

Route requests using the user's question, not injected context

What Changed

• The routing judge ignores complete agent-added reminder and memory blocks when reading user messages.
• Hook-only turns no longer replace the actual task or latest follow-up; if no other user text exists, the first hook-only message is kept.
• The selected answer model still receives the full original conversation.

Impact

✅ More accurate request routing
✅ Real follow-ups stay in routing context
✅ Full conversation remains available to the answer model

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

[github-actions]

PR Preview Action v1.8.2
🚀 View preview at https://chethanuk.github.io/Switchyard/pr-preview/pr-15/
Built to branch gh-pages at 2026-10-31 05:34 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[chethanuk]

Review of 6f81b68 (4 commits over goal-13-base, bdd7647).

Result: the code is fine to keep as is. The PR body needs two corrections and one tidy-up before this goes further. No code change requested.

PR body

1. Wrong sentence. "Tests are a table through the public algorithm, plus one through the classifier route." The branch adds one test, judge_reads_the_question_without_injected_context. It is the table, and it goes through the classifier route. There is no second test. It looks left over from the commit that moved the table onto the route. Suggested: "Tests are one table, run through the public algorithm."

2. Missing limit. A hook-only turn still reruns the judge on classify_trigger = "user_turn" routes, because is_user_turn in util/affinity.rs is unchanged. The row's design says to state this in the PR, and the body does not. I ran it with a session request, opening turn then follow-up:

   • real follow-up: judge called again
   • lone <system-reminder> follow-up: judge called again
   • lone <environment_context> follow-up: judge not called, target reused

   So the lone reminder no longer displaces the question, but it can still spend a judge call and release the held target mid-task. This is not a regression. Say it in the Limits line.

3. Tidy-up. "I did not open the other failure text" can go. gate.round-4.log shows permission denied on the docker socket for both failing ids, on both branch and base, in the -o addopts="" runs. The body can say that directly.

The rest of the body matches the evidence. From gate.round-4.log, headed at 6f81b68:

• fmt, clippy and cargo test --workspace recorded exit 0
• pytest tests/ -v stops at the same docker id on branch and base
• -o addopts="" gives 2 failed, 157 passed on both, with the same two ids

The before-and-after example (open, real q, lone reminder) matches the run on the base export recorded in regate.md.

Code

Read strip_injected, is_injected_text, is_injected_only, trim_messages, task_messages and TaskInput::messages. I found no wrong output on the paths the body describes.

• The judge's copy is built from &Request. All 27 table rows assert the answer request is unchanged.
• Cow::Owned as "a match happened" holds for replace_all, and the tagless row pins it.
• Stripping runs only on Role::User messages. An assistant message holding a reminder stays, and a row covers it.
• window_start runs after hook-only turns are filtered out. A tool pair stays whole and only the reminder beside the result goes.
• regex was already a libsy dependency. No new public item or config field.

New check: the rows start from protocol Message values, and earlier rounds did not run a wire body. I decoded these through switchyard_translation::decode_request and drove the result through the classifier on a scratch copy. In every case the judge got the expected text and the answer request was unchanged.

Wire format	Input	Judge got
Anthropic	reminder and question as separate text blocks, with cache_control	real q
Anthropic	one string, reminder then question	real q
Anthropic	open, real q, tool call, then tool_result plus reminder	open, real q
Anthropic	open, tool call, then tool_result, reminder and new q	open, new q
Chat	string messages with a lone reminder last	open, real q
Chat	reminder and question as text parts	real q
Responses	input_text messages with a lone reminder last	open, real q

The bodies are hand-written to the wire format. They are not captured client traffic, so whether a given client sends these shapes is still unproven.

Non-blocking

• Decision judges have no stripping test, and the body says so. Adding one hook block to the user message in decision_judge_routes_on_relative_advantage would close that. It is worth doing if this branch is rebased over other changes to TaskInput::messages.
• docs/routing_algorithms/llm_classifier_routing.md does not say that mode = "escalation" is outside this behaviour. The body does. One sentence in the docs would help readers of that page.
• With fail_open = false, the tool-result-plus-reminder-only case sends the judge an empty list on the default path. From reading the code, a provider rejecting that request would stop the run. The body lists the empty payload but not this consequence. A bare tool result already gives the empty list on the base.

Scope and text

• Two files changed. 83 non-test lines as recorded in the gate log, which is under the 100 line bar. I did not recount them.
• The title, body and all four commit messages have no issue number, closing keyword, upstream link or mention. All four commits have Signed-off-by.

Ran and not run

Ran on a git archive of 6f81b68: cargo test -p switchyard-libsy --lib --offline llm_class, exit 0, 32 passed (30 from the branch plus my 2 print-only probes). The probes added a test-only path dependency on the translation crate in the scratch copy.

Not run by me: cargo test --workspace, clippy, fmt, pytest, mkdocs, a real judge model. For those I rely on the gate log above.

CI on this commit at 05:36 UTC: Rust, Tests (Python 3.12), Tests (Python 3.14), Wheel and Slim install were still pending. All other checks had passed, apart from Deploy to GitHub Pages, which was skipped. Check those five before relying on this review, since the Python jobs are the first run that may have Docker access.

[codeant-ai]

  CodeAnt Nitpicks

2 code suggestions

1. When a user quotes a complete matching tag, this pattern removes the quoted content from the judge's copy, so routing evaluates a different task than the selected model receives.

Incorrect condition logic · crates/libsy/src/algorithms/llm_class.rs:101-104

---

2. is_injected_only rejects messages containing a ToolResult, so a tool-result-plus-reminder-only request misses the fallback and sends the judge no task or only routing instructions.

Incorrect condition logic · crates/libsy/src/algorithms/llm_class.rs:128-132

[codeant-ai]

  CodeAnt PR Risk

🟢  Low Risk

• The PR appears safe to merge; no actionable issues were identified in the review.

Assessed commit: 6f81b68d47d0