Repository navigation
feat(libsy): re-decide the classifier tier on fresh user turns #347
Description
Problem
llm_classifier classifies a conversation once and, with session_affinity, holds that decision forever. Both halves are wrong for long agentic sessions:
- Classify every turn and a tool chain switches tier mid-flight — the weak tier picks up a conversation the strong tier was three tool calls into, with no idea what it was doing.
- Classify once (
session_affinity) and the opening verdict goes stale. A session that opened "summarise this file" becomes a hard debugging session ten turns later and is still served by the tier chosen for the summary.
The unit that should drive a routing decision is not the request and not the session — it is the user turn. Tool results are the agent continuing work the user already asked for.
Proposal
A followup table that re-classifies when the user asks something new and pins that target across the tool-call turns in between:
[routes.assistant]
type = "llm_classifier"
followup = { refresh = "on_user_turn", recent_turn_window = 4 }| Key | Default | Meaning |
|---|---|---|
refresh |
on_user_turn |
on_user_turn re-classifies on each new user message; pin_once holds the first decision |
recent_turn_window |
unset | Trailing turns the judge sees. At least 1 |
fail_open_tier |
capable |
Tier served when the judge yields nothing usable and nothing is pinned. Mode capability only |
abstain_selector |
unset | JSON Pointer to a boolean abstain flag in the verdict. Mode custom only |
Available on modes capability and custom. Rejected alongside session_affinity, whose latch is first-wins and would prevent every refresh — silently turning on_user_turn into pin_once; use refresh = "pin_once" if that is what you want.
The deliberate asymmetry, which is the part worth reviewing:
- an unusable verdict, or an unreachable judge, holds the pinned target. No evidence is not evidence of change. It falls open only when nothing is pinned.
- in mode
custom, an explicit abstain outranks the pin — the judge declining to decide is itself a decision. Modecapabilityhas no equivalent, because its packaged rubric answers with a solve probability and carries no abstain field. - neither path retries the judge.
Relationship to #298 and #308
These overlap and the difference is intentional. #298's RoutingIdentity and #308's Harbor proxy_x_session_id normalisation both improve who a conversation is; this is about when its tier is re-decided. The pin lives in per-session State rather than the affinity map precisely because AffinityRouter's assignment is first-wins and process-local, so it can never be refreshed — which is the behaviour this issue exists to change.
Where a request carries no session id the pin has nothing to persist in, so it can optionally be keyed on a hash of the first user message, the same fallback session_affinity already offers.
Rebased onto current main and green. Depends on nothing else; sibling issue for the graded-verdict labels map filed separately.
Activity
feat(libsy): replace session_affinity with classify_trigger (#487)
gburachas commentedon Aug 20, 2026
Closing this: #487 implemented the core of it.
classify_trigger = "user_turn"re-runs the classifier when the newest message is from the user and reuses the previous target across the tool-call turns in between, which is what this issue asked for.classify_trigger = "new_session"covers the classify-once behaviour, andrecent_turn_windowon the route covers the judge's view of the conversation. The implementation is incrates/libsy/src/algorithms/util/turn_pin.rs.Two parts of the proposal here are not covered by #487, and are better tracked separately than by keeping this issue open with a config table for a
followupblock that no longer applies:TurnPinclears the retained target on an abstention and the route's default decides. There is no way to say which of the two tiers that should be in capability mode, and in custom mode an explicit abstain cannot be distinguished from an unparseable verdict. I will open a separate issue for this.recent_turn_windowis set. Tracked in feat(libsy): render the judge payload as one condensed block under a char budget #349.Thanks for picking this up.