Add tuned-config retrial levers: respect-requested-model, static cluster pin, tool_result tier ceiling - #902
Conversation
…ter pin, tool_result tier ceiling Bundles three independent, default-off env-flag-gated features built for the weave-router-tuned-config-retrial evaluation (S-133259) — a re-trial of the router against real agent traffic to see whether tuning these levers changes the prior DROP verdict: - ROUTER_RESPECT_REQUESTED_MODEL: serve a client's own requested model verbatim when it's on the operator's honor list, bypassing planner/scorer. - ROUTER_STATIC_CLUSTER_PIN: pin a request's nearest cluster directly to a configured model, skipping blendScoresV2, to measure pinning against the scorer's own latency overhead. - ROUTER_TOOL_RESULT_TIER_CEILING: exclude every model above the turn's requested-model tier, applied only to ToolResult turns before the scorer runs, leaving MainLoop free to upshift. Each flag defaults to false/off and is independently toggleable; none change default routing behavior on their own.
|
PR author is not in the allowed authors list. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 3 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 54d7f3a. Configure here.
| res.HardPinned = true | ||
| res.PinTier = reasonRequestedModelRespected | ||
| return res, nil | ||
| } |
There was a problem hiding this comment.
Force-model loses to honor path
High Severity
honoredRequestedModel returns before session-pin lookup, so a live ReasonUserForceModel pin is never consulted when the client's RequestedModel is on the honor list. Explicit /force-model and x-weave-force-model choices are silently overridden for that turn.
Triggered by learned rule: Force-model and user-facing model commands must resolve via catalog, not assume default provider
Reviewed by Cursor Bugbot for commit 54d7f3a. Configure here.
| } | ||
| } | ||
| return out | ||
| } |
There was a problem hiding this comment.
Tier ceiling undone by fit-check
High Severity
applyToolResultTierCeiling only augments ExcludedModels, but the session-pin fit-check treats non-policy entries there as liftable context-overflow exclusions. An above-ceiling sticky pin that still fits the window is pruned back into eligibility and reused, defeating the documented ceiling.
Additional Locations (1)
Triggered by learned rule: Context window pre-filter must be wired in both proxy surfaces using FullTokenEstimate
Reviewed by Cursor Bugbot for commit 54d7f3a. Configure here.
| "excluded_count", len(excluded)-before, | ||
| ) | ||
| } | ||
| } |
There was a problem hiding this comment.
Ceiling missing on OpenAI path
Medium Severity
applyToolResultTierCeiling is applied only in ProxyMessages before building ExcludedModels. ProxyOpenAIChatCompletion still builds its exclusion set from context-overflow and Gemini filters alone, so ToolResult turns on that surface ignore ROUTER_TOOL_RESULT_TIER_CEILING.
Additional Locations (1)
Triggered by learned rule: Context window pre-filter must be wired in both proxy surfaces using FullTokenEstimate
Reviewed by Cursor Bugbot for commit 54d7f3a. Configure here.
|
Thank you so much for this — the three levers here are genuinely useful for routing experiments, and the design instinct behind them (default-off, documented, one env var per lever) is exactly right. I've rewritten the change to match some internal conventions and rebased it onto current Please don't read this as anything you did wrong. Most of what follows comes from conventions that live in our Main things applied, in case they're useful next time:
Really appreciate the contribution — hope you'll send more. |
1 participant
Summary
Bundles three independent, default-off env-flag-gated features built for the
weave-router-tuned-config-retrialevaluation — a re-trial of the router against real agent traffic to see whether tuning these levers changes a prior DROP verdict.ROUTER_RESPECT_REQUESTED_MODEL— serve a client's own requested model verbatim when it's on the operator's honor list, bypassing planner/scorer.ROUTER_STATIC_CLUSTER_PIN— pin a request's nearest cluster directly to a configured model, skippingblendScoresV2, to measure pinning against the scorer's own latency overhead.ROUTER_TOOL_RESULT_TIER_CEILING— exclude every model above the turn's requested-model tier, applied only toToolResultturns before the scorer runs, leavingMainLoopfree to upshift.Each flag defaults to
false/off and is independently toggleable; none change default routing behavior on their own. Documented indocs/CONFIGURATION.md.Test plan
go build ./...go vet ./...go test ./...(full repo, clean)honoredRequestedModel,applyToolResultTierCeiling, andstaticClusterPinFor