opensecret: health-aware Auto model selection for Router V2 (#968)

* opensecret: health-aware Auto model selection for Router V2

Router V2 resolved an Auto alias to one public model before any health
check, so a single-route model with no eligible provider left Auto Quick
with nothing to do. Add a small compiled Auto model policy that runs after
alias resolution and before any model-dependent work: paid Quick may move
from DeepSeek V4.1 Flash to GLM 5.3 Flash and paid Powerful from GLM 5.3 to
Kimi K3 only when every route of the preferred model is unavailable under
that model's own failover policy. Free tiers keep their single target,
explicit selections never change public model, and Router V1 is untouched.

The decision consumes one health snapshot across all candidate routes,
claims nothing, and feeds the chosen model into the existing same-model
planner, first-send claim, tool-loop pinning, persistence, and usage. An
alternate is skipped when the request needs more context than its window
(Chat uses a text-only estimate with a byte bound before tokenizing, plus
tool schemas and the caller's output limit), when it lacks vision the
request needs, or, for Kimi on Chat, when tool-call ids are not in Kimi
form. Before any provider send, a chosen Auto model that cannot hold the
request's context or loses its routes earns exactly one further decision
with that model excluded; nothing is ever replayed after a send starts.

Router V2 also remembers the route a provider accepted per account,
surface, and selector for one idle hour so warmed prompt caches are reused:
the planner prefers the remembered same-model provider while eligible, and
Auto retries the remembered alternate before the preferred model while it
stays healthy. Responses now binds the user key, conversation ownership,
the usage gate, and the caller's message against the tier's own model
before the decision; its input handling is split so the budget check and
both context builds use the final model.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* opensecret: keep the capacity result when Auto reselection has no alternate

A bounded second Auto decision excluded the lost model as a context
overflow even when the cause was a route that became unavailable after
the first snapshot. With no other candidate (free Quick, or a paid tier
whose alternate is incompatible) the selector reported no health rejection
and the request ended as a 500 instead of the typed capacity result, losing
the retry hint although nothing had been sent.

Carry the exclusion cause: a lost route is excluded as unavailable with the
capacity result's own retry hint, so the second decision reports it and
keeps the 503 contract and Retry-After; a context overflow stays distinct.
The Responses replay-safety downgrade after billed image helpers still
applies. Tests cover free single-candidate Auto and paid Auto with an
incompatible alternate on both surfaces, the hint fallback, the compatible
alternate still being taken once, and the helper downgrade.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>

Author: AnthonyRonning

.agents/skills/change-opensecret-provider/SKILL.md

@@ -67,10 +67,22 @@ preferences. Keep image and title helpers on the request's appropriate routing
 context. A helper candidate identifies a public model; resolve its provider
 through shared routing rather than pinning an independently guessed provider.
 
-Trace alias resolution separately from provider selection. Current V2 failover
-stays within the resolved public model; an Auto alias alone does not implement
-cross-model failover. Do not infer inactive behavior from historical `Shadow`
-type names: active V2 selection consumes health snapshots.
+Trace alias resolution separately from model selection and provider selection.
+Router V2 resolves an Auto alias to its tier's preferred model, then
+`src/inference/auto_model.rs` may choose another compiled candidate of that
+tier before context assembly, persistence, and route pinning when every route
+of the preferred model is unavailable under that model's own failover policy.
+Explicit selections never change public model, and the chosen model still goes
+through the same-model provider planner and first-send claim. Before any
+provider send, a chosen Auto model that cannot hold the request or loses its
+routes earns exactly one further decision with that model excluded; after the
+first send the logical response stays pinned. Router V2 also
+prefers the account's remembered route from `src/inference/sticky_routes.rs`
+(model and provider, per surface and selector, recorded once the provider
+accepts a request) while it stays eligible and the account has not been idle
+for the sticky window; Router V1 never consults either. Do not
+infer inactive behavior from historical `Shadow` type names: active V2
+selection consumes health snapshots.
 
 Preserve first-send claim handling and later-turn pinning. A claim lost before
 the first send may select another same-model provider; later Responses tool

services/opensecret/src/bounded_ttl_cache.rs

@@ -62,7 +62,7 @@ where
         self.insert_evicting_at(key, value, Instant::now())
     }
 
-    fn insert_evicting_at(&mut self, key: K, value: V, now: Instant) -> InsertOutcome {
+    pub(crate) fn insert_evicting_at(&mut self, key: K, value: V, now: Instant) -> InsertOutcome {
         let expires_at = now
             .checked_add(self.ttl)
             .expect("bounded cache TTL must fit in Instant");
@@ -96,6 +96,27 @@ where
         outcome
     }
 
+    /// Returns a live value without consuming it. Reads do not change the
+    /// recency order, so LRU order keeps matching expiry order as
+    /// [`Self::retire_expired_at`] requires. An expired entry is removed
+    /// instead of returned.
+    #[cfg(test)]
+    pub(crate) fn get_live(&mut self, key: &K) -> Option<&V> {
+        self.get_live_at(key, Instant::now())
+    }
+
+    pub(crate) fn get_live_at(&mut self, key: &K, now: Instant) -> Option<&V> {
+        if self
+            .entries
+            .peek(key)
+            .is_some_and(|entry| entry.expires_at <= now)
+        {
+            drop(self.entries.pop(key));
+            return None;
+        }
+        self.entries.peek(key).map(|entry| &entry.value)
+    }
+
     /// Removes a value exactly once and returns it only while its TTL is live.
     pub(crate) fn take_live(&mut self, key: &K) -> Option<V> {
         self.take_live_at(key, Instant::now())
@@ -148,7 +169,7 @@ where
     }
 
     #[cfg(test)]
-    fn len(&self) -> usize {
+    pub(crate) fn len(&self) -> usize {
         self.entries.len()
     }
 }
@@ -464,4 +485,46 @@ mod tests {
             Some(128 * 12 - 1)
         );
     }
+
+    #[test]
+    fn get_live_returns_without_consuming_and_drops_expired_entries() {
+        let start = Instant::now();
+        let mut cache = cache(2, Duration::from_secs(60));
+        cache.insert_evicting_at(1, "one", start);
+
+        assert_eq!(
+            cache.get_live_at(&1, start + Duration::from_secs(59)),
+            Some(&"one")
+        );
+        assert_eq!(
+            cache.get_live_at(&1, start + Duration::from_secs(59)),
+            Some(&"one")
+        );
+        assert_eq!(cache.get_live_at(&1, start + Duration::from_secs(60)), None);
+        assert_eq!(cache.len(), 0);
+        assert_eq!(cache.get_live(&1), None);
+    }
+
+    #[test]
+    fn get_live_does_not_reorder_entries_so_oldest_expiry_is_still_retired_first() {
+        let start = Instant::now();
+        let mut cache = cache(3, Duration::from_secs(60));
+        cache.insert_evicting_at(1, 1u8, start);
+        cache.insert_evicting_at(2, 2u8, start + Duration::from_secs(10));
+        cache.insert_evicting_at(3, 3u8, start + Duration::from_secs(20));
+
+        // Reading the oldest entry must not move it to the front.
+        assert_eq!(
+            cache.get_live_at(&1, start + Duration::from_secs(30)),
+            Some(&1u8)
+        );
+        let retired = cache.retire_expired_at(start + Duration::from_secs(65));
+        assert_eq!(retired.removed_count(), 1);
+        assert_eq!(cache.get_live_at(&1, start + Duration::from_secs(65)), None);
+        assert_eq!(
+            cache.get_live_at(&2, start + Duration::from_secs(65)),
+            Some(&2u8)
+        );
+        assert_eq!(cache.len(), 2);
+    }
 }

services/opensecret/src/inference.rs

@@ -1,10 +1,13 @@
 use crate::model_config::{ModelPlan, AUTO_POWERFUL_MODEL_ID, AUTO_QUICK_MODEL_ID};
 use crate::provider_registry::{ProviderId, RouteSelectionSource};
+use auto_model::{AutoModelDecision, AutoModelReason};
 use std::fmt;
 use std::time::Duration;
 use uuid::Uuid;
 
+pub(crate) mod auto_model;
 pub(crate) mod health;
+pub(crate) mod sticky_routes;
 
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub(crate) enum ModelSelectionMode {
@@ -25,9 +28,19 @@ impl ModelSelectionMode {
     pub(crate) const fn is_auto(self) -> bool {
         matches!(self, Self::AutoQuick | Self::AutoPowerful)
     }
+
+    /// The public alias string for an Auto mode, the inverse of
+    /// [`Self::from_requested_model`].
+    pub(crate) const fn alias(self) -> Option<&'static str> {
+        match self {
+            Self::AutoQuick => Some(AUTO_QUICK_MODEL_ID),
+            Self::AutoPowerful => Some(AUTO_POWERFUL_MODEL_ID),
+            Self::Explicit => None,
+        }
+    }
 }
 
-#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
 pub(crate) enum InferenceSurface {
     ChatCompletions,
     Responses,
@@ -67,8 +80,14 @@ inference_id!(InferenceAttemptId);
 pub(crate) struct InferenceIntent {
     pub(crate) request_id: InferenceRequestId,
     pub(crate) account_uuid: Uuid,
+    /// The caller's selector: an Auto alias or an explicit public model.
     pub(crate) requested_model_id: String,
+    /// The model this logical request executes. Provider planning, context
+    /// preparation, persistence, and usage all use this identity.
     pub(crate) public_model_id: String,
+    /// The Router v2 Auto decision that produced `public_model_id`, when one
+    /// ran. Explicit selections and Router v1 requests carry none.
+    pub(crate) auto_model: Option<AutoModelDecision>,
     pub(crate) selection_mode: ModelSelectionMode,
     pub(crate) model_plan: ModelPlan,
     pub(crate) surface: InferenceSurface,
@@ -91,12 +110,37 @@ impl InferenceIntent {
             selection_mode: ModelSelectionMode::from_requested_model(&requested_model_id),
             requested_model_id,
             public_model_id: public_model_id.into(),
+            auto_model: None,
             model_plan,
             surface,
             workload_class,
         }
     }
 
+    /// Applies a finished Router v2 Auto model decision. The chosen model
+    /// becomes the executed identity while the decision stays visible as
+    /// intent metadata.
+    pub(crate) fn with_auto_model_decision(mut self, decision: AutoModelDecision) -> Self {
+        debug_assert_eq!(decision.selector, self.requested_model_id);
+        self.public_model_id = decision.chosen_model_id.to_string();
+        self.auto_model = Some(decision);
+        self
+    }
+
+    /// The model the selector resolves to when healthy. Differs from
+    /// `public_model_id` only after an Auto decision chose an alternate.
+    pub(crate) fn preferred_model_id(&self) -> &str {
+        self.auto_model
+            .as_ref()
+            .map_or(self.public_model_id.as_str(), |decision| {
+                decision.preferred_model_id
+            })
+    }
+
+    pub(crate) fn auto_model_reason(&self) -> Option<AutoModelReason> {
+        self.auto_model.as_ref().map(|decision| decision.reason)
+    }
+
     pub(crate) fn begin_execution(&self) -> InferenceExecution {
         InferenceExecution {
             request_id: self.request_id,