From accdcefc3916982f311b3e0d5932a5f3e2b8031b Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 18:33:58 +0900 Subject: [PATCH 1/9] feat(combos): require explicit consent for metered targets (carry #6839) Port the metered target and subscription-chain regression work from #6839 at 5428b42440d68c73b8c12e14329e9e32cf9ff32e onto current dev. Require configured-key opt-in for all callers, retain metadata on dashboard saves, and preserve structured runTurn failures. Keep the newer fresh-read quota recovery; omit the timer cap and unrelated effort/tier changes. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- docs-site/src/content/docs/guides/combos.md | 6 + .../docs/reference/configuration/providers.md | 2 +- .../docs/reference/configuration/routing.md | 2 +- scripts/test-layout/layout.json | 2 +- src/cli/combo-input.ts | 5 +- src/combos/types.ts | 9 + src/config/schema/leaf-validators.ts | 2 + src/server/admission-model-scope.ts | 10 + src/server/management/combo-routes.ts | 13 +- src/server/responses/core-combo.ts | 9 +- src/server/responses/run-turn-execution.ts | 21 +- src/types/config.ts | 11 + structure/config.md | 2 +- structure/gui-and-management-api.md | 2 +- structure/providers-and-adapters.md | 2 +- structure/runtime.md | 2 +- structure/transports/responses-failover.md | 2 +- .../anthropic-subscription-chain.test.ts | 291 ++++++++++++++++++ tests/codex-integration/combos.test.ts | 2 +- tests/fixtures/test-layout-expected.json | 2 +- tests/server/combo-metered-consent.test.ts | 75 +++++ 21 files changed, 448 insertions(+), 24 deletions(-) create mode 100644 tests/adapters/anthropic/anthropic-subscription-chain.test.ts create mode 100644 tests/server/combo-metered-consent.test.ts diff --git a/docs-site/src/content/docs/guides/combos.md b/docs-site/src/content/docs/guides/combos.md index 60e3db08e6b..375b14dbe75 100644 --- a/docs-site/src/content/docs/guides/combos.md +++ b/docs-site/src/content/docs/guides/combos.md @@ -15,6 +15,12 @@ This is useful when you want either: Combos sit in front of normal provider routing. Read [Model Routing](/guides/model-routing/) first if `provider/model` selectors are new to you. +## Metered fallback consent + +Mark a pay-per-use combo target with `"metered": true` when it follows subscription targets. Every caller skips marked targets unless its configured `apiKeys[]` entry has `"allowMeteredComboTargets": true`. Omitted or false consent denies metered dispatch; environment-token and loopback callers must use an opted-in configured key. Model/provider scopes still apply. This applies to initial selection, failover and recovery across combo strategies. Unmarked targets retain existing behavior; this flag does not detect billing automatically. + +For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without key consent, exhaustion returns the last unmetered error. Dashboard saves retain an omitted metered flag; explicitly setting it to false clears it. Set the key consent in the config file; the dashboard does not expose this field. + ## 60-second quickstart This example creates `combo/main` with Anthropic first and OpenAI second. Both providers must diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 50c3520b0df..ff4dc8f4c1d 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -1224,7 +1224,7 @@ provider in question. ### Managed record shapes -`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. +`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. Optional `allowMeteredComboTargets: true` explicitly permits targets marked `metered: true`; omitted or false consent denies them. Loopback and environment-token callers must use an opted-in configured key. See [Metered fallback consent](/guides/combos/#metered-fallback-consent). `codexAccounts[]` entries require `id`, `email`, and `isMain`, with optional `plan`, `chatgptAccountId`, and privacy-safe `logLabel`. These records are normally dashboard-managed. diff --git a/docs-site/src/content/docs/reference/configuration/routing.md b/docs-site/src/content/docs/reference/configuration/routing.md index ff299a2b440..19d397ec460 100644 --- a/docs-site/src/content/docs/reference/configuration/routing.md +++ b/docs-site/src/content/docs/reference/configuration/routing.md @@ -138,7 +138,7 @@ namespace, and cannot use reserved bare native families such as `gpt-*`, `o1-*`, | Key | Type | Default | Meaning | | --- | --- | --- | --- | -| `targets` | `{ provider: string; model: string; weight?: number; lastResort?: boolean }[]` | required | Ordered concrete routes. `weight` is 1–10000 and defaults to `1`. `lastResort` marks an emergency-only target; see `cooldownWaitPolicy`. | +| `targets` | `{ provider: string; model: string; weight?: number; lastResort?: boolean; metered?: boolean }[]` | required | Ordered concrete routes. `weight` is 1–10000 and defaults to `1`. `lastResort` marks an emergency-only target; see `cooldownWaitPolicy`. `metered: true` requires configured-key `allowMeteredComboTargets: true`, including local callers; see [consent](/guides/combos/#metered-fallback-consent). | | `strategy?` | `"failover" \| "round-robin" \| "random" \| "least-used" \| "reset-window" \| "jev"` | `"failover"` | Selection strategy. Target order is failover priority; weights shape round-robin and random draws; least-used follows recorded successes; reset-window follows the soonest quota reset; JEV makes one bounded decision for the initial eligible target and effort, then uses ordinary ordered fallback. | | `stickyLimit?` | `number` | `1` | Successful requests retained in one round-robin batch. Range 1–100. Applies only to round-robin. | | `cooldownMs?` | `number` | unset → upstream fallback (5 s for request-rate 429 codes `1302`/`1305`, otherwise 60 s) | Range 1–600000. When set, applies whenever no usable upstream `Retry-After` or Codex reset signal exists, including request-rate 429s; when unset, uses the upstream fallback. Upstream signals take precedence. An explicit upstream `Retry-After` is capped at 24 hours; reset-derived, configured, and fallback cooldowns are capped at 10 minutes. | diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 16227fe5cf1..8e11bae7b0f 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -14,7 +14,7 @@ "anthropic-instance-isolation.test.ts": "adapters/anthropic", "anthropic-instance-pool-parity.test.ts": "adapters/anthropic", "anthropic-instance-quota.test.ts": "adapters/anthropic", "anthropic-instance-recovery.test.ts": "adapters/anthropic", "anthropic2-fast-parity.test.ts": "adapters/anthropic", "anthropic2-native-routing.test.ts": "claude-integration", - "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", + "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", "oauth-anthropic-instance-multiprocess.test.ts": "oauth", "anthropic2-discovery-ownership.test.ts": "oauth", "codex-shim-runtime-preflight.test.ts": "codex-integration", diff --git a/src/cli/combo-input.ts b/src/cli/combo-input.ts index e45f616a2a4..2ef107645ff 100644 --- a/src/cli/combo-input.ts +++ b/src/cli/combo-input.ts @@ -9,7 +9,7 @@ type ComboInput = { reasoningEffortMode?: "strict" | "adaptive"; nativeAlias?: boolean; }; -const TARGET_KEYS = new Set(["provider", "model", "weight", "reasoningEfforts", "modelProfile", "lastResort"]); +const TARGET_KEYS = new Set(["provider", "model", "weight", "reasoningEfforts", "modelProfile", "lastResort", "metered"]); export function parseComboTargets(value: unknown): Array> { if (!Array.isArray(value) || value.length === 0) throw new CliUsageError("Targets input must be a nonempty array"); @@ -41,6 +41,9 @@ export function parseComboTargets(value: unknown): Array if (Object.hasOwn(target, "lastResort") && typeof target.lastResort !== "boolean") { throw new CliUsageError("Target lastResort must be a boolean"); } + if (Object.hasOwn(target, "metered") && typeof target.metered !== "boolean") { + throw new CliUsageError("Target metered must be a boolean"); + } // Preserve raw slashes, target order, explicit false and omitted metadata. return { ...target }; }); diff --git a/src/combos/types.ts b/src/combos/types.ts index 6353db3b7a5..d11a27baf9a 100644 --- a/src/combos/types.ts +++ b/src/combos/types.ts @@ -39,6 +39,8 @@ export interface NormalizedComboTarget { weight: number; /** Emergency-only target, deferred under `cooldownWaitPolicy` (#5691). */ lastResort: boolean; + /** Pay-per-use target; issued keys reach it only with `allowMeteredComboTargets`. */ + metered: boolean; reasoningEfforts?: OcxComboDefaultEffort[]; /** Optional JEV decision description. */ modelProfile?: string; @@ -427,6 +429,12 @@ export function comboConfigIssues( message: `targets[${i}].lastResort must be a boolean`, }); } + if (target.metered !== undefined && typeof target.metered !== "boolean") { + issues.push({ + path: ["targets", i, "metered"], + message: `targets[${i}].metered must be a boolean`, + }); + } if (target.modelProfile !== undefined && (typeof target.modelProfile !== "string" || target.modelProfile.trim().length === 0 @@ -503,6 +511,7 @@ export function normalizeComboConfig(raw: OcxComboConfig): NormalizedComboConfig ? { modelProfile: target.modelProfile.trim() } : {}), lastResort: target.lastResort === true, + metered: target.metered === true, })), }; } diff --git a/src/config/schema/leaf-validators.ts b/src/config/schema/leaf-validators.ts index 75b39a29ba3..4e0a7d20ce2 100644 --- a/src/config/schema/leaf-validators.ts +++ b/src/config/schema/leaf-validators.ts @@ -794,6 +794,8 @@ export const apiKeyEntrySchema = z.object({ // rather than promoting it. allowedProviders: z.array(z.string().trim().min(1).max(256)).optional(), allowedModels: z.array(z.string().trim().min(1).max(256)).optional(), + // Not `.catch`ed for the same reason: a damaged value must not grant metered spend. + allowMeteredComboTargets: z.boolean().optional(), }).passthrough(); /** diff --git a/src/server/admission-model-scope.ts b/src/server/admission-model-scope.ts index 54798ff98ce..08c98681ad0 100644 --- a/src/server/admission-model-scope.ts +++ b/src/server/admission-model-scope.ts @@ -77,6 +77,16 @@ export function resolveAdmissionModelScope( return entry ? admissionModelScopeOf(entry) : undefined; } +/** Metered combo dispatch requires an explicitly opted-in configured key. */ +export function admissionMayUseMeteredComboTargets( + config: Pick, + admission: DataPlaneAdmission | undefined, +): boolean { + if (!admission || admission.kind !== "configured") return false; + const entry = (config.apiKeys ?? []).find(key => key.id === admission.keyId); + return entry?.allowMeteredComboTargets === true; +} + /** * Does this scope admit this resolved destination? * diff --git a/src/server/management/combo-routes.ts b/src/server/management/combo-routes.ts index a1f64030cce..30ca7ab341c 100644 --- a/src/server/management/combo-routes.ts +++ b/src/server/management/combo-routes.ts @@ -203,7 +203,7 @@ export async function handleComboRoutes(ctx: ManagementContext): Promise Object.hasOwn(target, key))) return target; const rawProvider = target.provider; const rawModel = target.model; if (typeof rawProvider !== "string" || typeof rawModel !== "string") return target; @@ -214,7 +214,13 @@ export async function handleComboRoutes(ctx: ManagementContext): Promise candidate.provider === provider && candidate.model === model, ); - return before?.lastResort ? { ...target, lastResort: true } : target; + // `metered` is carried the same way: dropping it on a GUI round-trip would open a + // pay-per-use target to every issued key. + return { + ...target, + ...(!Object.hasOwn(target, "lastResort") && before?.lastResort ? { lastResort: true } : {}), + ...(!Object.hasOwn(target, "metered") && before?.metered ? { metered: true } : {}), + }; }), } : {}), @@ -295,9 +301,10 @@ export async function handleComboRoutes(ctx: ManagementContext): Promise ({ + targets: normalizedBase.targets.map(({ lastResort, metered, modelProfile, ...target }) => ({ ...target, ...(lastResort ? { lastResort: true } : {}), + ...(metered ? { metered: true } : {}), ...(modelProfile ? { modelProfile } : {}), })), ...(normalizedAlias ? { alias: normalizedAlias } : {}), diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts index 63159dc4bdf..657bc8c5665 100644 --- a/src/server/responses/core-combo.ts +++ b/src/server/responses/core-combo.ts @@ -6,7 +6,7 @@ import { isDeclaredReasoningEffort, resolveEffortAtOrBelow, } from "../../reasoning-effort"; -import { resolveAdmissionModelScope, routeAllowedByScope } from "../admission-model-scope"; +import { admissionMayUseMeteredComboTargets, resolveAdmissionModelScope, routeAllowedByScope } from "../admission-model-scope"; import { recordAttemptRequestedEffort } from "../request-log"; import { CODEX_TEXT_GUARDED_BUDGET_POLICY, @@ -449,8 +449,10 @@ export async function executeComboResponses( let comboPayloadReadable = false; const payloadEligible = (target: (typeof combo.targets)[number]): boolean => comboPayloadReadable || !unreadableEncryptedAgentTask || canDecryptUnreadableAgentTask(target); + const meteredAllowed = admissionMayUseMeteredComboTargets(config, options.admission); const targetEligible = (target: (typeof combo.targets)[number]): boolean => - (combo.strategy !== "jev" || target.provider !== JEV_PROVIDER_ID) + (meteredAllowed || !target.metered) + && (combo.strategy !== "jev" || target.provider !== JEV_PROVIDER_ID) && payloadEligible(target) && reasoningReplayEligible(target) && (protocolLanes?.pickable(target) ?? true); @@ -524,6 +526,9 @@ export async function executeComboResponses( now?: number; }) => pickComboTargetWithWait(config, comboId, { ...pickOptions, + // Every pick, including the recovery picks that skip `targetEligible`, keeps metered + // targets away from callers that may not spend on them. + eligible: target => (meteredAllowed || !target.metered) && (pickOptions.eligible?.(target) ?? true), waitForCooldownMs: combo.waitForCooldownMs, abortSignal: options.abortSignal, }); diff --git a/src/server/responses/run-turn-execution.ts b/src/server/responses/run-turn-execution.ts index 689730cfae0..1ea1f55210e 100644 --- a/src/server/responses/run-turn-execution.ts +++ b/src/server/responses/run-turn-execution.ts @@ -776,9 +776,13 @@ export async function executeResponsesRunTurn( if (preflight.error || preflight.empty) { runTurnAbort.abort(); queue.close(); - const message = preflight.error?.message ?? "Adapter ended before producing a response"; - const failure = formatErrorResponse(preflight.error?.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE ? 503 : 502, - preflight.error?.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : "upstream_error", redactSecretString(message)); + const classified = adapterFailureFromEvent(preflight.error ?? { + type: "error", status: 502, message: "Adapter ended before producing a response", + }); + const storageRefusal = preflight.error?.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE; + const failure = formatErrorResponse(storageRefusal ? 503 : classified.httpStatus, + storageRefusal ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : classified.error.type, + redactSecretString(classified.error.message), { code: classified.error.code }); // A replay-unsafe heartbeat means the adapter already ran a local side effect, so the // combo must not send this turn to another target: the failure stays with this child. if (preflight.replayUnsafe) markResponseNonReplayable(failure); @@ -974,12 +978,13 @@ export async function executeResponsesRunTurn( ? classifyUndeclaredFirstTool(firstMeaningful) : undefined; if (!firstMeaningful || firstMeaningful.type === "error" || classifiedError) { - const message = classifiedError?.message ?? (firstMeaningful?.type === "error" - ? firstMeaningful.message - : "Adapter ended before producing a response"); + const classified = adapterFailureFromEvent(classifiedError ?? (firstMeaningful?.type === "error" + ? firstMeaningful + : { type: "error", status: 502, message: "Adapter ended before producing a response" })); const storageRefusal = firstMeaningful?.type === "error" && firstMeaningful.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE; - const failure = formatErrorResponse(storageRefusal ? 503 : 502, - storageRefusal ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : "upstream_error", redactSecretString(message)); + const failure = formatErrorResponse(storageRefusal ? 503 : classified.httpStatus, + storageRefusal ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : classified.error.type, + redactSecretString(classified.error.message), { code: classified.error.code }); if (replayUnsafe) markResponseNonReplayable(failure); return failure; } diff --git a/src/types/config.ts b/src/types/config.ts index 9b56063c760..9f04ed1baa6 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -349,6 +349,12 @@ export interface OcxApiKeyEntry { * that is the only point at which the model about to be billed is known. */ allowedModels?: string[]; + /** + * Lets this key reach combo targets marked `metered`. Absent means no: an issued key stops + * at the last unmetered target. Environment-token and loopback callers must use an opted-in + * configured key to reach metered targets. + */ + allowMeteredComboTargets?: boolean; } export interface OcxPendingApiKeyRotation { @@ -1338,6 +1344,11 @@ export interface OcxComboTarget { * see `OcxComboConfig.cooldownWaitPolicy` (#5691). */ lastResort?: boolean; + /** + * Pay-per-use target (for example an API-key row behind subscription targets). Only the + * configured keys with `allowMeteredComboTargets: true` may reach it. + */ + metered?: boolean; } export interface OcxComboConfig { diff --git a/structure/config.md b/structure/config.md index f803d408c5c..cd8d1821996 100644 --- a/structure/config.md +++ b/structure/config.md @@ -1,6 +1,6 @@ # Config Surface -Quota activation reuses the existing next-reset fields without adding a polling configuration key. See the [quota activation contract](providers/openai-tiers.md#public-provider-contract). `src/types/config.ts` defines `anthropicAccountPool.routes` as ordered model rules. `src/oauth/anthropic-model-routes.ts` validates bounded names, full case-sensitive globs and stored account IDs; `src/config/diagnostics.ts` rejects malformed candidate writes. Load retains a malformed hand edit so unrelated providers survive, while the enabled Anthropic selector refuses it locally before dispatch. Saved rules remain inert when the pool is disabled; affinity is process-local. Request logs use only the rule’s 1-based `route:#` position, never its configured name. `src/cli/hub-disable.ts` removes interdependent hub/loopback/role/hostname settings through `mutatePersistedConfig` and `validateConfigCandidate` in one commit. Top-level deletion intent uses `src/config/rebase-provenance.ts` and authoritative disk provenance in `src/config/live-reconcile.ts` to preserve removed hub bindings through later live saves, `src/config/live-topology.ts` keeps an independent active-topology baseline and merges desired next-start settings into a separate persistence projection, retaining actual socket identity and live hub/client topology during guarded saves and OAuth disk adoption. Client connections and runtimeRole remain one topology unit through connect/disconnect, including fresh disk-only connections. Disk additions/value updates and deletions all survive repeated saves without changing active topology. `src/config/api-key-reconcile.ts` merges key rows by stable ID under the existing save lock; either side's revocation defeats stale edits while disjoint names and rotations merge; unchanged live key membership/order adopts persisted ordering. `tests/config/hub-disable-live-save.test.ts` exercises disable followed by live key creation, rename, rotation, unrelated settings and repeated saves. Saved API keys have no immutable enrollment marker; explicit exact-ID revocation preserves all unselected keys. +`src/config/schema/leaf-validators.ts` accepts only boolean `apiKeys[].allowMeteredComboTargets`; malformed entries are dropped rather than promoted. `src/types/config.ts` and combo validation define optional boolean target `metered`, normalized false when omitted. Explicit consent is required under the [combo runtime contract](runtime.md#request-local-target-compatibility). Quota activation reuses the existing next-reset fields without adding a polling configuration key. See the [quota activation contract](providers/openai-tiers.md#public-provider-contract). `src/types/config.ts` defines `anthropicAccountPool.routes` as ordered model rules. `src/oauth/anthropic-model-routes.ts` validates bounded names, full case-sensitive globs and stored account IDs; `src/config/diagnostics.ts` rejects malformed candidate writes. Load retains a malformed hand edit so unrelated providers survive, while the enabled Anthropic selector refuses it locally before dispatch. Saved rules remain inert when the pool is disabled; affinity is process-local. Request logs use only the rule’s 1-based `route:#` position, never its configured name. `src/cli/hub-disable.ts` removes interdependent hub/loopback/role/hostname settings through `mutatePersistedConfig` and `validateConfigCandidate` in one commit. Top-level deletion intent uses `src/config/rebase-provenance.ts` and authoritative disk provenance in `src/config/live-reconcile.ts` to preserve removed hub bindings through later live saves, `src/config/live-topology.ts` keeps an independent active-topology baseline and merges desired next-start settings into a separate persistence projection, retaining actual socket identity and live hub/client topology during guarded saves and OAuth disk adoption. Client connections and runtimeRole remain one topology unit through connect/disconnect, including fresh disk-only connections. Disk additions/value updates and deletions all survive repeated saves without changing active topology. `src/config/api-key-reconcile.ts` merges key rows by stable ID under the existing save lock; either side's revocation defeats stale edits while disjoint names and rotations merge; unchanged live key membership/order adopts persisted ordering. `tests/config/hub-disable-live-save.test.ts` exercises disable followed by live key creation, rename, rotation, unrelated settings and repeated saves. Saved API keys have no immutable enrollment marker; explicit exact-ID revocation preserves all unselected keys. Native function-result injection follows [the separate opt-in control contract](transports/streaming-health.md#experimental-native-function-result-injection); this surface does not infer upstream support or alter its defaults. diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index 961a55f07ed..f2d1d691aae 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -1,6 +1,6 @@ # GUI And Management API -Anthropic OAuth account DTOs include `autoSwitchThresholdOverride` (integer or null), +`src/server/management/combo-routes.ts` retains omitted `metered: true` metadata by trimmed provider/model identity during dashboard-shaped saves. Explicit false clears the flag; false defaults are omitted from persisted targets. Key consent follows the [runtime contract](runtime.md#request-local-target-compatibility). Anthropic OAuth account DTOs include `autoSwitchThresholdOverride` (integer or null), `autoSwitchThreshold` (pool default) and `effectiveAutoSwitchThreshold`. The dedicated `PUT /api/oauth/accounts/auto-switch` accepts `{ provider: "anthropic", accountId, threshold }`; explicit null restores inheritance and missing/invalid values fail. Other providers are rejected. diff --git a/structure/providers-and-adapters.md b/structure/providers-and-adapters.md index c522806f375..24c4f501452 100644 --- a/structure/providers-and-adapters.md +++ b/structure/providers-and-adapters.md @@ -1,6 +1,6 @@ # Providers And Adapters -Configured spend ceilings bind adapter inference executors to the [seed and physical-start contract](transports/responses-spend.md#historical-pool-continuity-and-rollback). Zed completion and auth-recovery replays claim separately; token acquisition stays outside inference accounting. Claude CLI, CodeBuddy and Qoder each count one invocation toward the frozen request limit, seed before spawn and retain a producer through completion. Child-internal retries/tool turns are not separately counted; reported actual usage settles in full, including amounts above the seed estimate. +Combo target `metered` metadata and configured-key opt-in follow the [runtime consent contract](runtime.md#request-local-target-compatibility). Subscription-to-Cursor-to-key regression coverage in `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` checks unchanged healthy subscription wire bodies, effort, key authentication and cache controls, and structured runTurn 429 propagation. Configured spend ceilings bind adapter inference executors to the [seed and physical-start contract](transports/responses-spend.md#historical-pool-continuity-and-rollback). Zed completion and auth-recovery replays claim separately; token acquisition stays outside inference accounting. Claude CLI, CodeBuddy and Qoder each count one invocation toward the frozen request limit, seed before spawn and retain a producer through completion. Child-internal retries/tool turns are not separately counted; reported actual usage settles in full, including amounts above the seed estimate. Devin combines only [consecutive same-ID tool results](adapters/registry.md#devin-consecutive-tool-results), using linear accumulation while preserving intervening message slots, image parts and error markers without mutating the parsed request. Anthropic account pause, model routes, quota labels, and the separate `anthropic2` pool (its helpers, quota, reset grants and management surfaces) follow the [Anthropic account-pool contract](providers/anthropic-account-pool.md). Devin Messages follows the [per-turn output ordering contract](clients/claude-desktop.md#devin-messages-output-ordering), preserving late signatures before text/tools without changing Responses or Chat ordering. diff --git a/structure/runtime.md b/structure/runtime.md index 2a31215cb52..3b7c2126fb1 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -488,7 +488,7 @@ Renamed fixed-key providers receive [missing reasoning metadata](catalog.md#rena Translated audio/file admission follows the [final-adapter input contract](adapters/registry.md#untranslated-input-media); native raw passthrough remains separate. ## Request-local target compatibility -The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. +`src/server/admission-model-scope.ts` grants metered combo eligibility only to a matching configured key with literal `allowMeteredComboTargets: true`. `src/server/responses/core-combo.ts` filters marked targets on initial, failure and recovery selection for every strategy. Missing identity, environment-token and loopback admission deny marked targets; provider/model scopes remain independent. `src/cli/combo-input.ts` accepts and validates the optional target boolean. Unmarked targets retain existing behavior; metering is operator metadata, not automatic billing detection. Coverage: `tests/server/combo-metered-consent.test.ts` and `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. `src/adapters/openai-responses.ts` omits only top-level `user` at the canonical ChatGPT Codex forward destination. Claude translation retains its original identity and prompt-cache key; public API and noncanonical gateways retain their `user` field. Input roles, tool-schema properties, safety identifiers and original replay bodies are not changed. diff --git a/structure/transports/responses-failover.md b/structure/transports/responses-failover.md index 535ffb4c842..a28b890f286 100644 --- a/structure/transports/responses-failover.md +++ b/structure/transports/responses-failover.md @@ -1,6 +1,6 @@ # Responses Failover And Replay - `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. +Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`; spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. Classified 429 admission follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#classified-429-admission). Classified account-entitlement 403 recovery follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#account-entitlement-refusal-recovery). `src/server/responses/compaction-recovery-policy.ts` is a pure eligibility policy, not a dispatcher. It requires explicit configuration and normalized attempt evidence, preserves ordinary requests, diff --git a/tests/adapters/anthropic/anthropic-subscription-chain.test.ts b/tests/adapters/anthropic/anthropic-subscription-chain.test.ts new file mode 100644 index 00000000000..0d02f8c9fc1 --- /dev/null +++ b/tests/adapters/anthropic/anthropic-subscription-chain.test.ts @@ -0,0 +1,291 @@ +/** + * One model id, three billing paths: an Anthropic OAuth subscription pool, Cursor's copy of the + * same model, then a pay-per-use Anthropic API key. A failover combo whose alias is the model id + * the clients already send gives that order; `metered` keeps issued keys off the API-key target, + * and every caller needs explicit configured-key consent to reach a metered target. + */ +import { afterEach, beforeEach, expect, mock, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { ProviderAdapter } from "../../../src/adapters/base"; +import type { AdapterEvent, OcxConfig, OcxProviderConfig } from "../../../src/types"; +import type { DataPlaneAdmission } from "../../../src/server/auth-cors"; +import { acquireOwnedSpendHome } from "../../helpers/owned-spend-home"; +import { removeTreeWithRetry } from "../../helpers/remove-tree"; + +const MODEL = "claude-opus-5-5"; +const ISSUED_KEY_ID = "issued-key"; +let cursorLimited = false; +let cursorModels: string[] = []; +let cursorEfforts: unknown[] = []; + +const actualResolver = await import("../../../src/server/adapter-resolve"); +const actualResolveAdapter = actualResolver.resolveAdapter; +mock.module("../../../src/server/adapter-resolve", () => ({ + ...actualResolver, + resolveAdapter(provider: OcxProviderConfig, cacheRetention?: "none" | "short" | "long") { + if (provider.adapter !== "cursor") return actualResolveAdapter(provider, cacheRetention); + return { + name: "cursor", + buildRequest: () => ({ url: provider.baseUrl, method: "POST", headers: {}, body: "" }), + async *parseStream() { yield { type: "error", message: "fixture uses runTurn" } as AdapterEvent; }, + async runTurn(parsed: { modelId: string; options?: { reasoning?: unknown } }, _incoming: unknown, emit: (event: AdapterEvent) => void) { + cursorModels.push(parsed.modelId); + cursorEfforts.push(parsed.options?.reasoning); + if (cursorLimited) { + emit({ type: "error", status: 429, message: "You've hit your usage limit", code: "rate_limit_exceeded" }); + return; + } + emit({ type: "text_delta", text: "cursor answered" }); + emit({ type: "done" }); + }, + } as unknown as ProviderAdapter; + }, +})); + +const { handleChatCompletions } = await import("../../../src/server/chat-completions"); +const { saveCredential, getAccountSet, setActiveAccount } = await import("../../../src/oauth/store"); +const { + clearAnthropicAccountPoolState, forgetAnthropicFailoverQuorum, +} = await import("../../../src/oauth/anthropic-routing"); +const { clearComboSelectionState } = await import("../../../src/combos/resolve"); +const { clearComboTargetCooldowns } = await import("../../../src/combos/failover"); +const { clearGenericFailoverHealth } = await import("../../../src/oauth/generic-account-failover"); +const { clearAccountQuotaCache } = await import("../../../src/providers/quota"); + +type Lane = "subscription" | "api-key"; +type Sent = { lane: Lane; authorization: string | null; apiKey: string | null; body: Record }; +let sent: Sent[] = []; +let subscriptionSpent = false; +let apiKeyLimited = false; +let home = ""; +let releaseSpendHome: (() => void) | undefined; +const originalHome = process.env.OPENCODEX_HOME; + +function sse(): Response { + const event = (name: string, data: unknown) => `event: ${name}\ndata: ${JSON.stringify(data)}\n\n`; + return new Response([ + event("message_start", { type: "message_start", message: { id: "msg_chain", type: "message", role: "assistant", model: MODEL, content: [], stop_reason: null, usage: { input_tokens: 8, output_tokens: 0 } } }), + event("content_block_start", { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }), + event("content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "anthropic answered" } }), + event("content_block_stop", { type: "content_block_stop", index: 0 }), + event("message_delta", { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 3 } }), + event("message_stop", { type: "message_stop" }), + ].join(""), { headers: { "content-type": "text/event-stream" } }); +} + +/** The 2026-09-28 shape: a spent weekly window with an 88.5-hour Retry-After. */ +function weeklySpent(): Response { + return Response.json( + { type: "error", error: { type: "rate_limit_error", message: "usage limit reached" } }, + { status: 429, headers: { "anthropic-ratelimit-unified-7d-status": "rejected", "retry-after": "318747" } }, + ); +} + +function transport(lane: Lane): typeof fetch { + return (async (_input: RequestInfo | URL, init?: RequestInit) => { + const headers = new Headers(init?.headers); + sent.push({ + lane, + authorization: headers.get("authorization"), + apiKey: headers.get("x-api-key"), + body: JSON.parse(String(init?.body ?? "{}")) as Record, + }); + const limited = lane === "subscription" ? subscriptionSpent : apiKeyLimited; + return limited ? weeklySpent() : sse(); + }) as typeof fetch; +} + +function config(options: { combo?: boolean; allowMetered?: boolean } = {}): OcxConfig { + const subscription = { adapter: "anthropic", baseUrl: "https://subscription.test", authMode: "oauth", models: [MODEL], fetch: transport("subscription") }; + const apiKey = { adapter: "anthropic", baseUrl: "https://api-key.test", authMode: "key", apiKey: "${CHAIN_TEST_ANTHROPIC_KEY}", models: [MODEL], fetch: transport("api-key") }; + return { + port: 0, + defaultProvider: "anthropic", + providers: { + anthropic: subscription as OcxProviderConfig, + cursor: { adapter: "cursor", baseUrl: "https://api2.cursor.sh", authMode: "oauth", models: [MODEL] }, + "anthropic-metered": apiKey as OcxProviderConfig, + }, + apiKeys: [{ + id: ISSUED_KEY_ID, name: "issued", key: "ocx_data_issued_key_for_chain_test", createdAt: "2026-10-09T00:00:00.000Z", + allowedModels: [MODEL], + ...(options.allowMetered ? { allowMeteredComboTargets: true } : {}), + }], + ...(options.combo === false ? {} : { + combos: { + "claude-chain": { + alias: `anthropic/${MODEL}`, + strategy: "failover", + targets: [ + { provider: "anthropic", model: MODEL }, + { provider: "cursor", model: MODEL }, + { provider: "anthropic-metered", model: MODEL, metered: true }, + ], + }, + }, + }), + } as OcxConfig; +} + +const LOOPBACK: DataPlaneAdmission = { kind: "loopback", source: "loopback" }; +const ISSUED: DataPlaneAdmission = { kind: "configured", keyId: ISSUED_KEY_ID, source: "bearer" }; + +async function chat(cfg: OcxConfig, admission: DataPlaneAdmission = LOOPBACK, stream = false) { + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, + fetch: req => handleChatCompletions(req, cfg, { model: "", provider: "" }, + { requestId: "chain-test", start: Date.now(), admission } as never), + }); + try { + const response = await fetch(new URL("/v1/chat/completions", server.url), { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + model: `anthropic/${MODEL}`, + stream, + reasoning_effort: "medium", + messages: [{ role: "system", content: "S".repeat(5000) }, { role: "user", content: "hi" }], + }), + }); + return { status: response.status, text: await response.text() }; + } finally { + await server.stop(true); + } +} + +const lanes = () => sent.map(entry => entry.lane); + +beforeEach(async () => { + home = mkdtempSync(join(tmpdir(), "ocx-subscription-chain-")); + process.env.OPENCODEX_HOME = home; + process.env.CHAIN_TEST_ANTHROPIC_KEY = "sk-ant-chain-test"; + releaseSpendHome = acquireOwnedSpendHome(); + for (let index = 0; index < 2; index++) { + await saveCredential("anthropic", { + access: `subscription-access-${index}`, refresh: `subscription-refresh-${index}`, + expires: Date.now() + 3_600_000, accountId: `subscription-account-${index}`, + }); + } + await setActiveAccount("anthropic", getAccountSet("anthropic")!.accounts[0]!.id); + await saveCredential("cursor", { access: "cursor-access", refresh: "cursor-refresh", expires: Date.now() + 3_600_000, accountId: "cursor-account" }); + sent = []; cursorModels = []; cursorEfforts = []; + subscriptionSpent = false; cursorLimited = false; apiKeyLimited = false; + clearComboSelectionState(); clearComboTargetCooldowns(); clearAnthropicAccountPoolState(); + forgetAnthropicFailoverQuorum(); clearGenericFailoverHealth(); clearAccountQuotaCache(); +}); + +afterEach(() => { + releaseSpendHome?.(); + releaseSpendHome = undefined; + clearComboSelectionState(); clearComboTargetCooldowns(); clearAnthropicAccountPoolState(); + forgetAnthropicFailoverQuorum(); clearGenericFailoverHealth(); clearAccountQuotaCache(); + if (originalHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = originalHome; + delete process.env.CHAIN_TEST_ANTHROPIC_KEY; + removeTreeWithRetry(home); +}); + +test("a healthy subscription serves the aliased id with the same wire body as the plain route", async () => { + expect((await chat(config({ combo: false }))).status).toBe(200); + const plainBody = sent[0]!.body; + sent = []; + clearAnthropicAccountPoolState(); + + const response = await chat(config()); + expect(response.status).toBe(200); + expect(response.text).toContain("anthropic answered"); + expect(lanes()).toEqual(["subscription"]); + expect(sent[0]!.authorization).toStartWith("Bearer subscription-access-"); + expect(JSON.stringify({ ...sent[0]!.body, metadata: undefined })).toBe(JSON.stringify({ ...plainBody, metadata: undefined })); + expect(JSON.stringify(sent[0]!.body)).toContain("cache_control"); +}); + +test("every subscription account spent moves to Cursor's claude-opus-5-5 at the same effort", async () => { + subscriptionSpent = true; + const response = await chat(config()); + expect(response.status).toBe(200); + expect(response.text).toContain("cursor answered"); + expect(cursorModels).toEqual([MODEL]); + expect(cursorEfforts).toEqual(["medium"]); + expect(lanes().every(lane => lane === "subscription")).toBe(true); +}); + +test("subscription and Cursor limited moves an opted-in caller to the API key with x-api-key and cache_control", async () => { + subscriptionSpent = true; + cursorLimited = true; + const response = await chat(config({ allowMetered: true }), ISSUED); + expect(response.status).toBe(200); + const metered = sent.filter(entry => entry.lane === "api-key"); + expect(metered).toHaveLength(1); + expect(metered[0]!.apiKey).toBe("sk-ant-chain-test"); + expect(metered[0]!.authorization).toBeNull(); + expect(JSON.stringify(metered[0]!.body)).toContain("cache_control"); +}); + +test("all three limited returns the limit to the caller so its own fallback takes over", async () => { + subscriptionSpent = true; + cursorLimited = true; + apiKeyLimited = true; + const response = await chat(config({ allowMetered: true }), ISSUED); + expect(response.status).toBe(429); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(1); +}); + +test("an issued key stops before the metered target and gets the error instead", async () => { + subscriptionSpent = true; + cursorLimited = true; + const response = await chat(config(), ISSUED); + expect(response.status).toBeGreaterThanOrEqual(400); + expect(response.text).not.toContain("answered"); + expect(cursorModels).toEqual([MODEL]); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(0); +}); + +test("an issued key that opts in may reach the metered target", async () => { + subscriptionSpent = true; + cursorLimited = true; + const response = await chat(config({ allowMetered: true }), ISSUED); + expect(response.status).toBe(200); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(1); +}); + +test("buffered combo preflight preserves Cursor's structured rate-limit status", async () => { + cursorLimited = true; + const cfg = config(); + const combo = cfg.combos?.["claude-chain"]; + if (!combo) throw new Error("Missing fixture combo"); + combo.targets = [{ provider: "cursor", model: MODEL }]; + const response = await chat(cfg); + expect(response.status).toBe(429); + expect(JSON.parse(response.text).error.code).toBe("rate_limit_exceeded"); +}); + +for (const admission of [LOOPBACK, { kind: "environment", source: "bearer" } as const]) { + test(`${admission.kind} admission cannot spend on a metered fallback without configured-key consent`, async () => { + subscriptionSpent = true; + cursorLimited = true; + const response = await chat(config({ allowMetered: true }), admission); + expect(response.status).toBe(429); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(0); + }); +} + +test("metered consent does not bypass destination model scope", async () => { + subscriptionSpent = true; + cursorLimited = true; + const cfg = config({ allowMetered: true }); + cfg.apiKeys![0]!.allowedProviders = ["anthropic", "cursor"]; + expect((await chat(cfg, ISSUED)).status).toBeGreaterThanOrEqual(400); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(0); +}); + + +test("streaming combo preflight preserves Cursor's structured rate-limit status", async () => { + cursorLimited = true; + const cfg = config(); + cfg.combos!["claude-chain"]!.targets = [{ provider: "cursor", model: MODEL }]; + const response = await chat(cfg, LOOPBACK, true); + expect(response.status).toBe(429); + expect(JSON.parse(response.text).error.code).toBe("rate_limit_exceeded"); +}); diff --git a/tests/codex-integration/combos.test.ts b/tests/codex-integration/combos.test.ts index dd76fa1ce57..d6c370bf141 100644 --- a/tests/codex-integration/combos.test.ts +++ b/tests/codex-integration/combos.test.ts @@ -1639,7 +1639,7 @@ describe("combo validation and normalization", () => { alias: null, nativeAlias: false, displayName: null, - targets: [{ provider: "a", model: "m1", weight: 2, lastResort: false }], + targets: [{ provider: "a", model: "m1", weight: 2, lastResort: false, metered: false }], }); expect(normalizeComboConfig({ targets: [{ provider: "a", model: "m1" }] }).defaultEffort).toBeNull(); const targetReasoningEfforts: OcxComboDefaultEffort[] = ["low", "high"]; diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 210221f4d50..0082a1fec4b 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -26,7 +26,7 @@ "anthropic-instance-isolation.test.ts": "adapters/anthropic", "anthropic-instance-pool-parity.test.ts": "adapters/anthropic", "anthropic-instance-quota.test.ts": "adapters/anthropic", "anthropic-instance-recovery.test.ts": "adapters/anthropic", "anthropic2-fast-parity.test.ts": "adapters/anthropic", "anthropic2-native-routing.test.ts": "claude-integration", - "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", + "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", "oauth-anthropic-instance-multiprocess.test.ts": "oauth", "anthropic2-discovery-ownership.test.ts": "oauth", "codex-shim-runtime-preflight.test.ts": "codex-integration", diff --git a/tests/server/combo-metered-consent.test.ts b/tests/server/combo-metered-consent.test.ts new file mode 100644 index 00000000000..82100ce9637 --- /dev/null +++ b/tests/server/combo-metered-consent.test.ts @@ -0,0 +1,75 @@ +import { expect, test } from "bun:test"; +import { loadConfig, saveConfig } from "../../src/config"; +import { mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import type { OcxConfig } from "../../src/types"; +import { admissionMayUseMeteredComboTargets } from "../../src/server/admission-model-scope"; +import { parseComboTargets } from "../../src/cli/combo-input"; +import { comboConfigError, normalizeComboConfig } from "../../src/combos"; +import { handleManagementAPI } from "../../src/server/management-api"; +import { ManagementRequest } from "../helpers/management-auth"; +import { catalogConvergenceFactory } from "../helpers/catalog-convergence"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +const key = { id: "issued", name: "issued", key: "ocx_data_" + "a".repeat(40), createdAt: "2026-10-15T00:00:00Z" }; +const admission = { kind: "configured", keyId: key.id, source: "bearer" } as const; + +test("metered consent is literal true on a matching configured key", () => { + for (const consent of [undefined, false, "true", 1, null, true]) { + const config = { apiKeys: [{ ...key, allowMeteredComboTargets: consent }] } as OcxConfig; + expect(admissionMayUseMeteredComboTargets(config, admission)).toBe(consent === true); + expect(admissionMayUseMeteredComboTargets(config, { ...admission, keyId: "missing" })).toBe(false); + expect(admissionMayUseMeteredComboTargets(config, undefined)).toBe(false); + expect(admissionMayUseMeteredComboTargets(config, { kind: "loopback", source: "loopback" })).toBe(false); + expect(admissionMayUseMeteredComboTargets(config, { kind: "environment", source: "bearer" })).toBe(false); + } +}); + +test("CLI and combo normalization validate and preserve metered metadata", () => { + const target = { provider: "a", model: "m", metered: true }; + expect(parseComboTargets([target])).toEqual([target]); + expect(() => parseComboTargets([{ ...target, metered: "true" }])).toThrow(); + expect(comboConfigError("chain", { targets: [{ ...target, metered: "true" }] }, { a: { adapter: "openai-chat", baseUrl: "https://a.test/v1" } })).toContain("metered"); + expect(normalizeComboConfig({ targets: [target] }).targets[0]?.metered).toBe(true); + expect(normalizeComboConfig({ targets: [{ provider: "a", model: "m" }] }).targets[0]?.metered).toBe(false); +}); + +test("metered metadata survives dashboard saves, explicit false clears it, malformed key consent is dropped", async () => { + const previous = process.env.OPENCODEX_HOME; + const home = mkdtempSync(join(tmpdir(), "ocx-metered-consent-")); + process.env.OPENCODEX_HOME = home; + try { + const config: OcxConfig = { + port: 16389, defaultProvider: "a", + providers: { a: { adapter: "openai-chat", baseUrl: "https://a.test/v1", models: ["m"] } }, + apiKeys: [{ ...key, allowMeteredComboTargets: true }], + }; + saveConfig(config); + expect(loadConfig().apiKeys?.[0]?.allowMeteredComboTargets).toBe(true); + const put = async (target: unknown) => { + const req = new ManagementRequest("http://localhost:16389/api/combos", { + method: "PUT", headers: { "content-type": "application/json" }, + body: JSON.stringify({ id: "chain", combo: { strategy: "failover", targets: [target] } }), + }); + return handleManagementAPI(req, new URL(req.url), config, { + createManagementConvergeCodex: catalogConvergenceFactory(async () => {}), + }); + }; + expect((await put({ provider: "a", model: "m", metered: true }))?.status).toBe(200); + expect((await put({ provider: " a ", model: " m " }))?.status).toBe(200); + expect(config.combos?.chain?.targets[0]?.metered).toBe(true); + const disk = () => JSON.parse(readFileSync(join(home, "config.json"), "utf8")) as OcxConfig; + expect(disk().combos?.chain?.targets[0]?.metered).toBe(true); + expect((await put({ provider: "a", model: "m", metered: "true" }))?.status).toBe(400); + expect(disk().combos?.chain?.targets[0]?.metered).toBe(true); + expect((await put({ provider: "a", model: "m", metered: false }))?.status).toBe(200); + expect(disk().combos?.chain?.targets[0]).not.toHaveProperty("metered"); + writeFileSync(join(home, "config.json"), JSON.stringify({ ...config, apiKeys: [{ ...key, allowMeteredComboTargets: "true" }] })); + expect(loadConfig().apiKeys ?? []).toHaveLength(0); + } finally { + if (previous === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = previous; + removeTreeWithRetry(home); + } +}); From c57e86315c260ceee7d6c07f5613b16aa07c0781 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 18:45:11 +0900 Subject: [PATCH 2/9] fix(combos): refuse metered picks on collapsed combo routes without consent A subagent model fallback that names a combo alias is resolved by the router into one combo pick and dispatched outside the combo loop, so its metered-target filter never ran: a caller without allowMeteredComboTargets reached the metered target once the subscription target was cooling. captureInboundRoutePolicy, which every route on the Responses path passes through, now refuses a metered collapsed pick without consent. Docs state that a loopback bind reads no key, so metered targets need a keyed listener. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- docs-site/src/content/docs/guides/combos.md | 2 +- .../docs/reference/configuration/providers.md | 2 +- src/server/responses/request-prepare.ts | 7 +++ src/types/config.ts | 4 +- structure/runtime.md | 2 +- tests/server/combo-metered-consent.test.ts | 58 +++++++++++++++++++ 6 files changed, 70 insertions(+), 5 deletions(-) diff --git a/docs-site/src/content/docs/guides/combos.md b/docs-site/src/content/docs/guides/combos.md index 375b14dbe75..ce25e3e4fc0 100644 --- a/docs-site/src/content/docs/guides/combos.md +++ b/docs-site/src/content/docs/guides/combos.md @@ -17,7 +17,7 @@ if `provider/model` selectors are new to you. ## Metered fallback consent -Mark a pay-per-use combo target with `"metered": true` when it follows subscription targets. Every caller skips marked targets unless its configured `apiKeys[]` entry has `"allowMeteredComboTargets": true`. Omitted or false consent denies metered dispatch; environment-token and loopback callers must use an opted-in configured key. Model/provider scopes still apply. This applies to initial selection, failover and recovery across combo strategies. Unmarked targets retain existing behavior; this flag does not detect billing automatically. +Mark a pay-per-use combo target with `"metered": true` when it follows subscription targets. Every caller skips marked targets unless its configured `apiKeys[]` entry has `"allowMeteredComboTargets": true`. Omitted or false consent denies metered dispatch, and the environment token never qualifies. A loopback-bound proxy admits requests without reading any key, so marked targets are reachable only through a listener that requires API keys, using an opted-in key. Model/provider scopes still apply. This applies to initial selection, failover and recovery across combo strategies, and to a subagent model fallback that names a combo alias. Unmarked targets retain existing behavior; this flag does not detect billing automatically. For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without key consent, exhaustion returns the last unmetered error. Dashboard saves retain an omitted metered flag; explicitly setting it to false clears it. Set the key consent in the config file; the dashboard does not expose this field. diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index ff4dc8f4c1d..26128453037 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -1224,7 +1224,7 @@ provider in question. ### Managed record shapes -`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. Optional `allowMeteredComboTargets: true` explicitly permits targets marked `metered: true`; omitted or false consent denies them. Loopback and environment-token callers must use an opted-in configured key. See [Metered fallback consent](/guides/combos/#metered-fallback-consent). +`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. Optional `allowMeteredComboTargets: true` explicitly permits targets marked `metered: true`; omitted or false consent denies them. The environment token and unauthenticated loopback requests never qualify. See [Metered fallback consent](/guides/combos/#metered-fallback-consent). `codexAccounts[]` entries require `id`, `email`, and `isMain`, with optional `plan`, `chatgptAccountId`, and privacy-safe `logLabel`. These records are normally dashboard-managed. diff --git a/src/server/responses/request-prepare.ts b/src/server/responses/request-prepare.ts index 4cf050b1abd..0e151a638bd 100644 --- a/src/server/responses/request-prepare.ts +++ b/src/server/responses/request-prepare.ts @@ -133,6 +133,7 @@ import { } from "../../codex/loopback-target"; import { checkComboTargetInputAdmission, checkInputAdmission } from "./input-admission"; import { + admissionMayUseMeteredComboTargets, admissionModelDeniedResponse, AdmissionModelDeniedError, assertRouteAllowedByScope, @@ -601,6 +602,7 @@ export async function prepareResponsesRequest( return formatErrorResponse(401, "authentication_error", "Anthropic Pool 2 requires an enabled configured provider"); } const admissionScope = resolveAdmissionModelScope(config, options.admission); + const meteredComboTargetsAllowed = admissionMayUseMeteredComboTargets(config, options.admission); const captureInboundRoutePolicy = (candidate: RouteResult, captureRequestPolicy = true): RouteResult => { // Every route this request path produces passes through here: the direct // name, an alias, a policy or combo selection, a compaction override, a @@ -624,6 +626,11 @@ export async function prepareResponsesRequest( providerName: candidate.providerName, modelId: previewXaiOauthWireModel(parsed, candidate, config, inboundWire), }); + // A combo the router collapsed to one pick here (a subagent fallback naming a combo alias) + // never reaches the combo loop's metered filter, so consent is enforced again and fails closed. + if (candidate.combo?.target.metered && !meteredComboTargetsAllowed) { + throw new NoAvailableComboTargetsError(candidate.combo.comboId); + } return candidate; }; try { diff --git a/src/types/config.ts b/src/types/config.ts index 9f04ed1baa6..93b21cc7e95 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -351,8 +351,8 @@ export interface OcxApiKeyEntry { allowedModels?: string[]; /** * Lets this key reach combo targets marked `metered`. Absent means no: an issued key stops - * at the last unmetered target. Environment-token and loopback callers must use an opted-in - * configured key to reach metered targets. + * at the last unmetered target. The environment token and unauthenticated loopback requests + * never qualify; a loopback bind reads no key at all. */ allowMeteredComboTargets?: boolean; } diff --git a/structure/runtime.md b/structure/runtime.md index 3b7c2126fb1..a8b54f87620 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -488,7 +488,7 @@ Renamed fixed-key providers receive [missing reasoning metadata](catalog.md#rena Translated audio/file admission follows the [final-adapter input contract](adapters/registry.md#untranslated-input-media); native raw passthrough remains separate. ## Request-local target compatibility -`src/server/admission-model-scope.ts` grants metered combo eligibility only to a matching configured key with literal `allowMeteredComboTargets: true`. `src/server/responses/core-combo.ts` filters marked targets on initial, failure and recovery selection for every strategy. Missing identity, environment-token and loopback admission deny marked targets; provider/model scopes remain independent. `src/cli/combo-input.ts` accepts and validates the optional target boolean. Unmarked targets retain existing behavior; metering is operator metadata, not automatic billing detection. Coverage: `tests/server/combo-metered-consent.test.ts` and `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. +`src/server/admission-model-scope.ts` grants metered combo eligibility only to a matching configured key with literal `allowMeteredComboTargets: true`. `src/server/responses/core-combo.ts` filters marked targets on initial, failure and recovery selection for every strategy. A route the router collapsed to one combo pick outside that loop (a subagent fallback naming a combo alias) is refused in `captureInboundRoutePolicy` of `src/server/responses/request-prepare.ts` when the pick is metered and consent is absent. Missing identity, environment-token and loopback admission deny marked targets; provider/model scopes remain independent. `src/cli/combo-input.ts` accepts and validates the optional target boolean. Unmarked targets retain existing behavior; metering is operator metadata, not automatic billing detection. Coverage: `tests/server/combo-metered-consent.test.ts` and `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. `src/adapters/openai-responses.ts` omits only top-level `user` at the canonical ChatGPT Codex forward destination. Claude translation retains its original identity and prompt-cache key; public API and noncanonical gateways retain their `user` field. Input roles, tool-schema properties, safety identifiers and original replay bodies are not changed. diff --git a/tests/server/combo-metered-consent.test.ts b/tests/server/combo-metered-consent.test.ts index 82100ce9637..d98db2709ce 100644 --- a/tests/server/combo-metered-consent.test.ts +++ b/tests/server/combo-metered-consent.test.ts @@ -11,6 +11,10 @@ import { handleManagementAPI } from "../../src/server/management-api"; import { ManagementRequest } from "../helpers/management-auth"; import { catalogConvergenceFactory } from "../helpers/catalog-convergence"; import { removeTreeWithRetry } from "../helpers/remove-tree"; +import { handleResponses } from "../../src/server/responses"; +import { noteSubagentModelFailure, resetSubagentModelFallbackStateForTests } from "../../src/codex/subagent-model-fallback"; +import { clearComboTargetCooldowns, coolComboTarget } from "../../src/combos/failover"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; const key = { id: "issued", name: "issued", key: "ocx_data_" + "a".repeat(40), createdAt: "2026-10-15T00:00:00Z" }; const admission = { kind: "configured", keyId: key.id, source: "bearer" } as const; @@ -73,3 +77,57 @@ test("metered metadata survives dashboard saves, explicit false clears it, malfo removeTreeWithRetry(home); } }); + +// A subagent fallback naming a combo alias is collapsed to one pick by the router and dispatched +// outside the combo loop, so its metered pick must be refused there too. +test("a subagent fallback that collapses a combo onto its metered target needs key consent", async () => { + const previous = { home: process.env.OPENCODEX_HOME, codex: process.env.CODEX_HOME }; + const home = mkdtempSync(join(tmpdir(), "ocx-metered-fallback-")); + process.env.OPENCODEX_HOME = home; + process.env.CODEX_HOME = home; + const release = acquireOwnedSpendHome(); + const originalFetch = globalThis.fetch; + const provider = (host: string) => ({ adapter: "openai-chat" as const, baseUrl: `https://${host}.test/v1`, apiKey: "k", models: ["m"] }); + const urls: string[] = []; + globalThis.fetch = (async (input: string | URL | Request) => { + urls.push(String(input)); + return new Response(JSON.stringify({ error: { message: "boom" } }), { status: 500, headers: { "content-type": "application/json" } }); + }) as typeof fetch; + const post = (config: OcxConfig, consented: boolean) => { + resetSubagentModelFallbackStateForTests(); + noteSubagentModelFailure("primary/m", "429", config); + clearComboTargetCooldowns("chain"); + coolComboTarget("chain", { provider: "subscription", model: "m" }, { cooldownMs: 600_000 }); + urls.length = 0; + return handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", + headers: { "content-type": "application/json", "x-openai-subagent": "collab_spawn" }, + body: JSON.stringify({ model: "primary/m", input: "hi", stream: false }), + }), config, { model: "", provider: "" }, consented ? { admission } : {}); + }; + try { + const config = (consent: boolean): OcxConfig => ({ + port: 16389, defaultProvider: "primary", + providers: { primary: provider("primary"), subscription: provider("subscription"), paid: provider("paid") }, + combos: { chain: { alias: "chain-model", targets: [{ provider: "subscription", model: "m" }, { provider: "paid", model: "m", metered: true }] } }, + subagentModelFallback: ["chain-model"], + apiKeys: [{ ...key, allowMeteredComboTargets: consent }], + }); + expect((await post(config(false), false)).status).toBe(503); + expect(urls).toEqual([]); + expect((await post(config(false), true)).status).toBe(503); + expect(urls).toEqual([]); + await post(config(true), true); + expect(urls).toEqual(["https://paid.test/v1/chat/completions"]); + } finally { + globalThis.fetch = originalFetch; + resetSubagentModelFallbackStateForTests(); + clearComboTargetCooldowns("chain"); + release(); + for (const [name, value] of [["OPENCODEX_HOME", previous.home], ["CODEX_HOME", previous.codex]] as const) { + if (value === undefined) delete process.env[name]; + else process.env[name] = value; + } + removeTreeWithRetry(home); + } +}); From f7a724c07e7b3a728a4f294e34a0b1a91ad47002 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 18:58:36 +0900 Subject: [PATCH 3/9] test(responses): pin structured combo-child preflight limits The carry now forwards a runTurn adapter's structured status, type and code at the combo preflight boundary, so a Devin pre-output limit inside a combo child reaches the combo loop as a 429 rate_limit_exceeded instead of a generic 502. #5890 pinned the 502 only to keep its own non-combo change out of combo children; the pin now states the new contract, still without a Retry-After header, which the combo loop owns. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- tests/responses/responses-grok-devin-preflight.test.ts | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/responses/responses-grok-devin-preflight.test.ts b/tests/responses/responses-grok-devin-preflight.test.ts index 34909f10501..eb2dbb52c58 100644 --- a/tests/responses/responses-grok-devin-preflight.test.ts +++ b/tests/responses/responses-grok-devin-preflight.test.ts @@ -338,11 +338,12 @@ test("timed-out OAuth replay keeps its reserved dispatch permit until Devin send expect(retryBudgetUsed).toBe(2); }); -test("combo children retain their existing preflight failure", async () => { +// A combo child hands the structured limit to the combo loop, which owns cooldown and Retry-After. +test("combo children carry the structured preflight limit without Retry-After", async () => { const response = await run({ comboAttempt: true }); - expect(response.status).toBe(502); + expect(response.status).toBe(429); expect(response.headers.get("retry-after")).toBeNull(); - await response.text(); + expect(JSON.parse(await response.text()).error).toMatchObject({ type: "rate_limit_error", code: "rate_limit_exceeded" }); }); test.each([ From 59d565eb8a5d5c59958a033435bb96ef4152e172 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 19:09:25 +0900 Subject: [PATCH 4/9] feat(combos): add explicit loopback metered consent --- docs-site/src/content/docs/guides/combos.md | 11 +- .../src/content/docs/reference/cli/agents.md | 1 + .../docs/reference/configuration/providers.md | 2 +- .../docs/reference/configuration/routing.md | 3 +- scripts/test-layout/layout.json | 2 +- .../references/01_surface_agents-routing.md | 3 +- src/cli/capabilities-agents-routing.ts | 4 +- src/cli/combo-input.ts | 6 + src/cli/combo-result.ts | 2 +- src/cli/combo.ts | 3 +- src/combos/types.ts | 8 +- src/server/admission-model-scope.ts | 17 +- src/server/management/combo-routes.ts | 3 + src/server/responses/core-combo.ts | 2 +- src/server/responses/request-prepare.ts | 18 ++- src/types/config.ts | 9 +- structure/cli-management.md | 2 +- structure/config.md | 2 +- structure/gui-and-management-api.md | 2 +- structure/runtime.md | 2 +- .../anthropic-subscription-chain.test.ts | 26 ++- tests/cli/cli-combo-partial-update.test.ts | 17 ++ tests/fixtures/test-layout-expected.json | 2 +- .../combo-loopback-metered-consent.test.ts | 150 ++++++++++++++++++ tests/server/combo-metered-consent.test.ts | 20 ++- 25 files changed, 281 insertions(+), 36 deletions(-) create mode 100644 tests/server/combo-loopback-metered-consent.test.ts diff --git a/docs-site/src/content/docs/guides/combos.md b/docs-site/src/content/docs/guides/combos.md index ce25e3e4fc0..a0d52d0182e 100644 --- a/docs-site/src/content/docs/guides/combos.md +++ b/docs-site/src/content/docs/guides/combos.md @@ -17,9 +17,14 @@ if `provider/model` selectors are new to you. ## Metered fallback consent -Mark a pay-per-use combo target with `"metered": true` when it follows subscription targets. Every caller skips marked targets unless its configured `apiKeys[]` entry has `"allowMeteredComboTargets": true`. Omitted or false consent denies metered dispatch, and the environment token never qualifies. A loopback-bound proxy admits requests without reading any key, so marked targets are reachable only through a listener that requires API keys, using an opted-in key. Model/provider scopes still apply. This applies to initial selection, failover and recovery across combo strategies, and to a subagent model fallback that names a combo alias. Unmarked targets retain existing behavior; this flag does not detect billing automatically. +Mark a pay-per-use combo target with `"metered": true` when it follows subscription targets. Omitted or false consent denies metered dispatch. Choose consent for your deployment: -For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without key consent, exhaustion returns the last unmetered error. Dashboard saves retain an omitted metered flag; explicitly setting it to false clears it. Set the key consent in the config file; the dashboard does not expose this field. +- **Local Claude Code or Codex:** on a loopback-only proxy, set `"allowMeteredTargets": true` on the combo, or run `ocx combo set --allow-metered-targets on`. This explicitly permits local callers to use its marked targets. The default is false; `off` clears consent. Loopback authentication does not read API keys. +- **Network or linked callers:** the caller's configured `apiKeys[]` entry must have `"allowMeteredComboTargets": true`. A proxy with a non-loopback listener ignores the combo flag, including on its secondary loopback listener. Linked callers also retain their configured-key consent requirement when the primary listener is loopback. The environment token never qualifies. + +Request headers cannot grant consent. Model/provider scopes still apply. This covers initial selection, failover and recovery across combo strategies, compaction, Messages, Chat Completions and WebSocket requests, and a subagent model fallback naming a combo alias. A collapsed fallback tries a usable unmetered target before refusing. Unmarked targets retain existing behavior; metering is declared by the operator rather than detected automatically. + +For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without consent, exhaustion returns the last unmetered error. Dashboard saves retain omitted target metering and combo consent; explicitly setting either to false clears it. These consent fields are configured through the config file or API; the combo flag is also available through the CLI, while the dashboard does not expose it. ## 60-second quickstart @@ -767,7 +772,7 @@ ocx combo remove --yes ``` `set` also accepts `--strategy`, `--sticky`, `--effort`, `--alias`, `--native-alias`, -`--display-name`, `--decision-provider`, `--decision-timeout`, and `--rename-from`. Use `-` as the +`--allow-metered-targets on|off`, `--display-name`, `--decision-provider`, `--decision-timeout`, and `--rename-from`. Use `-` as the value of `--effort`, `--alias`, `--display-name`, `--decision-provider`, or `--decision-timeout` to clear that field. The two decision flags apply only to `--strategy jev`. `--native-alias` requires a currently supported bare native model alias and a non-empty display name. `create` and `update` are aliases for `set`; `delete` is an alias for diff --git a/docs-site/src/content/docs/reference/cli/agents.md b/docs-site/src/content/docs/reference/cli/agents.md index 882c7ef59db..dcbe3e8548a 100644 --- a/docs-site/src/content/docs/reference/cli/agents.md +++ b/docs-site/src/content/docs/reference/cli/agents.md @@ -302,6 +302,7 @@ explicit `lastResort: false`. `--image-input auto|disabled` and `--reasoning-effort-mode strict|adaptive` preserve omission, while explicit `auto` and `strict` override saved disabled/adaptive values. Reasoning-effort mode is separate from `--effort-mode fallback|force`, which governs the default effort. +`--allow-metered-targets on|off` grants or clears operator consent for marked combo targets on a loopback-only proxy; public and linked callers still need configured-key consent. `--native-alias on|off` preserves explicit false; the legacy bare flag means true. Turning it off may require the intended alias clear, `--native-alias off --alias -`, to avoid retaining an incompatible native alias. Set/rename is an upsert without diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 26128453037..3fb4b1649de 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -1224,7 +1224,7 @@ provider in question. ### Managed record shapes -`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. Optional `allowMeteredComboTargets: true` explicitly permits targets marked `metered: true`; omitted or false consent denies them. The environment token and unauthenticated loopback requests never qualify. See [Metered fallback consent](/guides/combos/#metered-fallback-consent). +`apiKeys[]` entries contain `id`, `name`, generated `key`, and ISO `createdAt` strings. Optional `allowMeteredComboTargets: true` explicitly permits targets marked `metered: true`; omitted or false consent denies them. The environment token never qualifies. Local loopback-only callers can instead use per-combo `allowMeteredTargets: true`; configured-key and linked callers still need key consent. See [Metered fallback consent](/guides/combos/#metered-fallback-consent). `codexAccounts[]` entries require `id`, `email`, and `isMain`, with optional `plan`, `chatgptAccountId`, and privacy-safe `logLabel`. These records are normally dashboard-managed. diff --git a/docs-site/src/content/docs/reference/configuration/routing.md b/docs-site/src/content/docs/reference/configuration/routing.md index 19d397ec460..0a299f7cf93 100644 --- a/docs-site/src/content/docs/reference/configuration/routing.md +++ b/docs-site/src/content/docs/reference/configuration/routing.md @@ -138,7 +138,8 @@ namespace, and cannot use reserved bare native families such as `gpt-*`, `o1-*`, | Key | Type | Default | Meaning | | --- | --- | --- | --- | -| `targets` | `{ provider: string; model: string; weight?: number; lastResort?: boolean; metered?: boolean }[]` | required | Ordered concrete routes. `weight` is 1–10000 and defaults to `1`. `lastResort` marks an emergency-only target; see `cooldownWaitPolicy`. `metered: true` requires configured-key `allowMeteredComboTargets: true`, including local callers; see [consent](/guides/combos/#metered-fallback-consent). | +| `targets` | `{ provider: string; model: string; weight?: number; lastResort?: boolean; metered?: boolean }[]` | required | Ordered concrete routes. `weight` is 1–10000 and defaults to `1`. `lastResort` marks an emergency-only target; see `cooldownWaitPolicy`. `metered: true` requires key consent or loopback-only combo consent; see [consent](/guides/combos/#metered-fallback-consent). | +| `allowMeteredTargets?` | `boolean` | `false` | Operator consent for local callers on a loopback-only proxy. A non-loopback listener disables this flag; configured-key and linked callers still need key consent. | | `strategy?` | `"failover" \| "round-robin" \| "random" \| "least-used" \| "reset-window" \| "jev"` | `"failover"` | Selection strategy. Target order is failover priority; weights shape round-robin and random draws; least-used follows recorded successes; reset-window follows the soonest quota reset; JEV makes one bounded decision for the initial eligible target and effort, then uses ordinary ordered fallback. | | `stickyLimit?` | `number` | `1` | Successful requests retained in one round-robin batch. Range 1–100. Applies only to round-robin. | | `cooldownMs?` | `number` | unset → upstream fallback (5 s for request-rate 429 codes `1302`/`1305`, otherwise 60 s) | Range 1–600000. When set, applies whenever no usable upstream `Retry-After` or Codex reset signal exists, including request-rate 429s; when unset, uses the upstream fallback. Upstream signals take precedence. An explicit upstream `Retry-After` is capped at 24 hours; reset-derived, configured, and fallback cooldowns are capped at 10 minutes. | diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 8e11bae7b0f..b8ae9848571 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -14,7 +14,7 @@ "anthropic-instance-isolation.test.ts": "adapters/anthropic", "anthropic-instance-pool-parity.test.ts": "adapters/anthropic", "anthropic-instance-quota.test.ts": "adapters/anthropic", "anthropic-instance-recovery.test.ts": "adapters/anthropic", "anthropic2-fast-parity.test.ts": "adapters/anthropic", "anthropic2-native-routing.test.ts": "claude-integration", - "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", + "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "combo-loopback-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", "oauth-anthropic-instance-multiprocess.test.ts": "oauth", "anthropic2-discovery-ownership.test.ts": "oauth", "codex-shim-runtime-preflight.test.ts": "codex-integration", diff --git a/skills/ocx/references/01_surface_agents-routing.md b/skills/ocx/references/01_surface_agents-routing.md index 271f857e6ca..5be913fe439 100644 --- a/skills/ocx/references/01_surface_agents-routing.md +++ b/skills/ocx/references/01_surface_agents-routing.md @@ -769,7 +769,7 @@ JSON mode: `payload`. ### `ocx combo set` -Usage: `ocx combo set [--targets | --targets-file ] [--strategy ] [--sticky <1-100|->] [--effort ] [--effort-mode ] [--alias ] [--native-alias [on|off]] [--display-name ] [--decision-provider | --decision-model ] [--decision-timeout ] [--rename-from ] [--image-input ] [--reasoning-effort-mode ] [--json]` +Usage: `ocx combo set [--targets | --targets-file ] [--strategy ] [--sticky <1-100|->] [--effort ] [--effort-mode ] [--alias ] [--native-alias [on|off]] [--allow-metered-targets ] [--display-name ] [--decision-provider | --decision-model ] [--decision-timeout ] [--rename-from ] [--image-input ] [--reasoning-effort-mode ] [--json]` Create, update or rename a combo. @@ -782,6 +782,7 @@ State-changing: yes. | Flag | Value | Meaning | |---|---|---| +| `--allow-metered-targets` | string | on grants operator consent to metered targets only on a loopback-only proxy; off clears it. | | `--targets` | string | Ordered provider/model[:weight] list; required for creation. | | `--strategy` | string | Routing strategy. | | `--sticky` | string | Round-robin limit 1-100; - resets. | diff --git a/src/cli/capabilities-agents-routing.ts b/src/cli/capabilities-agents-routing.ts index bc5ac262e47..cfd7ec51d07 100644 --- a/src/cli/capabilities-agents-routing.ts +++ b/src/cli/capabilities-agents-routing.ts @@ -295,10 +295,10 @@ export const AGENT_ROUTING_CAPABILITIES: readonly Capability[] = [ }, { command: ["combo","set"], - usage: "ocx combo set [--targets | --targets-file ] [--strategy ] [--sticky <1-100|->] [--effort ] [--effort-mode ] [--alias ] [--native-alias [on|off]] [--display-name ] [--decision-provider | --decision-model ] [--decision-timeout ] [--rename-from ] [--image-input ] [--reasoning-effort-mode ] [--json]", + usage: "ocx combo set [--targets | --targets-file ] [--strategy ] [--sticky <1-100|->] [--effort ] [--effort-mode ] [--alias ] [--native-alias [on|off]] [--allow-metered-targets ] [--display-name ] [--decision-provider | --decision-model ] [--decision-timeout ] [--rename-from ] [--image-input ] [--reasoning-effort-mode ] [--json]", summary: "Create, update or rename a combo.", routes: [{"method":"GET","path":"/api/combos"},{"method":"PUT","path":"/api/combos"}], - flags: [{"name":"--targets","value":"string","summary":"Ordered provider/model[:weight] list; required for creation."},{"name":"--strategy","value":"string","summary":"Routing strategy."},{"name":"--sticky","value":"string","summary":"Round-robin limit 1-100; - resets."},{"name":"--effort","value":"string","summary":"Default effort; - clears."},{"name":"--effort-mode","value":"string","summary":"fallback or force; - resets to fallback."},{"name":"--alias","value":"string","summary":"Alias; - clears."},{"name":"--display-name","value":"string","summary":"Label; - clears."},{"name":"--decision-provider","value":"string","summary":"JEV provider; - clears."},{"name":"--decision-model","value":"string","summary":"JEV route; - clears; exclusive with provider."},{"name":"--decision-timeout","value":"string","summary":"JEV deadline 1000-120000 ms; - clears."},{"name":"--rename-from","value":"string","summary":"Existing combo ID to rename."},{"name":"--json","value":"boolean","summary":"Emit structured JSON."},{"name":"--native-alias","value":"string","summary":"Bare legacy flag means on; explicit on/off preserves presence, including false."},{"name":"--targets-file","value":"string","summary":"Bounded ordered target JSON array, exclusive with --targets; preserves effort/profile/last-resort metadata."},{"name":"--image-input","value":"string","summary":"auto or disabled; explicit auto clears the previous disabled choice."},{"name":"--reasoning-effort-mode","value":"string","summary":"strict or adaptive, distinct from --effort-mode fallback/force."}], + flags: [{"name":"--allow-metered-targets","value":"string","summary":"on grants operator consent to metered targets only on a loopback-only proxy; off clears it."},{"name":"--targets","value":"string","summary":"Ordered provider/model[:weight] list; required for creation."},{"name":"--strategy","value":"string","summary":"Routing strategy."},{"name":"--sticky","value":"string","summary":"Round-robin limit 1-100; - resets."},{"name":"--effort","value":"string","summary":"Default effort; - clears."},{"name":"--effort-mode","value":"string","summary":"fallback or force; - resets to fallback."},{"name":"--alias","value":"string","summary":"Alias; - clears."},{"name":"--display-name","value":"string","summary":"Label; - clears."},{"name":"--decision-provider","value":"string","summary":"JEV provider; - clears."},{"name":"--decision-model","value":"string","summary":"JEV route; - clears; exclusive with provider."},{"name":"--decision-timeout","value":"string","summary":"JEV deadline 1000-120000 ms; - clears."},{"name":"--rename-from","value":"string","summary":"Existing combo ID to rename."},{"name":"--json","value":"boolean","summary":"Emit structured JSON."},{"name":"--native-alias","value":"string","summary":"Bare legacy flag means on; explicit on/off preserves presence, including false."},{"name":"--targets-file","value":"string","summary":"Bounded ordered target JSON array, exclusive with --targets; preserves effort/profile/last-resort metadata."},{"name":"--image-input","value":"string","summary":"auto or disabled; explicit auto clears the previous disabled choice."},{"name":"--reasoning-effort-mode","value":"string","summary":"strict or adaptive, distinct from --effort-mode fallback/force."}], mutates: true, json: "payload", details: ["create and update are aliases of set. Existing fields are read before writing; --targets replaces targets.","force overrides valid client effort and may increase cost and latency.","Only the implemented target grammar is supported; full GUI target metadata and routing-profile writes are separate tasks.","New file/flag conflicts refuse before transport. Partial edits preserve target metadata; explicit false/default values override server carry. Turning a native alias off may also require clearing its incompatible alias.","Read/merge/write uses one target and remains an upsert without revision CAS. Saved-but-incomplete catalog outcomes return nonzero with their receipt."], diff --git a/src/cli/combo-input.ts b/src/cli/combo-input.ts index 2ef107645ff..6293358ebf4 100644 --- a/src/cli/combo-input.ts +++ b/src/cli/combo-input.ts @@ -8,6 +8,7 @@ type ComboInput = { imageInput?: "auto" | "disabled"; reasoningEffortMode?: "strict" | "adaptive"; nativeAlias?: boolean; + allowMeteredTargets?: boolean; }; const TARGET_KEYS = new Set(["provider", "model", "weight", "reasoningEfforts", "modelProfile", "lastResort", "metered"]); @@ -73,6 +74,10 @@ export async function prepareComboInput(args: string[], deps: RuntimeApiDeps = { const image = takeOptionWithSyntax(args, "--image-input")?.value; const reasoning = takeOptionWithSyntax(args, "--reasoning-effort-mode")?.value; const nativeAlias = takeNativeAlias(args); + const metered = takeOptionWithSyntax(args, "--allow-metered-targets")?.value; + if (metered !== undefined && metered !== "on" && metered !== "off") { + throw new CliUsageError("--allow-metered-targets must be on or off"); + } if (file !== undefined && args.some(arg => arg === "--targets" || arg.startsWith("--targets="))) { throw new CliUsageError("--targets-file cannot be combined with --targets"); } @@ -87,5 +92,6 @@ export async function prepareComboInput(args: string[], deps: RuntimeApiDeps = { ...(image === undefined ? {} : { imageInput: image }), ...(reasoning === undefined ? {} : { reasoningEffortMode: reasoning }), ...(nativeAlias === undefined ? {} : { nativeAlias }), + ...(metered === undefined ? {} : { allowMeteredTargets: metered === "on" }), }; } diff --git a/src/cli/combo-result.ts b/src/cli/combo-result.ts index 0b90b0b5a61..97a0ad0aac3 100644 --- a/src/cli/combo-result.ts +++ b/src/cli/combo-result.ts @@ -33,7 +33,7 @@ function publicCombo(value: unknown): Record { } else if (Object.hasOwn(RANGES, key)) { const [min, max] = RANGES[key]!; if (typeof child !== "number" || !Number.isSafeInteger(child) || child < min || child > max) invalid(); - } else if (key === "nativeAlias") { + } else if (key === "nativeAlias" || key === "allowMeteredTargets") { if (typeof child !== "boolean") invalid(); } else invalid(); result[key] = child; diff --git a/src/cli/combo.ts b/src/cli/combo.ts index ad4c1e52e75..c29dbf1fa35 100644 --- a/src/cli/combo.ts +++ b/src/cli/combo.ts @@ -23,7 +23,7 @@ const USAGE = `Usage: [--strategy ] [--sticky <1-100|->] [--effort ] [--effort-mode ] (force overrides valid client effort and can increase cost/latency) [--alias ] - [--native-alias [on|off]] [--display-name ] + [--native-alias [on|off]] [--allow-metered-targets ] [--display-name ] [--image-input ] [--reasoning-effort-mode ] [--decision-provider | --decision-model ] [--decision-timeout ] (jev only; the provider must be a configured jev-decision row) @@ -160,6 +160,7 @@ async function set(argv: string[], deps: RuntimeApiDeps): Promise { if (effortMode !== undefined) combo.defaultEffortMode = effortMode === "-" ? "fallback" : effortMode; if (alias !== undefined) combo.alias = alias === "-" ? "" : alias; if (additional.nativeAlias !== undefined) combo.nativeAlias = additional.nativeAlias; + if (additional.allowMeteredTargets !== undefined) combo.allowMeteredTargets = additional.allowMeteredTargets; if (additional.imageInput !== undefined) combo.imageInput = additional.imageInput; if (additional.reasoningEffortMode !== undefined) combo.reasoningEffortMode = additional.reasoningEffortMode; if (displayName !== undefined) combo.displayName = displayName === "-" ? "" : displayName; diff --git a/src/combos/types.ts b/src/combos/types.ts index d11a27baf9a..2823301dd1c 100644 --- a/src/combos/types.ts +++ b/src/combos/types.ts @@ -39,7 +39,7 @@ export interface NormalizedComboTarget { weight: number; /** Emergency-only target, deferred under `cooldownWaitPolicy` (#5691). */ lastResort: boolean; - /** Pay-per-use target; issued keys reach it only with `allowMeteredComboTargets`. */ + /** Pay-per-use target; requires key consent or loopback-only operator consent. */ metered: boolean; reasoningEfforts?: OcxComboDefaultEffort[]; /** Optional JEV decision description. */ @@ -47,6 +47,8 @@ export interface NormalizedComboTarget { } export interface NormalizedComboConfig { + /** Operator consent, effective only on a loopback-only proxy. */ + allowMeteredTargets?: boolean; strategy: OcxComboStrategy; stickyLimit: number; cooldownMs?: number; @@ -166,6 +168,9 @@ export function comboConfigIssues( } const body = raw as Record; + if (body.allowMeteredTargets !== undefined && typeof body.allowMeteredTargets !== "boolean") { + issues.push({ path: ["allowMeteredTargets"], message: "allowMeteredTargets must be a boolean" }); + } if (typeof body.alias === "string") { const alias = body.alias.toLowerCase(); for (const [providerName, provider] of Object.entries(providers)) { @@ -485,6 +490,7 @@ export function normalizeComboConfig(raw: OcxComboConfig): NormalizedComboConfig : null; return { strategy: raw.strategy ?? "failover", + ...(raw.allowMeteredTargets === true ? { allowMeteredTargets: true } : {}), stickyLimit: raw.stickyLimit ?? 1, cooldownMs: raw.cooldownMs, waitForCooldownMs: raw.waitForCooldownMs ?? COMBO_DEFAULT_WAIT_FOR_COOLDOWN_MS, diff --git a/src/server/admission-model-scope.ts b/src/server/admission-model-scope.ts index 08c98681ad0..4a1b4aee57b 100644 --- a/src/server/admission-model-scope.ts +++ b/src/server/admission-model-scope.ts @@ -1,4 +1,5 @@ import type { DataPlaneAdmission } from "./auth-cors"; +import { isApiAuthRequired } from "./auth-cors"; import type { OcxApiKeyEntry, OcxConfig } from "../types"; import { markResponseNonReplayable } from "../lib/upstream-retry"; @@ -77,14 +78,20 @@ export function resolveAdmissionModelScope( return entry ? admissionModelScopeOf(entry) : undefined; } -/** Metered combo dispatch requires an explicitly opted-in configured key. */ +/** Consent comes from stored operator config, never request headers. Use the full listener config. */ export function admissionMayUseMeteredComboTargets( - config: Pick, + config: Pick, admission: DataPlaneAdmission | undefined, + combo?: { allowMeteredTargets?: boolean }, ): boolean { - if (!admission || admission.kind !== "configured") return false; - const entry = (config.apiKeys ?? []).find(key => key.id === admission.keyId); - return entry?.allowMeteredComboTargets === true; + if (admission?.kind === "configured") { + const entry = (config.apiKeys ?? []).find(key => key.id === admission.keyId); + return entry?.allowMeteredComboTargets === true; + } + // Linked ingress always carries keyed admission, even when the primary bind is loopback. + // A secondary loopback listener on a public proxy does not qualify either. + return (!admission || admission.kind === "loopback") + && !isApiAuthRequired(config) && combo?.allowMeteredTargets === true; } /** diff --git a/src/server/management/combo-routes.ts b/src/server/management/combo-routes.ts index 30ca7ab341c..ff2a89fe06a 100644 --- a/src/server/management/combo-routes.ts +++ b/src/server/management/combo-routes.ts @@ -176,6 +176,9 @@ export async function handleComboRoutes(ctx: ManagementContext): Promise = body.combo; const effectiveCombo = { ...requestedCombo, + ...(!Object.hasOwn(requestedCombo, "allowMeteredTargets") && previous?.allowMeteredTargets === true + ? { allowMeteredTargets: true } + : {}), ...(!Object.hasOwn(requestedCombo, "cooldownMs") && previous?.cooldownMs !== undefined ? { cooldownMs: previous.cooldownMs } : {}), diff --git a/src/server/responses/core-combo.ts b/src/server/responses/core-combo.ts index 657bc8c5665..8ed7f33ac5a 100644 --- a/src/server/responses/core-combo.ts +++ b/src/server/responses/core-combo.ts @@ -449,7 +449,7 @@ export async function executeComboResponses( let comboPayloadReadable = false; const payloadEligible = (target: (typeof combo.targets)[number]): boolean => comboPayloadReadable || !unreadableEncryptedAgentTask || canDecryptUnreadableAgentTask(target); - const meteredAllowed = admissionMayUseMeteredComboTargets(config, options.admission); + const meteredAllowed = admissionMayUseMeteredComboTargets(config, options.admission, combo); const targetEligible = (target: (typeof combo.targets)[number]): boolean => (meteredAllowed || !target.metered) && (combo.strategy !== "jev" || target.provider !== JEV_PROVIDER_ID) diff --git a/src/server/responses/request-prepare.ts b/src/server/responses/request-prepare.ts index 0e151a638bd..33a7dc26051 100644 --- a/src/server/responses/request-prepare.ts +++ b/src/server/responses/request-prepare.ts @@ -17,7 +17,7 @@ import { unreadableEncryptedAgentTaskResponse, } from "./core-errors"; import { parseSyntheticRowId } from "../fast-row"; -import { resolveComboId, comboIdFromRawBody, NoAvailableComboTargetsError } from "../../combos"; +import { resolveComboId, comboIdFromRawBody, NoAvailableComboTargetsError, pickComboTarget } from "../../combos"; import { INTERCEPT_TARGET_UNAVAILABLE_CODE, interceptTargetUnavailableResponse, resolveChosenTarget, resolveShadowCallTarget } from "./shadow-target-availability"; import { MEMORY_MODEL_TARGET_UNAVAILABLE_CODE, @@ -602,8 +602,17 @@ export async function prepareResponsesRequest( return formatErrorResponse(401, "authentication_error", "Anthropic Pool 2 requires an enabled configured provider"); } const admissionScope = resolveAdmissionModelScope(config, options.admission); - const meteredComboTargetsAllowed = admissionMayUseMeteredComboTargets(config, options.admission); const captureInboundRoutePolicy = (candidate: RouteResult, captureRequestPolicy = true): RouteResult => { + // Router-collapsed aliases bypass the combo loop. Re-pick before capturing destination + // policy so random/RR cannot refuse while an unmetered target is still usable. + if (candidate.combo?.target.metered && !admissionMayUseMeteredComboTargets( + config, options.admission, config.combos?.[candidate.combo.comboId], + )) { + const pick = pickComboTarget(config, candidate.combo.comboId, { eligible: target => !target.metered }); + if (!pick) throw new NoAvailableComboTargetsError(candidate.combo.comboId); + const routed = routeConcreteModel(config, `${pick.target.provider}/${pick.target.model}`); + candidate = { ...routed, combo: pick, routeKind: "combo", routeReason: routed.routeReason === "blocked-model-redirect" ? "blocked-model-redirect" : "combo-pick" }; + } // Every route this request path produces passes through here: the direct // name, an alias, a policy or combo selection, a compaction override, a // shadow-intercept target and both subagent-fallback re-routes. Checking @@ -626,11 +635,6 @@ export async function prepareResponsesRequest( providerName: candidate.providerName, modelId: previewXaiOauthWireModel(parsed, candidate, config, inboundWire), }); - // A combo the router collapsed to one pick here (a subagent fallback naming a combo alias) - // never reaches the combo loop's metered filter, so consent is enforced again and fails closed. - if (candidate.combo?.target.metered && !meteredComboTargetsAllowed) { - throw new NoAvailableComboTargetsError(candidate.combo.comboId); - } return candidate; }; try { diff --git a/src/types/config.ts b/src/types/config.ts index 93b21cc7e95..75546824ddf 100644 --- a/src/types/config.ts +++ b/src/types/config.ts @@ -351,8 +351,8 @@ export interface OcxApiKeyEntry { allowedModels?: string[]; /** * Lets this key reach combo targets marked `metered`. Absent means no: an issued key stops - * at the last unmetered target. The environment token and unauthenticated loopback requests - * never qualify; a loopback bind reads no key at all. + * at the last unmetered target. The environment token never qualifies. A loopback bind + * reads no key; local operator consent is instead declared on the combo. */ allowMeteredComboTargets?: boolean; } @@ -1346,13 +1346,16 @@ export interface OcxComboTarget { lastResort?: boolean; /** * Pay-per-use target (for example an API-key row behind subscription targets). Only the - * configured keys with `allowMeteredComboTargets: true` may reach it. + * configured keys with `allowMeteredComboTargets: true`, or a loopback-only combo + * with `allowMeteredTargets: true`, may reach it. */ metered?: boolean; } export interface OcxComboConfig { targets: OcxComboTarget[]; + /** Operator consent for metered targets on a loopback-only proxy. Default false; public listeners still require key consent. */ + allowMeteredTargets?: boolean; /** Ordered failover (default), round-robin, weighted random, least-used, or quota reset-window selection. */ strategy?: OcxComboStrategy; /** Successful requests retained on one RR selection batch. Default 1; range 1..100. */ diff --git a/structure/cli-management.md b/structure/cli-management.md index ec0a2af8f04..6e5e4a868c0 100644 --- a/structure/cli-management.md +++ b/structure/cli-management.md @@ -64,7 +64,7 @@ Management route declarations describe actual HTTP calls only; local Lab automat `src/cli/models-order.ts` owns raw upstream display-name addressing and routed picker operations. Status preserves saved native identities, while manual/preset replacement requires explicit reset of native-inclusive order. Manual input is a full unique routed permutation with the observed featured prefix; settings/identity rereads reject observable drift without claiming CAS. `src/cli/model-picker-ordering.ts` mirrors the GUI's pure identity/sort projection without importing GUI code into the CLI. Independent ordering fixtures and test-only GUI conformance exercise the deliberate shared contract. Most-used consumes complete all/all requested-model usage, not representative resolved-model attribution. The registry declares the existing display-name PUT; its handler remains authoritative, including saved-but-failed HTTP503. -`src/cli/route-policy-write.ts` accepts editable-only documents, requires the caller's update revision and uses fixed create/update/delete requests. Strict nested field ownership is local; provider/alias/routing validation stays on the server, without fabricated config context. Profile writes may activate already-enabled Lab automation. `src/cli/combo-input.ts` preserves target metadata and explicit false/default options before transport; `src/cli/combo.ts` applies them through its existing merge stages, retaining a carried force policy's omitted default effort. These combo upserts are not revision CAS. `src/cli/combo-stats.ts` reports actual JEV decision/token/coverage facts and nullable/incomplete observations without a savings estimate or decision probe. +`src/cli/route-policy-write.ts` accepts editable-only documents, requires the caller's update revision and uses fixed create/update/delete requests. Strict nested field ownership is local; provider/alias/routing validation stays on the server, without fabricated config context. Profile writes may activate already-enabled Lab automation. `src/cli/combo-input.ts` preserves target metadata and explicit false/default options before transport; `src/cli/combo.ts` applies them through its existing merge stages, retaining a carried force policy's omitted default effort. `--allow-metered-targets on|off` sets the optional combo boolean; omission preserves it, explicit off clears it, and the receipt retains it. Eligibility follows the [runtime consent contract](runtime.md#request-local-target-compatibility). These combo upserts are not revision CAS. `src/cli/combo-stats.ts` reports actual JEV decision/token/coverage facts and nullable/incomplete observations without a savings estimate or decision probe. `src/cli/catalog-command-result.ts` owns fixed safe errors and shared catalog evidence only; callers validate their distinct public DTOs. `src/cli/combo-result.ts` rebuilds the combo write receipt and verifies the requested identity. Numeric partial outcomes reach the original root dispatch, and read-back is required before any caller retry. The helpers do not select endpoints, grant authority, perform writes or execute generic CRUD. Existing provider receipt semantics remain separate. Local discovery refusals raised by `runtimeBaseUrl` before any request carry a typed `RuntimeApiError.code` (`proxy_not_running`, `client_role_management_unavailable`) and render a fixed "No request was sent" message instead of uncertain-write wording; integration-route errors (normalized, unredirected path) may append a fixed recovery line keyed on the closed `reason` set; the writer's `message` prose is never echoed. diff --git a/structure/config.md b/structure/config.md index cd8d1821996..ed9285b7ce4 100644 --- a/structure/config.md +++ b/structure/config.md @@ -1,6 +1,6 @@ # Config Surface -`src/config/schema/leaf-validators.ts` accepts only boolean `apiKeys[].allowMeteredComboTargets`; malformed entries are dropped rather than promoted. `src/types/config.ts` and combo validation define optional boolean target `metered`, normalized false when omitted. Explicit consent is required under the [combo runtime contract](runtime.md#request-local-target-compatibility). Quota activation reuses the existing next-reset fields without adding a polling configuration key. See the [quota activation contract](providers/openai-tiers.md#public-provider-contract). `src/types/config.ts` defines `anthropicAccountPool.routes` as ordered model rules. `src/oauth/anthropic-model-routes.ts` validates bounded names, full case-sensitive globs and stored account IDs; `src/config/diagnostics.ts` rejects malformed candidate writes. Load retains a malformed hand edit so unrelated providers survive, while the enabled Anthropic selector refuses it locally before dispatch. Saved rules remain inert when the pool is disabled; affinity is process-local. Request logs use only the rule’s 1-based `route:#` position, never its configured name. `src/cli/hub-disable.ts` removes interdependent hub/loopback/role/hostname settings through `mutatePersistedConfig` and `validateConfigCandidate` in one commit. Top-level deletion intent uses `src/config/rebase-provenance.ts` and authoritative disk provenance in `src/config/live-reconcile.ts` to preserve removed hub bindings through later live saves, `src/config/live-topology.ts` keeps an independent active-topology baseline and merges desired next-start settings into a separate persistence projection, retaining actual socket identity and live hub/client topology during guarded saves and OAuth disk adoption. Client connections and runtimeRole remain one topology unit through connect/disconnect, including fresh disk-only connections. Disk additions/value updates and deletions all survive repeated saves without changing active topology. `src/config/api-key-reconcile.ts` merges key rows by stable ID under the existing save lock; either side's revocation defeats stale edits while disjoint names and rotations merge; unchanged live key membership/order adopts persisted ordering. `tests/config/hub-disable-live-save.test.ts` exercises disable followed by live key creation, rename, rotation, unrelated settings and repeated saves. Saved API keys have no immutable enrollment marker; explicit exact-ID revocation preserves all unselected keys. +`src/config/schema/leaf-validators.ts` accepts only boolean `apiKeys[].allowMeteredComboTargets`; malformed entries are dropped rather than promoted. `src/types/config.ts` and combo validation define optional boolean target `metered`, normalized false when omitted, and optional per-combo `allowMeteredTargets` (default false, literal boolean validation shared by config schema and management API). Explicit consent is required under the [combo runtime contract](runtime.md#request-local-target-compatibility). Quota activation reuses the existing next-reset fields without adding a polling configuration key. See the [quota activation contract](providers/openai-tiers.md#public-provider-contract). `src/types/config.ts` defines `anthropicAccountPool.routes` as ordered model rules. `src/oauth/anthropic-model-routes.ts` validates bounded names, full case-sensitive globs and stored account IDs; `src/config/diagnostics.ts` rejects malformed candidate writes. Load retains a malformed hand edit so unrelated providers survive, while the enabled Anthropic selector refuses it locally before dispatch. Saved rules remain inert when the pool is disabled; affinity is process-local. Request logs use only the rule’s 1-based `route:#` position, never its configured name. `src/cli/hub-disable.ts` removes interdependent hub/loopback/role/hostname settings through `mutatePersistedConfig` and `validateConfigCandidate` in one commit. Top-level deletion intent uses `src/config/rebase-provenance.ts` and authoritative disk provenance in `src/config/live-reconcile.ts` to preserve removed hub bindings through later live saves, `src/config/live-topology.ts` keeps an independent active-topology baseline and merges desired next-start settings into a separate persistence projection, retaining actual socket identity and live hub/client topology during guarded saves and OAuth disk adoption. Client connections and runtimeRole remain one topology unit through connect/disconnect, including fresh disk-only connections. Disk additions/value updates and deletions all survive repeated saves without changing active topology. `src/config/api-key-reconcile.ts` merges key rows by stable ID under the existing save lock; either side's revocation defeats stale edits while disjoint names and rotations merge; unchanged live key membership/order adopts persisted ordering. `tests/config/hub-disable-live-save.test.ts` exercises disable followed by live key creation, rename, rotation, unrelated settings and repeated saves. Saved API keys have no immutable enrollment marker; explicit exact-ID revocation preserves all unselected keys. Native function-result injection follows [the separate opt-in control contract](transports/streaming-health.md#experimental-native-function-result-injection); this surface does not infer upstream support or alter its defaults. diff --git a/structure/gui-and-management-api.md b/structure/gui-and-management-api.md index f2d1d691aae..8508ee3d176 100644 --- a/structure/gui-and-management-api.md +++ b/structure/gui-and-management-api.md @@ -1,6 +1,6 @@ # GUI And Management API -`src/server/management/combo-routes.ts` retains omitted `metered: true` metadata by trimmed provider/model identity during dashboard-shaped saves. Explicit false clears the flag; false defaults are omitted from persisted targets. Key consent follows the [runtime contract](runtime.md#request-local-target-compatibility). Anthropic OAuth account DTOs include `autoSwitchThresholdOverride` (integer or null), +`src/server/management/combo-routes.ts` retains omitted `metered: true` metadata by trimmed provider/model identity during dashboard-shaped saves. Explicit false clears the flag; false defaults are omitted from persisted targets. Omitted combo `allowMeteredTargets` survives saves and renames; explicit false clears it and stays sparse. The CLI exposes `--allow-metered-targets on|off`, and receipts preserve the boolean. Key and loopback operator consent follow the [runtime contract](runtime.md#request-local-target-compatibility). Anthropic OAuth account DTOs include `autoSwitchThresholdOverride` (integer or null), `autoSwitchThreshold` (pool default) and `effectiveAutoSwitchThreshold`. The dedicated `PUT /api/oauth/accounts/auto-switch` accepts `{ provider: "anthropic", accountId, threshold }`; explicit null restores inheritance and missing/invalid values fail. Other providers are rejected. diff --git a/structure/runtime.md b/structure/runtime.md index a8b54f87620..c08df08e4cd 100644 --- a/structure/runtime.md +++ b/structure/runtime.md @@ -488,7 +488,7 @@ Renamed fixed-key providers receive [missing reasoning metadata](catalog.md#rena Translated audio/file admission follows the [final-adapter input contract](adapters/registry.md#untranslated-input-media); native raw passthrough remains separate. ## Request-local target compatibility -`src/server/admission-model-scope.ts` grants metered combo eligibility only to a matching configured key with literal `allowMeteredComboTargets: true`. `src/server/responses/core-combo.ts` filters marked targets on initial, failure and recovery selection for every strategy. A route the router collapsed to one combo pick outside that loop (a subagent fallback naming a combo alias) is refused in `captureInboundRoutePolicy` of `src/server/responses/request-prepare.ts` when the pick is metered and consent is absent. Missing identity, environment-token and loopback admission deny marked targets; provider/model scopes remain independent. `src/cli/combo-input.ts` accepts and validates the optional target boolean. Unmarked targets retain existing behavior; metering is operator metadata, not automatic billing detection. Coverage: `tests/server/combo-metered-consent.test.ts` and `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. +`src/server/admission-model-scope.ts` grants metered combo eligibility to a matching configured key with literal `allowMeteredComboTargets: true`, or to local/missing admission when the selected combo has literal `allowMeteredTargets: true` and the full business config satisfies `!isApiAuthRequired(config)`. Configured and environment admissions never inherit combo consent; linked keys retain their key consent requirement even on a loopback Home. The full hostname also prevents secondary loopback listeners on public proxies from granting operator consent. `src/server/responses/core-combo.ts` filters marked targets on initial, failure and recovery selection for every strategy. A route the router collapsed to one combo pick outside that loop (a subagent fallback naming a combo alias) is re-picked among usable unmetered targets in `captureInboundRoutePolicy` of `src/server/responses/request-prepare.ts` when its pick is metered without consent, or refused if none remain; the replacement retains combo metadata and passes destination policy/scope checks. Request headers never grant consent; absent/false consent denies marked targets; provider/model scopes remain independent. `src/cli/combo-input.ts` accepts and validates the optional target boolean. Unmarked targets retain existing behavior; metering is operator metadata, not automatic billing detection. Coverage: `tests/server/combo-metered-consent.test.ts`, `tests/server/combo-loopback-metered-consent.test.ts` and `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. The exact Anthropic credit-balance refusal on HTTP 400/402 is a provider-state failure under the [Anthropic API-key billing contract](providers/anthropic-key-billing.md). `src/combos/failover.ts` recognizes complete bounded JSON or exact formatted prose with an accepted structured code before the generic invalid-request stop, granting a provider-scoped hop; malformed, truncated and unrelated 400s keep their existing decision. Google's final adapter compiler may emit an opt-in, content-free [tool-schema loss diagnostic](providers/google.md#google-tool-schema-loss-reporting). It observes adapter-local narrowing only and changes neither provider routing nor the serialized request body. `src/adapters/openai-responses.ts` omits only top-level `user` at the canonical ChatGPT Codex forward destination. Claude translation retains its original identity and prompt-cache key; public API and noncanonical gateways retain their `user` field. Input roles, tool-schema properties, safety identifiers and original replay bodies are not changed. diff --git a/tests/adapters/anthropic/anthropic-subscription-chain.test.ts b/tests/adapters/anthropic/anthropic-subscription-chain.test.ts index 0d02f8c9fc1..6bfc7cf274c 100644 --- a/tests/adapters/anthropic/anthropic-subscription-chain.test.ts +++ b/tests/adapters/anthropic/anthropic-subscription-chain.test.ts @@ -2,7 +2,7 @@ * One model id, three billing paths: an Anthropic OAuth subscription pool, Cursor's copy of the * same model, then a pay-per-use Anthropic API key. A failover combo whose alias is the model id * the clients already send gives that order; `metered` keeps issued keys off the API-key target, - * and every caller needs explicit configured-key consent to reach a metered target. + * and callers need explicit key consent or loopback-only operator consent to reach a metered target. */ import { afterEach, beforeEach, expect, mock, test } from "bun:test"; import { mkdtempSync } from "node:fs"; @@ -289,3 +289,27 @@ test("streaming combo preflight preserves Cursor's structured rate-limit status" expect(response.status).toBe(429); expect(JSON.parse(response.text).error.code).toBe("rate_limit_exceeded"); }); + +for (const stream of [false, true]) { + test(`loopback operator opt-in reaches the API key after subscription exhaustion (stream=${stream})`, async () => { + subscriptionSpent = true; + cursorLimited = true; + const cfg = config(); + cfg.combos!["claude-chain"]!.allowMeteredTargets = true; + const response = await chat(cfg, LOOPBACK, stream); + expect(response.status).toBe(200); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(1); + }); +} + +test("public listener plus combo opt-in still refuses the metered fallback for local and unconsented keyed callers", async () => { + subscriptionSpent = true; + cursorLimited = true; + const cfg = config(); + cfg.hostname = "0.0.0.0"; + cfg.combos!["claude-chain"]!.allowMeteredTargets = true; + for (const admission of [LOOPBACK, ISSUED]) { + expect((await chat(cfg, admission)).status).toBe(429); + expect(sent.filter(entry => entry.lane === "api-key")).toHaveLength(0); + } +}); diff --git a/tests/cli/cli-combo-partial-update.test.ts b/tests/cli/cli-combo-partial-update.test.ts index 396f1938a46..4a539172b26 100644 --- a/tests/cli/cli-combo-partial-update.test.ts +++ b/tests/cli/cli-combo-partial-update.test.ts @@ -167,3 +167,20 @@ describe("combo decision probe and discovery verbs", () => { expect(r.requests[0]!.url).toContain("/api/combos/decision-discovery?q=jev"); }); }); + +test("loopback metered consent is explicit on/off, survives partial edits and rejects malformed input before transport", async () => { + quiet(); + for (const [arg, value] of [["on", true], ["off", false]] as const) { + const r = runtime({ ...original, allowMeteredTargets: true }); + expect(await handleComboCommand(["set", "auto", "--allow-metered-targets", arg, "--json"], r.deps)).toBe(0); + expect(r.body()?.combo.allowMeteredTargets).toBe(value); + } + const unchanged = runtime({ ...original, allowMeteredTargets: true }); + expect(await handleComboCommand(["set", "auto", "--effort", "high"], unchanged.deps)).toBe(0); + expect(unchanged.body()?.combo.allowMeteredTargets).toBe(true); + for (const args of [["--allow-metered-targets"], ["--allow-metered-targets", "true"], ["--allow-metered-targets", "on", "--allow-metered-targets", "off"]]) { + const r = runtime(); + expect(await handleComboCommand(["set", "auto", ...args], r.deps)).toBe(2); + expect(r.requests).toEqual([]); + } +}); diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index 0082a1fec4b..808ba28053a 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -26,7 +26,7 @@ "anthropic-instance-isolation.test.ts": "adapters/anthropic", "anthropic-instance-pool-parity.test.ts": "adapters/anthropic", "anthropic-instance-quota.test.ts": "adapters/anthropic", "anthropic-instance-recovery.test.ts": "adapters/anthropic", "anthropic2-fast-parity.test.ts": "adapters/anthropic", "anthropic2-native-routing.test.ts": "claude-integration", - "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", + "oauth-anthropic-instance-health.test.ts": "oauth", "anthropic-quota-cooldown-recovery.test.ts": "adapters/anthropic", "anthropic-subscription-chain.test.ts": "adapters/anthropic", "combo-metered-consent.test.ts": "server", "combo-loopback-metered-consent.test.ts": "server", "anthropic-pool-usage-limit.test.ts": "adapters/anthropic", "oauth-account-plans.test.ts": "oauth", "anthropic2-responses-parity.test.ts": "responses", "oauth-anthropic-instance-multiprocess.test.ts": "oauth", "anthropic2-discovery-ownership.test.ts": "oauth", "codex-shim-runtime-preflight.test.ts": "codex-integration", diff --git a/tests/server/combo-loopback-metered-consent.test.ts b/tests/server/combo-loopback-metered-consent.test.ts new file mode 100644 index 00000000000..e2dd0b3ea31 --- /dev/null +++ b/tests/server/combo-loopback-metered-consent.test.ts @@ -0,0 +1,150 @@ +import { afterEach, beforeEach, expect, test } from "bun:test"; +import type { OcxConfig, OcxComboStrategy } from "../../src/types"; +import { handleResponses } from "../../src/server/responses"; +import { handleChatCompletions } from "../../src/server/chat-completions"; +import { handleClaudeMessages } from "../../src/server/claude-messages"; +import { handleResponsesCompact } from "../../src/server/responses/compact"; +import { admissionMayUseMeteredComboTargets } from "../../src/server/admission-model-scope"; +import { resolveApiAuth, type DataPlaneAdmission } from "../../src/server/auth-cors"; +import { clearComboTargetCooldowns, coolComboTarget } from "../../src/combos/failover"; +import { clearComboSelectionState } from "../../src/combos/resolve"; +import { noteSubagentModelFailure, resetSubagentModelFallbackStateForTests } from "../../src/codex/subagent-model-fallback"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { createWebsocketHandler } from "../../src/server/index/websocket-handler"; +import { tryReserveCodexWebSocket } from "../../src/codex/websocket-registry"; +import { buildResponsesWsData, type WsData } from "../../src/server/ws-bridge"; + +const issued = { id: "issued", name: "issued", key: "ocx_data_" + "a".repeat(40), createdAt: "2026-10-15T00:00:00Z" }; +const keyed: DataPlaneAdmission = { kind: "configured", keyId: issued.id, source: "bearer" }; +const local: DataPlaneAdmission = { kind: "loopback", source: "loopback" }; +const originalRandom = Math.random; +let sent: string[]; +let release: () => void; +beforeEach(() => { sent = []; release = acquireOwnedSpendHome(); clearComboSelectionState(); clearComboTargetCooldowns(); }); +afterEach(() => { Math.random = originalRandom; release(); clearComboSelectionState(); clearComboTargetCooldowns(); resetSubagentModelFallbackStateForTests(); }); +function config(flag?: boolean, strategy: OcxComboStrategy = "failover"): OcxConfig { + const provider = (name: string) => ({ adapter: "openai-chat" as const, baseUrl: `https://${name}.test/v1`, apiKey: "fixture", models: ["m"], + fetch: (async () => { sent.push(name); return Response.json({ error: { message: "fixture refusal", type: "invalid_request_error" } }, { status: 400 }); }) as typeof fetch }); + return { port: 16389, defaultProvider: "paid", providers: { paid: provider("paid"), subscription: provider("subscription"), primary: provider("primary") }, + apiKeys: [issued], combos: { chain: { strategy, ...(flag === undefined ? {} : { allowMeteredTargets: flag }), targets: [{ provider: "paid", model: "m", metered: true }] } } }; +} +const spoof = { "x-ocx-allow-metered-targets": "true", "x-opencodex-allow-metered-combo-targets": "true", "allowMeteredTargets": "true" }; +function request(wire: string, model = "combo/chain", headers: Record = {}) { + const body = wire === "chat" || wire === "messages" + ? { model, stream: false, max_tokens: 16, messages: [{ role: "user", content: "hi" }] } + : { model, input: "hi", stream: false }; + return new Request(`http://localhost:16389/v1/${wire}`, { method: "POST", headers: { "content-type": "application/json", ...headers }, body: JSON.stringify(body) }); +} +async function post(wire: string, cfg: OcxConfig, admission: DataPlaneAdmission, headers = {}) { + const req = request(wire, "combo/chain", headers); + const ctx = { model: "", provider: "" }; + const ids = { requestId: "consent-test", start: Date.now(), admission }; + const response = wire === "chat" ? await handleChatCompletions(req, cfg, ctx, ids) + : wire === "messages" ? await handleClaudeMessages(req, cfg, ctx, ids) + : wire === "compact" ? await handleResponsesCompact(req, cfg, ctx, undefined, admission) + : await handleResponses(req, cfg, ctx, { admission }); + await response.text(); + return response.status; +} +for (const wire of ["responses", "chat", "messages", "compact"]) { + test(`${wire}: local operator opt-in permits metered targets, default and spoofed consent deny`, async () => { + expect(await post(wire, config(), local, spoof)).toBe(wire === "messages" ? 529 : 503); + expect(sent).toEqual([]); + expect(await post(wire, config(false), local)).toBe(wire === "messages" ? 529 : 503); + expect(sent).toEqual([]); + await post(wire, config(true), local); + expect(sent).toEqual(["paid"]); + }); + test(`${wire}: public listener ignores combo opt-in, including secondary loopback; opted-in key still works`, async () => { + const cfg = { ...config(true), hostname: "0.0.0.0" }; + for (const admission of [local, keyed, { kind: "environment", source: "bearer" } as const]) { + expect(await post(wire, cfg, admission, spoof)).toBe(wire === "messages" ? 529 : 503); + expect(sent).toEqual([]); + } + cfg.apiKeys![0] = { ...issued, allowMeteredComboTargets: true }; + await post(wire, cfg, keyed); + expect(sent).toEqual(["paid"]); + }); +} +test("consent uses the auth listener predicate and never overrides a configured or environment admission", () => { + for (const hostname of [undefined, "127.0.0.1", "localhost.", "::1", "[::1]", "0.0.0.0", "::", "192.0.2.1"]) { + const cfg = { ...config(true), hostname }; + const auth = resolveApiAuth(request("responses"), cfg); + expect(admissionMayUseMeteredComboTargets(cfg, local, cfg.combos!.chain)).toBe(auth?.kind === "loopback"); + expect(admissionMayUseMeteredComboTargets(cfg, undefined, cfg.combos!.chain)).toBe(auth?.kind === "loopback"); + expect(admissionMayUseMeteredComboTargets(cfg, keyed, cfg.combos!.chain)).toBe(false); + expect(admissionMayUseMeteredComboTargets(cfg, { kind: "environment", source: "bearer" }, cfg.combos!.chain)).toBe(false); + cfg.apiKeys![0] = { ...issued, allowMeteredComboTargets: true }; + expect(admissionMayUseMeteredComboTargets(cfg, keyed, cfg.combos!.chain)).toBe(true); + expect(admissionMayUseMeteredComboTargets(cfg, { ...keyed, keyId: "missing" }, cfg.combos!.chain)).toBe(false); + } +}); +for (const strategy of ["failover", "round-robin", "random", "least-used", "reset-window", "jev"] as const) { + test(`${strategy}: the loop's first pick honors local consent`, async () => { + await post("responses", config(true, strategy), local); + expect(sent).toEqual(["paid"]); + }); +} +for (const strategy of ["failover", "round-robin", "random"] as const) { + test(`${strategy}: collapsed subagent fallback honors operator consent and prefers usable unmetered targets`, async () => { + const cfg = config(true, strategy); + cfg.defaultProvider = "primary"; + cfg.combos!.chain!.alias = "chain-model"; + cfg.subagentModelFallback = ["chain-model"]; + const postFallback = (admission = local) => { + resetSubagentModelFallbackStateForTests(); + noteSubagentModelFailure("primary/m", "429", cfg); + return handleResponses(request("responses", "primary/m", { "x-openai-subagent": "collab_spawn" }), cfg, { model: "", provider: "" }, { admission }); + }; + await (await postFallback()).text(); + expect(sent).toEqual(["paid"]); + sent.length = 0; + cfg.combos!.chain!.allowMeteredTargets = false; + expect((await postFallback()).status).toBe(503); + expect(sent).toEqual([]); + // Force the paid first choice in random too; it must be re-picked rather than refused. + if (strategy === "random") Math.random = () => 0; + cfg.combos!.chain!.targets = [{ provider: "paid", model: "m", metered: true, weight: 10000 }, { provider: "subscription", model: "m", weight: 1 }]; + for (let i = 0; i < 8; i++) { + clearComboSelectionState(); + await (await postFallback()).text(); + } + expect(sent).toEqual(Array(8).fill("subscription")); + sent.length = 0; + cfg.apiKeys![0] = { ...issued, allowedProviders: ["paid"] }; + expect((await postFallback(keyed)).status).toBe(403); + expect(sent).toEqual([]); + coolComboTarget("chain", { provider: "subscription", model: "m" }, { cooldownMs: 600000 }); + expect((await postFallback()).status).toBe(503); + }); +} + +async function websocket(cfg: OcxConfig, admission: DataPlaneAdmission) { + const handler = createWebsocketHandler({ config: cfg, deps: {} } as Parameters[0]); + const server = Bun.serve({ hostname: "127.0.0.1", port: 0, websocket: handler, + fetch(req, server) { return server.upgrade(req, { data: buildResponsesWsData(new Headers(spoof), admission, tryReserveCodexWebSocket() ?? undefined) }) ? undefined : new Response("upgrade failed", { status: 400 }); }, + }); + const ws = new WebSocket(String(server.url).replace("http:", "ws:")); + try { + await new Promise((resolve, reject) => { + const timer = setTimeout(() => reject(new Error("websocket consent deadline")), 4000); + ws.onopen = () => ws.send(JSON.stringify({ type: "response.create", model: "combo/chain", input: "hi" })); + ws.onerror = () => { clearTimeout(timer); reject(new Error("websocket failed")); }; + ws.onmessage = () => { clearTimeout(timer); resolve(); }; + }); + } finally { ws.close(); await server.stop(true); } +} +test("WebSocket frames retain local or keyed consent and reject spoofed consent", async () => { + await websocket(config(), local); + expect(sent).toEqual([]); + await websocket(config(true), local); + expect(sent).toEqual(["paid"]); + sent.length = 0; + const cfg = { ...config(true), hostname: "0.0.0.0" }; + await websocket(cfg, local); + await websocket(cfg, keyed); + expect(sent).toEqual([]); + cfg.apiKeys![0] = { ...issued, allowMeteredComboTargets: true }; + await websocket(cfg, keyed); + expect(sent).toEqual(["paid"]); +}); diff --git a/tests/server/combo-metered-consent.test.ts b/tests/server/combo-metered-consent.test.ts index d98db2709ce..afa54cc8633 100644 --- a/tests/server/combo-metered-consent.test.ts +++ b/tests/server/combo-metered-consent.test.ts @@ -51,10 +51,10 @@ test("metered metadata survives dashboard saves, explicit false clears it, malfo }; saveConfig(config); expect(loadConfig().apiKeys?.[0]?.allowMeteredComboTargets).toBe(true); - const put = async (target: unknown) => { + const put = async (target: unknown, extra: Record = {}, id = "chain", renameFrom?: string) => { const req = new ManagementRequest("http://localhost:16389/api/combos", { method: "PUT", headers: { "content-type": "application/json" }, - body: JSON.stringify({ id: "chain", combo: { strategy: "failover", targets: [target] } }), + body: JSON.stringify({ id, ...(renameFrom ? { renameFrom } : {}), combo: { strategy: "failover", targets: [target], ...extra } }), }); return handleManagementAPI(req, new URL(req.url), config, { createManagementConvergeCodex: catalogConvergenceFactory(async () => {}), @@ -69,6 +69,22 @@ test("metered metadata survives dashboard saves, explicit false clears it, malfo expect(disk().combos?.chain?.targets[0]?.metered).toBe(true); expect((await put({ provider: "a", model: "m", metered: false }))?.status).toBe(200); expect(disk().combos?.chain?.targets[0]).not.toHaveProperty("metered"); + const target = { provider: "a", model: "m", metered: true }; + expect((await put(target, { allowMeteredTargets: true }))?.status).toBe(200); + expect(loadConfig().combos?.chain?.allowMeteredTargets).toBe(true); + expect((await put(target))?.status).toBe(200); + expect(disk().combos?.chain?.allowMeteredTargets).toBe(true); + for (const invalid of ["true", 1, null]) { + expect((await put(target, { allowMeteredTargets: invalid }))?.status).toBe(400); + expect(disk().combos?.chain?.allowMeteredTargets).toBe(true); + writeFileSync(join(home, "config.json"), JSON.stringify({ ...config, combos: { chain: { targets: [target], allowMeteredTargets: invalid } } })); + expect(loadConfig().combos?.chain).toBeUndefined(); + saveConfig(config); + } + expect((await put(target, {}, "renamed", "chain"))?.status).toBe(200); + expect(disk().combos?.renamed?.allowMeteredTargets).toBe(true); + expect((await put(target, { allowMeteredTargets: false }, "renamed"))?.status).toBe(200); + expect(disk().combos?.renamed).not.toHaveProperty("allowMeteredTargets"); writeFileSync(join(home, "config.json"), JSON.stringify({ ...config, apiKeys: [{ ...key, allowMeteredComboTargets: "true" }] })); expect(loadConfig().apiKeys ?? []).toHaveLength(0); } finally { From 78b9fa30415240d007c31f9ebd7c4d25031618f1 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 21:33:05 +0900 Subject: [PATCH 5/9] fix(combos): keep runTurn child verdicts through combo failover A runTurn combo child already classifies its pre-output failure with adapterFailureFromEvent, but formatErrorResponse re-derived the type and code from the status. insufficient_quota became rate_limit_exceeded, losing its provider-wide cooldown and the client's quota verdict, and a 400 unsupported_input_modality became invalid_request_error, which now stopped the combo where dev's flat 502 had hopped to the next target. formatErrorResponse gains a classified option that keeps the caller's type and code while its spend-ledger, cyber-policy and replay-refusal overrides still win. Only the two combo-child blocks use it; consumeComboFailure re-applies the client allowlist before anything reaches a caller. Combo failover treats unsupported_input_modality as a target-local hop that cools nothing. Docs: an earlier 429/402 quota refusal outranks a later 400/401/403 when a metered-consent combo exhausts, and the modality hop is listed in every locale's combo guide. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- .../src/content/docs/fr/guides/combos.md | 4 +- docs-site/src/content/docs/guides/combos.md | 6 +- .../src/content/docs/ja/guides/combos.md | 4 +- .../src/content/docs/ko/guides/combos.md | 4 +- .../src/content/docs/ru/guides/combos.md | 4 +- .../src/content/docs/tr/guides/combos.md | 4 +- .../src/content/docs/zh-cn/guides/combos.md | 4 +- .../src/content/docs/zh-tw/guides/combos.md | 4 +- scripts/test-layout/layout.json | 1 + src/bridge/errors.ts | 11 +- src/combos/failover.ts | 7 +- src/server/responses/run-turn-execution.ts | 4 +- structure/transports/responses-failover.md | 2 +- tests/fixtures/test-layout-expected.json | 1 + ...-combo-run-turn-classified-failure.test.ts | 152 ++++++++++++++++++ ...uter-combo-failover-classification.test.ts | 3 +- 16 files changed, 192 insertions(+), 23 deletions(-) create mode 100644 tests/responses/responses-combo-run-turn-classified-failure.test.ts diff --git a/docs-site/src/content/docs/fr/guides/combos.md b/docs-site/src/content/docs/fr/guides/combos.md index 4cb051fc727..c35725b8c34 100644 --- a/docs-site/src/content/docs/fr/guides/combos.md +++ b/docs-site/src/content/docs/fr/guides/combos.md @@ -201,7 +201,7 @@ Les échecs d’un combo se répartissent entre ceux qui entraînent un **bascul | HTTP 401, 403, 404, 408, 429, ou n'importe quel 5xx | Refroidissez la cible et passez à la prochaine cible éligible. | | Erreur classée comme erreur d’authentification, d’abonnement, de quota, de limitation de débit, de surcharge ou de serveur en amont | Place la cible en période de refroidissement et bascule, même si le statut seul ne suffit pas. | | Annulation client (499), `origin_rejected`, refus de cyber-politique, débordement de contexte ou autre demande invalide | Arrêtez et renvoyez l'erreur ; une autre cible ne rendrait pas la demande valide. | -| Rejet structuré de `user`, valeur non prise en charge pour `reasoning.effort`/`reasoning_effort`, ou rejet d'entrée d'image propre à un modèle (`param: input`) | Bascule vers la cible admissible suivante avant le début de la sortie, sans délai de refroidissement ; voir Compatibilité des paramètres facultatifs ci-dessous. | +| Rejet structuré de `user`, valeur non prise en charge pour `reasoning.effort`/`reasoning_effort`, rejet d'entrée d'image propre à un modèle (`param: input`), ou refus d'un média d'entrée que la route ne peut pas lire (`unsupported_input_modality`) | Bascule vers la cible admissible suivante avant le début de la sortie, sans délai de refroidissement ; voir Compatibilité des paramètres facultatifs ci-dessous. | | Premier appel d'outil d'un tour Responses exécuté par un adaptateur interne (`runTurn`) que la requête courante n'a pas déclaré, avant toute sortie et tout effet de bord non rejouable | Met la cible en refroidissement et bascule avec le même catalogue d'outils. Après une sortie visible ou un effet de bord non rejouable, le refus est définitif. Les requêtes Chat Completions et Anthropic Messages ne changent pas. | | Toute autre erreur non classifiée | Arrêtez et renvoyez l'erreur. | @@ -370,6 +370,6 @@ gérer un refus de politique ou corriger l’origine de la demande rejetée. Les ## Compatibilité des paramètres facultatifs -Exception aux erreurs 400 terminales : un rejet structuré de `user`, une valeur non prise en charge pour `reasoning.effort`/`reasoning_effort`, ou un rejet d’entrée d’image propre à un modèle (`param: input`) peut faire passer le combo à la cible admissible suivante avant le début de la sortie, sans délai de refroidissement. Le refus de sécurité, l’annulation et une sortie déjà commencée restent non rejouables. +Exception aux erreurs 400 terminales : un rejet structuré de `user`, une valeur non prise en charge pour `reasoning.effort`/`reasoning_effort`, un rejet d’entrée d’image propre à un modèle (`param: input`), ou le refus d’un média d’entrée que la route ne peut pas lire (`unsupported_input_modality`) peut faire passer le combo à la cible admissible suivante avant le début de la sortie, sans délai de refroidissement. Le refus de sécurité, l’annulation et une sortie déjà commencée restent non rejouables. [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/guides/combos.md b/docs-site/src/content/docs/guides/combos.md index a0d52d0182e..3b419ea9e6a 100644 --- a/docs-site/src/content/docs/guides/combos.md +++ b/docs-site/src/content/docs/guides/combos.md @@ -24,7 +24,7 @@ Mark a pay-per-use combo target with `"metered": true` when it follows subscript Request headers cannot grant consent. Model/provider scopes still apply. This covers initial selection, failover and recovery across combo strategies, compaction, Messages, Chat Completions and WebSocket requests, and a subagent model fallback naming a combo alias. A collapsed fallback tries a usable unmetered target before refusing. Unmarked targets retain existing behavior; metering is declared by the operator rather than detected automatically. -For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without consent, exhaustion returns the last unmetered error. Dashboard saves retain omitted target metering and combo consent; explicitly setting either to false clears it. These consent fields are configured through the config file or API; the combo flag is also available through the CLI, while the dashboard does not expose it. +For example, a failover chain can target Anthropic OAuth, then Cursor, then `{ "provider": "anthropic-apikey", "model": "claude-opus-5-5", "metered": true }`. Without consent, exhaustion returns the last unmetered error, with one exception: an earlier 429 or 402 quota refusal takes precedence over a later 400, 401 or 403. If the subscription target returns 429 and Cursor then refuses with 401 or 403, the caller receives the subscription's 429, because the spent subscription explains the failure and Cursor's credential refusal does not. Dashboard saves retain omitted target metering and combo consent; explicitly setting either to false clears it. These consent fields are configured through the config file or API; the combo flag is also available through the CLI, while the dashboard does not expose it. ## 60-second quickstart @@ -551,7 +551,7 @@ Combo failures are divided into **hop** failures and **terminal** failures. | Classified authentication, subscription, quota, rate-limit, overload, or upstream-server error | Cool the target and hop, even when the status alone is not sufficient. | | Client cancellation (499), `origin_rejected`, cyber-policy refusal, context overflow, or other invalid request | Stop and return the error; another target would not make the request valid. | | Exact HTTP 400 saying a model is not supported when using Codex with a ChatGPT account | Try the next declared target before output commitment. The message must be the whole refusal in `detail`, `error.message`, or bare text; competing `detail` and `error` envelopes stay terminal. This does not quarantine the account or cool every model on the provider. | -| Structured HTTP 400 rejecting optional `user`, an unsupported reasoning effort, or model-scoped image input | Hop before output commitment without cooling; see request-local target compatibility below. | +| Structured HTTP 400 rejecting optional `user`, an unsupported reasoning effort, model-scoped image input, or input media the route cannot read (`unsupported_input_modality`) | Hop before output commitment without cooling; see request-local target compatibility below. | | First tool call of a Responses turn run by an in-process adapter (`runTurn`) that the current request did not declare, before any output or replay-unsafe side effect | Cool the target and hop with the same tool catalog. After visible output or a replay-unsafe side effect the refusal is final. Chat Completions and Anthropic Messages requests are unchanged. | | Any other unclassified error | Stop and return the error. | @@ -661,7 +661,7 @@ instead of growing memory without a bound. When routing Claude Code to the canonical ChatGPT Codex backend, OpenCodex removes the unsupported top-level `user` metadata field without changing the session/cache key, input messages, tool schemas, or safety identifiers. Public Responses API and noncanonical forward gateways keep that field. -A combo can also advance after an intact HTTP 400 `invalid_request_error` that specifically rejects `user`, reports `unsupported_value` for `reasoning.effort`/`reasoning_effort`, or reports `param: input` with an exact model-scoped `does not support image inputs` rejection. This is a mismatch for that request, not evidence that the target is unhealthy, so it records no cooldown. This compatibility recovery does not silently change `none` into a different effort or broaden this exception to arbitrary invalid requests. Policy refusals, cancellation and already-committed output remain non-replayable. A single-target request still returns an unresolved upstream rejection. +A combo can also advance after an intact HTTP 400 `invalid_request_error` that specifically rejects `user`, reports `unsupported_value` for `reasoning.effort`/`reasoning_effort`, or reports `param: input` with an exact model-scoped `does not support image inputs` rejection. A route that refuses the request's media with code `unsupported_input_modality`, such as image input sent to a CLI provider route without a verified multimodal contract, advances the same way. This is a mismatch for that request, not evidence that the target is unhealthy, so it records no cooldown. This compatibility recovery does not silently change `none` into a different effort or broaden this exception to arbitrary invalid requests. Policy refusals, cancellation and already-committed output remain non-replayable. A single-target request still returns an unresolved upstream rejection. ## Default reasoning effort diff --git a/docs-site/src/content/docs/ja/guides/combos.md b/docs-site/src/content/docs/ja/guides/combos.md index 1a03b3e6a07..33f2b628fd0 100644 --- a/docs-site/src/content/docs/ja/guides/combos.md +++ b/docs-site/src/content/docs/ja/guides/combos.md @@ -126,7 +126,7 @@ ocx combo set balanced \ |機密認証、サブスクリプション、クォータ、レート制限、過負荷、またはアップストリーム サーバー エラー |ステータスだけでは物足りない場合でもターゲットを冷やしてホップさせましょう。 | |クライアントのキャンセル (499)、`origin_rejected`、サイバー ポリシーの拒否、コンテキスト オーバーフロー、またはその他の無効なリクエスト |停止してエラーを返します。別のターゲットではリクエストは有効になりません。 | | ChatGPT アカウントで Codex を使う際に、そのモデルをサポートしないことを示す正確な HTTP 400 の拒否 | 出力を確定する前に、次の設定済みターゲットを試します。拒否文全体が `detail`、`error.message`、または本文そのものにある場合に限ります。`detail` と `error` が競合する応答は終了扱いです。この拒否だけでアカウントを再認証待ちにしたり、プロバイダーの全モデルをクールダウンしたりしません。 | -|`user` の明示的な拒否、`reasoning.effort`/`reasoning_effort` の非対応値、またはモデル固有の画像入力拒否（`param: input`）を示す構造化 HTTP 400 |出力開始前に次の適格なターゲットへ進み、クールダウンを記録しません。任意パラメーターの互換性を参照してください。 | +|`user` の明示的な拒否、`reasoning.effort`/`reasoning_effort` の非対応値、モデル固有の画像入力拒否（`param: input`）、またはルートが読み取れない入力メディアの拒否（`unsupported_input_modality`）を示す構造化 HTTP 400 |出力開始前に次の適格なターゲットへ進み、クールダウンを記録しません。任意パラメーターの互換性を参照してください。 | |インプロセスアダプター（`runTurn`）が実行する Responses ターンで、現在のリクエストが宣言していない最初のツール呼び出し（出力やリプレイ不可の副作用より前） |ターゲットをクールダウンし、同じツールカタログで次へ進みます。可視出力やリプレイ不可の副作用の後は拒否が確定します。Chat Completions と Anthropic Messages のリクエストは変わりません。 | |その他の未分類のエラー |停止してエラーを返します。 | @@ -269,6 +269,6 @@ ocx combo remove --yes ## 任意パラメーターの互換性 -通常の 400 エラーは終了扱いですが、`user` の明示的な拒否、`reasoning.effort`/`reasoning_effort` の非対応値、またはモデル固有の画像入力拒否（`param: input`）を示す構造化エラーでは、出力開始前に次の適格なターゲットへ進めます。この不一致ではクールダウンを記録しません。安全ポリシーによる拒否、キャンセル、出力開始後の再実行は禁止のままです。 +通常の 400 エラーは終了扱いですが、`user` の明示的な拒否、`reasoning.effort`/`reasoning_effort` の非対応値、モデル固有の画像入力拒否（`param: input`）、またはルートが読み取れない入力メディアの拒否（`unsupported_input_modality`）を示す構造化エラーでは、出力開始前に次の適格なターゲットへ進めます。この不一致ではクールダウンを記録しません。安全ポリシーによる拒否、キャンセル、出力開始後の再実行は禁止のままです。 [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/ko/guides/combos.md b/docs-site/src/content/docs/ko/guides/combos.md index c9ccf3757b3..92be44fa008 100644 --- a/docs-site/src/content/docs/ko/guides/combos.md +++ b/docs-site/src/content/docs/ko/guides/combos.md @@ -132,7 +132,7 @@ ocx combo set balanced \ | 인증, 구독, 쿼터, 속도 제한, 과부하, 또는 상위 서버 오류로 분류됨 | 상태 코드만으로는 충분하지 않더라도 대상을 쿨다운으로 보내고 넘어갑니다. | | 클라이언트 취소(499), `origin_rejected`, cyber-policy refusal, context overflow, 또는 기타 invalid request | 멈추고 오류를 반환합니다. 다른 대상을 써도 요청이 유효해지지 않기 때문입니다. | | ChatGPT 계정으로 Codex를 사용할 때 해당 모델을 지원하지 않는다는 정확한 HTTP 400 거부 | 출력이 시작되기 전에 다음 설정 대상으로 넘어갑니다. 거부 문장 전체가 `detail`, `error.message` 또는 일반 텍스트에 있어야 하며, `detail`과 `error`가 충돌하면 중단합니다. 이 사유만으로 계정을 재인증 상태로 바꾸거나 공급자의 모든 모델에 쿨다운을 걸지는 않습니다. | -| `user`를 명시적으로 거부하거나, `reasoning.effort`/`reasoning_effort`의 지원되지 않는 값 또는 모델별 이미지 입력 거부(`param: input`)를 나타내는 구조화된 HTTP 400 | 출력 시작 전에 쿨다운 기록 없이 다음 적격 대상으로 넘어갑니다. 선택적 매개변수 호환성을 참조하세요. | +| `user`를 명시적으로 거부하거나, `reasoning.effort`/`reasoning_effort`의 지원되지 않는 값, 모델별 이미지 입력 거부(`param: input`) 또는 경로가 읽을 수 없는 입력 미디어 거부(`unsupported_input_modality`)를 나타내는 구조화된 HTTP 400 | 출력 시작 전에 쿨다운 기록 없이 다음 적격 대상으로 넘어갑니다. 선택적 매개변수 호환성을 참조하세요. | | 인프로세스 어댑터(`runTurn`)가 실행하는 Responses 턴에서 현재 요청이 선언하지 않은 첫 도구 호출(출력이나 재전송 불가 부작용 이전) | 대상을 쿨다운하고 같은 도구 카탈로그로 다음 대상으로 넘어갑니다. 출력이 보였거나 재전송 불가 부작용이 생긴 뒤에는 거부가 그대로 확정됩니다. Chat Completions와 Anthropic Messages 요청은 바뀌지 않습니다. | | 그 밖의 분류되지 않은 오류 | 멈추고 오류를 반환합니다. | @@ -285,6 +285,6 @@ opencodex 인스턴스에 기록했는지 확인하세요. ## 선택적 매개변수 호환성 -일반적인 400 오류는 종료되지만, `user`를 명시적으로 거부하거나 `reasoning.effort`/`reasoning_effort`의 지원되지 않는 값 또는 모델별 이미지 입력 거부(`param: input`)를 나타내는 구조화된 오류는 출력 시작 전에 다음 적격 대상으로 넘어갈 수 있습니다. 이 경우 쿨다운을 기록하지 않습니다. 보안 정책 거부, 취소 및 이미 시작된 출력은 재실행하지 않습니다. +일반적인 400 오류는 종료되지만, `user`를 명시적으로 거부하거나 `reasoning.effort`/`reasoning_effort`의 지원되지 않는 값, 모델별 이미지 입력 거부(`param: input`) 또는 경로가 읽을 수 없는 입력 미디어 거부(`unsupported_input_modality`)를 나타내는 구조화된 오류는 출력 시작 전에 다음 적격 대상으로 넘어갈 수 있습니다. 이 경우 쿨다운을 기록하지 않습니다. 보안 정책 거부, 취소 및 이미 시작된 출력은 재실행하지 않습니다. [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/ru/guides/combos.md b/docs-site/src/content/docs/ru/guides/combos.md index fe966d0cbf9..b6f139674c7 100644 --- a/docs-site/src/content/docs/ru/guides/combos.md +++ b/docs-site/src/content/docs/ru/guides/combos.md @@ -163,7 +163,7 @@ ocx combo set balanced \ | Классифицированная ошибка аутентификации, подписки, квоты, rate-limit, перегрузки или upstream-server | Перевести цель в cooldown и переключиться, даже если одного статуса недостаточно. | | Отмена клиентом (499), `origin_rejected`, отказ из-за cyber-policy, переполнение контекста или иной некорректный запрос | Остановиться и вернуть ошибку; другая цель не сделает такой запрос корректным. | | Точный отказ HTTP 400 о том, что модель не поддерживается в Codex с аккаунтом ChatGPT | До начала вывода пробует следующую настроенную цель. Полная фраза отказа должна находиться в `detail`, `error.message` или обычном тексте; смешанные объекты `detail` и `error` завершают запрос. Сам этот отказ не переводит аккаунт в состояние повторного входа и не включает период ожидания для всех моделей провайдера. | -| Структурированный HTTP 400, отклоняющий необязательный `user`, неподдерживаемое значение `reasoning.effort`/`reasoning_effort` или специфичный для модели отказ входного изображения (`param: input`) | До начала вывода переходит к следующей допустимой цели без охлаждения; см. «Совместимость необязательных параметров» ниже. | +| Структурированный HTTP 400, отклоняющий необязательный `user`, неподдерживаемое значение `reasoning.effort`/`reasoning_effort`, специфичный для модели отказ входного изображения (`param: input`) или отказ маршрута, который не может прочитать входные медиаданные (`unsupported_input_modality`) | До начала вывода переходит к следующей допустимой цели без охлаждения; см. «Совместимость необязательных параметров» ниже. | | Первый вызов инструмента в Responses-ходе внутрипроцессного адаптера (`runTurn`), который текущий запрос не объявлял, до любого вывода и до побочного эффекта, небезопасного для повтора | Охлаждает цель и переходит к следующей с тем же каталогом инструментов. После видимого вывода или небезопасного для повтора побочного эффекта отказ окончателен. Запросы Chat Completions и Anthropic Messages не меняются. | | Любая другая неклассифицированная ошибка | Остановиться и вернуть ошибку. | @@ -331,6 +331,6 @@ CLI и дашборд показывают точное сообщение ва ## Совместимость необязательных параметров -Исключение для завершающих ошибок 400: структурированный отказ от `user`, неподдерживаемое значение `reasoning.effort`/`reasoning_effort` или специфичный для модели отказ входного изображения (`param: input`) позволяет до начала вывода перейти к следующей допустимой цели без периода охлаждения. Отказ политики безопасности, отмена и уже начавшийся вывод по-прежнему запрещают повторное выполнение. +Исключение для завершающих ошибок 400: структурированный отказ от `user`, неподдерживаемое значение `reasoning.effort`/`reasoning_effort`, специфичный для модели отказ входного изображения (`param: input`) или отказ маршрута, который не может прочитать входные медиаданные (`unsupported_input_modality`) позволяет до начала вывода перейти к следующей допустимой цели без периода охлаждения. Отказ политики безопасности, отмена и уже начавшийся вывод по-прежнему запрещают повторное выполнение. [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/tr/guides/combos.md b/docs-site/src/content/docs/tr/guides/combos.md index d2c3be63d49..fe7381e0341 100644 --- a/docs-site/src/content/docs/tr/guides/combos.md +++ b/docs-site/src/content/docs/tr/guides/combos.md @@ -230,7 +230,7 @@ ikiye ayrılır. | HTTP 401, 403, 404, 408, 429 veya herhangi bir 5xx | Hedefi soğutun ve bir sonraki uygun hedefe atlayın. | | Sınıflandırılmış kimlik doğrulama, abonelik, kota, hız sınırı, aşırı yük veya yukarı akış sunucu hatası | Yalnızca durum yeterli olmadığında bile hedefi soğutun ve atlayın. | | İstemci iptali (499), `origin_rejected`, siber politika reddi, bağlam taşması veya diğer geçersiz istek | Durun ve hatayı döndürün; başka bir hedef isteği geçerli kılmaz. | -| `user` alanını açıkça reddeden, `reasoning.effort`/`reasoning_effort` için desteklenmeyen değer bildiren veya modele özgü görüntü girdisini reddeden (`param: input`) yapılandırılmış HTTP 400 | Çıktı başlamadan önce bekleme süresi kaydetmeden sonraki uygun hedefe atlar; aşağıdaki isteğe bağlı parametre uyumluluğuna bakın. | +| `user` alanını açıkça reddeden, `reasoning.effort`/`reasoning_effort` için desteklenmeyen değer bildiren, modele özgü görüntü girdisini reddeden (`param: input`) veya rotanın okuyamadığı girdi medyasını reddeden (`unsupported_input_modality`) yapılandırılmış HTTP 400 | Çıktı başlamadan önce bekleme süresi kaydetmeden sonraki uygun hedefe atlar; aşağıdaki isteğe bağlı parametre uyumluluğuna bakın. | | Süreç içi bir bağdaştırıcının (`runTurn`) yürüttüğü Responses turunda, geçerli isteğin bildirmediği ilk araç çağrısı (herhangi bir çıktıdan ve yeniden oynatılamaz yan etkiden önce) | Hedefi bekleme süresine alır ve aynı araç kataloğuyla sonraki hedefe atlar. Görünür çıktıdan veya yeniden oynatılamaz bir yan etkiden sonra ret kesindir. Chat Completions ve Anthropic Messages istekleri değişmez. | | Diğer sınıflandırılmamış hatalar | Durun ve hatayı döndürün. | @@ -409,6 +409,6 @@ reddedilen istek kaynağını düzeltin. Kombolar bu durumlar için atlama yapma ## İsteğe bağlı parametre uyumluluğu -Sonlandırıcı 400 hatalarının dar bir istisnası vardır: `user` alanını açıkça reddeden, `reasoning.effort`/`reasoning_effort` için desteklenmeyen değer bildiren veya modele özgü görüntü girdisini reddeden (`param: input`) yapılandırılmış hata, çıktı başlamadan önce sonraki uygun hedefe geçebilir. Bu uyumsuzluk için bekleme süresi kaydedilmez. Güvenlik politikası reddi, iptal ve başlamış çıktı yeniden yürütülmez. +Sonlandırıcı 400 hatalarının dar bir istisnası vardır: `user` alanını açıkça reddeden, `reasoning.effort`/`reasoning_effort` için desteklenmeyen değer bildiren, modele özgü görüntü girdisini reddeden (`param: input`) veya rotanın okuyamadığı girdi medyasını reddeden (`unsupported_input_modality`) yapılandırılmış hata, çıktı başlamadan önce sonraki uygun hedefe geçebilir. Bu uyumsuzluk için bekleme süresi kaydedilmez. Güvenlik politikası reddi, iptal ve başlamış çıktı yeniden yürütülmez. [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/zh-cn/guides/combos.md b/docs-site/src/content/docs/zh-cn/guides/combos.md index d0513216b58..35dc0d4dc82 100644 --- a/docs-site/src/content/docs/zh-cn/guides/combos.md +++ b/docs-site/src/content/docs/zh-cn/guides/combos.md @@ -152,7 +152,7 @@ combo 失败分为 **跳转** 失败和 **终止** 失败。 | 被分类为认证、订阅、配额、速率限制、过载或上游服务器错误 | 即使仅凭状态码不足以判断，也会使该目标进入冷却并跳转。 | | 客户端取消（499）、`origin_rejected`、cyber-policy 拒绝、上下文溢出，或其他无效请求 | 停止并返回错误；换其他目标也无法让请求变得有效。 | | HTTP 400 明确表示使用 ChatGPT 账户运行 Codex 时不支持该模型 | 在输出提交前尝试下一个已配置目标。完整拒绝句必须位于 `detail`、`error.message` 或纯文本中；同时包含 `detail` 与 `error` 的混合封装仍会终止请求。此原因本身不会将账户标为需要重新登录，也不会让供应商的所有模型进入冷却。 | -| 结构化 HTTP 400，明确拒绝 `user`、对 `reasoning.effort`/`reasoning_effort` 返回不支持值，或返回模型特定图像输入拒绝（`param: input`） | 在输出开始前跳转到下一个符合条件的目标，且不记录冷却时间；参见下方可选参数兼容性。 | +| 结构化 HTTP 400，明确拒绝 `user`、对 `reasoning.effort`/`reasoning_effort` 返回不支持值、返回模型特定图像输入拒绝（`param: input`），或拒绝该路由无法读取的输入媒体（`unsupported_input_modality`） | 在输出开始前跳转到下一个符合条件的目标，且不记录冷却时间；参见下方可选参数兼容性。 | | 由进程内适配器（`runTurn`）执行的 Responses 回合中，当前请求未声明的第一个工具调用（在任何输出和不可重放的副作用之前） | 让该目标进入冷却，并以相同的工具目录跳转到下一个目标。出现可见输出或不可重放的副作用之后，拒绝即为最终结果。Chat Completions 和 Anthropic Messages 请求不受影响。 | | 任何其他未分类错误 | 停止并返回错误。 | @@ -299,6 +299,6 @@ combo id 不存在。响应是 HTTP 404，类型为 `invalid_request_error`。 ## 可选参数兼容性 -一般 400 错误仍会终止请求，但明确拒绝 `user`、对 `reasoning.effort`/`reasoning_effort` 返回不支持值，或返回模型特定图像输入拒绝（`param: input`）的结构化错误，可让 combo 在输出开始前尝试下一个符合条件的目标，而不记录冷却时间。安全策略拒绝、取消以及已经开始的输出仍不可重放。 +一般 400 错误仍会终止请求，但明确拒绝 `user`、对 `reasoning.effort`/`reasoning_effort` 返回不支持值、返回模型特定图像输入拒绝（`param: input`），或拒绝该路由无法读取的输入媒体（`unsupported_input_modality`）的结构化错误，可让 combo 在输出开始前尝试下一个符合条件的目标，而不记录冷却时间。安全策略拒绝、取消以及已经开始的输出仍不可重放。 [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/docs-site/src/content/docs/zh-tw/guides/combos.md b/docs-site/src/content/docs/zh-tw/guides/combos.md index 3fdb413a417..7073a5baf6d 100644 --- a/docs-site/src/content/docs/zh-tw/guides/combos.md +++ b/docs-site/src/content/docs/zh-tw/guides/combos.md @@ -165,7 +165,7 @@ Combo 失敗分為**跳轉**失敗與**終端**失敗。 | HTTP 401、403、404、408、429 或任何 5xx | 冷卻目標並跳到下一個合格目標。 | | 分類為認證、訂閱、配額、限流、過載或上游伺服器錯誤 | 冷卻目標並跳轉，即使單靠狀態碼不足。 | | 客戶端取消（499）、`origin_rejected`、cyber-policy 拒絕、上下文溢出或其他無效請求 | 停止並回傳錯誤；另一個目標不會讓請求變為有效。 | -| 結構化 HTTP 400，明確拒絕 `user`、對 `reasoning.effort`/`reasoning_effort` 回傳不支援值，或回傳模型特定影像輸入拒絕（`param: input`） | 在輸出開始前跳轉到下一個符合條件的目標，且不記錄冷卻時間；參見下方選用參數相容性。 | +| 結構化 HTTP 400，明確拒絕 `user`、對 `reasoning.effort`/`reasoning_effort` 回傳不支援值、回傳模型特定影像輸入拒絕（`param: input`），或拒絕該路由無法讀取的輸入媒體（`unsupported_input_modality`） | 在輸出開始前跳轉到下一個符合條件的目標，且不記錄冷卻時間；參見下方選用參數相容性。 | | 由行程內轉接器（`runTurn`）執行的 Responses 回合中，目前請求未宣告的第一個工具呼叫（在任何輸出與不可重播的副作用之前） | 讓該目標進入冷卻，並以相同的工具目錄跳轉到下一個目標。出現可見輸出或不可重播的副作用之後，拒絕即為最終結果。Chat Completions 與 Anthropic Messages 請求不受影響。 | | 任何其他未分類錯誤 | 停止並回傳錯誤。 | @@ -296,6 +296,6 @@ Combo id 未知。回應為 HTTP 404 並帶 type `invalid_request_error`。執 ## 選用參數相容性 -一般 400 錯誤仍會終止請求，但明確拒絕 `user`、對 `reasoning.effort`/`reasoning_effort` 回傳不支援值，或回傳模型特定影像輸入拒絕（`param: input`）的結構化錯誤，可讓 combo 在輸出開始前嘗試下一個符合條件的目標，而不記錄冷卻時間。安全政策拒絕、取消及已開始的輸出仍不可重播。 +一般 400 錯誤仍會終止請求，但明確拒絕 `user`、對 `reasoning.effort`/`reasoning_effort` 回傳不支援值、回傳模型特定影像輸入拒絕（`param: input`），或拒絕該路由無法讀取的輸入媒體（`unsupported_input_modality`）的結構化錯誤，可讓 combo 在輸出開始前嘗試下一個符合條件的目標，而不記錄冷卻時間。安全政策拒絕、取消及已開始的輸出仍不可重播。 [Canonical compatibility details](/guides/combos/#request-local-target-compatibility). diff --git a/scripts/test-layout/layout.json b/scripts/test-layout/layout.json index 349c1948639..43077c729ad 100644 --- a/scripts/test-layout/layout.json +++ b/scripts/test-layout/layout.json @@ -1508,6 +1508,7 @@ "responses-routed-web-search-fields.test.ts": "responses", "responses-self-named-namespace-scrub.test.ts": "responses", "responses-combo-exhausted-error.test.ts": "responses", + "responses-combo-run-turn-classified-failure.test.ts": "responses", "responses-send-budget-counts.test.ts": "responses", "responses-send-budget-errors.test.ts": "responses", "responses-memory-models.test.ts": "responses", diff --git a/src/bridge/errors.ts b/src/bridge/errors.ts index 4c464e238d3..fd2016fc6e7 100644 --- a/src/bridge/errors.ts +++ b/src/bridge/errors.ts @@ -21,9 +21,18 @@ export function formatErrorResponse( status: number, type: string, message: string, - options?: { code?: string | null; retryAfter?: string | null }, + options?: { code?: string | null; retryAfter?: string | null; classified?: boolean }, ): Response { const error = classifyError(status, type, message); + // A combo child holding an adapterFailureFromEvent verdict has already classified this + // failure. Re-deriving it from the status folded insufficient_quota into rate_limit_exceeded + // and a 400 modality refusal into invalid_request_error, so the combo loop cooled and hopped + // on the wrong evidence. The child body reaches only that loop, whose consumer re-applies + // this formatter's allowlist before anything reaches a client. The overrides below still win. + if (options?.classified && options.code) { + error.type = type; + error.code = options.code; + } if (options?.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE) { error.type = "server_error"; error.code = SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE; diff --git a/src/combos/failover.ts b/src/combos/failover.ts index ee3c18cf88b..a2fae5e8689 100644 --- a/src/combos/failover.ts +++ b/src/combos/failover.ts @@ -403,6 +403,9 @@ const REQUEST_SHAPE_FAILURE_CODES = new Set([ "tool_catalog_too_large", "cursor_root_envelope_limit", "target_incompatible", + // A route that cannot read the request's media (image input to a text-only CLI route) is + // healthy for every other request, so cooling it would only block unrelated turns. + "unsupported_input_modality", ]); /** Credential/billing failures that every target sharing the provider inherits. */ @@ -836,7 +839,9 @@ export function comboFailureDecision( // says nothing about the next combo target. Structured code only, plus the explicit prose // form upstreams emit when they carry no code, so an unrelated 400 stays terminal. const failureCode = normalizedFailureCode(options?.code || error.code); - if (["model_not_found", "model_unavailable", "unsupported_model"].includes(failureCode)) { + // The input-modality refusal is the same kind of verdict: this route cannot read the turn's + // media, and a later target may. Its 400 would otherwise end the chain as invalid_request_error. + if (["model_not_found", "model_unavailable", "unsupported_model", "unsupported_input_modality"].includes(failureCode)) { return "hop"; } if (modelRefusal === "refusal") return "hop"; diff --git a/src/server/responses/run-turn-execution.ts b/src/server/responses/run-turn-execution.ts index 656eae550d7..d80ed82b0bd 100644 --- a/src/server/responses/run-turn-execution.ts +++ b/src/server/responses/run-turn-execution.ts @@ -789,7 +789,7 @@ export async function executeResponsesRunTurn( const storageRefusal = preflight.error?.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE; const failure = formatErrorResponse(storageRefusal ? 503 : classified.httpStatus, storageRefusal ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : classified.error.type, - redactSecretString(classified.error.message), { code: classified.error.code }); + redactSecretString(classified.error.message), { code: classified.error.code, classified: true }); // A replay-unsafe heartbeat means the adapter already ran a local side effect, so the // combo must not send this turn to another target: the failure stays with this child. if (preflight.replayUnsafe) markResponseNonReplayable(failure); @@ -992,7 +992,7 @@ export async function executeResponsesRunTurn( const storageRefusal = firstMeaningful?.type === "error" && firstMeaningful.code === SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE; const failure = formatErrorResponse(storageRefusal ? 503 : classified.httpStatus, storageRefusal ? SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE : classified.error.type, - redactSecretString(classified.error.message), { code: classified.error.code }); + redactSecretString(classified.error.message), { code: classified.error.code, classified: true }); if (replayUnsafe) markResponseNonReplayable(failure); return failure; } diff --git a/structure/transports/responses-failover.md b/structure/transports/responses-failover.md index a28b890f286..a45b3234e74 100644 --- a/structure/transports/responses-failover.md +++ b/structure/transports/responses-failover.md @@ -1,6 +1,6 @@ # Responses Failover And Replay -Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`; spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. +Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`, and `formatErrorResponse`'s `classified` option keeps that verdict in the child body instead of re-deriving it from the status; spend-ledger, cyber-policy and replay-refusal overrides still win, and `consumeComboFailure` re-applies the client allowlist. Spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response, a child's `insufficient_quota`, and the cool-free hop on `unsupported_input_modality` in `src/combos/failover.ts`. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` and `tests/responses/responses-combo-run-turn-classified-failure.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. Classified 429 admission follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#classified-429-admission). Classified account-entitlement 403 recovery follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#account-entitlement-refusal-recovery). `src/server/responses/compaction-recovery-policy.ts` is a pure eligibility policy, not a dispatcher. It requires explicit configuration and normalized attempt evidence, preserves ordinary requests, diff --git a/tests/fixtures/test-layout-expected.json b/tests/fixtures/test-layout-expected.json index f770d4efd30..5979132bd23 100644 --- a/tests/fixtures/test-layout-expected.json +++ b/tests/fixtures/test-layout-expected.json @@ -82,6 +82,7 @@ "codex-shim-overlay-candidates.test.ts": "codex-integration", "droid-normalized-reasoning-defaults.test.ts": "clients", "responses-combo-exhausted-error.test.ts": "responses", + "responses-combo-run-turn-classified-failure.test.ts": "responses", "cli-config-default-show.test.ts": "cli", "ollama-native-tool-continuations.test.ts": "providers/ollama", "gui-pair-delivery.test.ts": "gui", diff --git a/tests/responses/responses-combo-run-turn-classified-failure.test.ts b/tests/responses/responses-combo-run-turn-classified-failure.test.ts new file mode 100644 index 00000000000..0c4a10bfc60 --- /dev/null +++ b/tests/responses/responses-combo-run-turn-classified-failure.test.ts @@ -0,0 +1,152 @@ +/** + * A runTurn combo child hands the combo loop the failure its adapter already classified. + * The child's status and code drive hop and cooldown decisions; the combo's own failure + * consumer still decides what reaches the client. + */ +import { afterEach, beforeEach, expect, mock, test } from "bun:test"; +import { mkdtempSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import type { ProviderAdapter } from "../../src/adapters/base"; +import type { AdapterEvent, OcxConfig, OcxProviderConfig } from "../../src/types"; +import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; +import { removeTreeWithRetry } from "../helpers/remove-tree"; + +const MODEL = "claude-opus-5-5"; +let childFailure: AdapterEvent | undefined; +let runTurnCalls = 0; +let fallbackCalls = 0; + +const actualResolver = await import("../../src/server/adapter-resolve"); +const actualResolveAdapter = actualResolver.resolveAdapter; +mock.module("../../src/server/adapter-resolve", () => ({ + ...actualResolver, + resolveAdapter(provider: OcxProviderConfig, cacheRetention?: "none" | "short" | "long") { + if (provider.adapter !== "cursor") return actualResolveAdapter(provider, cacheRetention); + return { + name: "cursor", + buildRequest: () => ({ url: provider.baseUrl, method: "POST", headers: {}, body: "" }), + async *parseStream() { yield { type: "error", message: "fixture uses runTurn" } as AdapterEvent; }, + async runTurn(_parsed: unknown, _incoming: unknown, emit: (event: AdapterEvent) => void) { + runTurnCalls++; + if (childFailure) { + emit(childFailure); + return; + } + emit({ type: "text_delta", text: "cursor answered" }); + emit({ type: "done" }); + }, + } as unknown as ProviderAdapter; + }, +})); + +const { handleResponses } = await import("../../src/server/responses"); +const { formatErrorResponse } = await import("../../src/bridge"); +const { SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE } = await import("../../src/lib/errors"); +const { saveCredential } = await import("../../src/oauth/store"); +const { clearComboSelectionState } = await import("../../src/combos/resolve"); +const { clearComboTargetCooldowns } = await import("../../src/combos/failover"); +const { clearGenericFailoverHealth } = await import("../../src/oauth/generic-account-failover"); + +function answer(init?: RequestInit): Response { + fallbackCalls++; + if ((JSON.parse(String(init?.body ?? "{}")) as { stream?: unknown }).stream !== true) { + return Response.json({ + id: "msg_fallback", type: "message", role: "assistant", model: MODEL, + content: [{ type: "text", text: "fallback answered" }], stop_reason: "end_turn", + usage: { input_tokens: 8, output_tokens: 2 }, + }); + } + const event = (name: string, data: unknown) => `event: ${name}\ndata: ${JSON.stringify(data)}\n\n`; + return new Response([ + event("message_start", { type: "message_start", message: { id: "msg_fallback", type: "message", role: "assistant", model: MODEL, content: [], stop_reason: null, usage: { input_tokens: 8, output_tokens: 0 } } }), + event("content_block_start", { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }), + event("content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "fallback answered" } }), + event("content_block_stop", { type: "content_block_stop", index: 0 }), + event("message_delta", { type: "message_delta", delta: { stop_reason: "end_turn" }, usage: { output_tokens: 2 } }), + event("message_stop", { type: "message_stop" }), + ].join(""), { headers: { "content-type": "text/event-stream" } }); +} + +function config(targets: Array<{ provider: string; model: string }>): OcxConfig { + return { + port: 0, + defaultProvider: "cursor", + providers: { + cursor: { adapter: "cursor", baseUrl: "https://api2.cursor.sh", authMode: "oauth", models: [MODEL] }, + fallback: { + adapter: "anthropic", baseUrl: "https://fallback.test", authMode: "key", + apiKey: "fixture-fallback-key", models: [MODEL], + fetch: (async (_input: RequestInfo | URL, init?: RequestInit) => answer(init)) as typeof fetch, + } as OcxProviderConfig, + }, + combos: { "classified-chain": { alias: "chain/classified", strategy: "failover", targets } }, + } as OcxConfig; +} + +async function respond(cfg: OcxConfig, stream: boolean) { + const response = await handleResponses(new Request("http://localhost/v1/responses", { + method: "POST", headers: { "content-type": "application/json" }, + body: JSON.stringify({ model: "chain/classified", input: "describe this", stream }), + }), cfg, { model: "", provider: "" }); + return { status: response.status, text: await response.text() }; +} + +let home = ""; +let releaseSpendHome: (() => void) | undefined; +const originalHome = process.env.OPENCODEX_HOME; +beforeEach(async () => { + home = mkdtempSync(join(tmpdir(), "ocx-combo-classified-failure-")); + process.env.OPENCODEX_HOME = home; + releaseSpendHome = acquireOwnedSpendHome(); + await saveCredential("cursor", { access: "cursor-access", refresh: "cursor-refresh", expires: Date.now() + 3_600_000, accountId: "cursor-account" }); + childFailure = undefined; + runTurnCalls = 0; + fallbackCalls = 0; + clearComboSelectionState(); clearComboTargetCooldowns(); clearGenericFailoverHealth(); +}); +afterEach(() => { + releaseSpendHome?.(); + releaseSpendHome = undefined; + clearComboSelectionState(); clearComboTargetCooldowns(); clearGenericFailoverHealth(); + if (originalHome === undefined) delete process.env.OPENCODEX_HOME; + else process.env.OPENCODEX_HOME = originalHome; + removeTreeWithRetry(home); +}); + +for (const stream of [false, true]) { + test(`a child that cannot take image input hands the turn to the next target (stream=${stream})`, async () => { + childFailure = { + type: "error", status: 400, errorType: "invalid_request_error", code: "unsupported_input_modality", + retryable: false, message: "Claude Code CLI image input is not enabled because the CLI provider route has no verified multimodal contract.", + }; + const response = await respond(config([{ provider: "cursor", model: MODEL }, { provider: "fallback", model: MODEL }]), stream); + expect(response.status).toBe(200); + expect(response.text).toContain("fallback answered"); + expect(runTurnCalls).toBe(1); + expect(fallbackCalls).toBe(1); + }); + + test(`a child's insufficient_quota verdict survives to the exhausted combo (stream=${stream})`, async () => { + childFailure = { + type: "error", status: 429, errorType: "insufficient_quota", code: "insufficient_quota", + retryable: false, message: "Qoder credits are exhausted for this account.", + }; + const response = await respond(config([{ provider: "cursor", model: MODEL }]), stream); + expect(response.status).toBe(429); + expect(JSON.parse(response.text).error).toMatchObject({ type: "insufficient_quota", code: "insufficient_quota" }); + }); +} + +test("a classified child failure keeps its verdict without displacing the formatter's own refusals", async () => { + const quota = formatErrorResponse(429, "insufficient_quota", "credits exhausted", { code: "insufficient_quota", classified: true }); + expect(quota.status).toBe(429); + expect((await quota.json()).error).toMatchObject({ type: "insufficient_quota", code: "insufficient_quota" }); + const storage = formatErrorResponse(429, "insufficient_quota", "ledger offline", + { code: SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE, classified: true }); + expect(storage.status).toBe(503); + expect((await storage.json()).error).toMatchObject({ type: "server_error", code: SPEND_LEDGER_STORAGE_UNAVAILABLE_CODE }); + const cyber = formatErrorResponse(502, "upstream_error", "flagged", { code: "cyber_policy", classified: true }); + expect(cyber.status).toBe(400); + expect((await cyber.json()).error.code).toBe("cyber_policy"); +}); diff --git a/tests/routing/router-combo-failover-classification.test.ts b/tests/routing/router-combo-failover-classification.test.ts index dde82d304d5..b9b05088687 100644 --- a/tests/routing/router-combo-failover-classification.test.ts +++ b/tests/routing/router-combo-failover-classification.test.ts @@ -57,6 +57,7 @@ describe("combo failure cooldown scope", () => { "tool_catalog_too_large", "cursor_root_envelope_limit", "target_incompatible", + "unsupported_input_modality", ]) { expect(comboFailureCooldownScope(400, "refused locally", { code })).toBe("none"); } @@ -111,7 +112,7 @@ describe("combo failure cooldown scope", () => { describe("combo failure hop/stop verdicts", () => { test("model-scoped rejections hop to the next target", () => { - for (const code of ["model_not_found", "model_unavailable", "unsupported_model"]) { + for (const code of ["model_not_found", "model_unavailable", "unsupported_model", "unsupported_input_modality"]) { expect(comboFailureDecision(400, "upstream rejected the model", { code })).toBe("hop"); } }); From 6f78dd707818c4a9893ec04bfba20631f4763234 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 21:38:34 +0900 Subject: [PATCH 6/9] fix(bridge): keep message-read verdicts over a classified child code formatErrorResponse's classified mode let a runTurn child's generic code replace a cyber_policy verdict classifyError read from the full message. consumeComboFailure sees only the first 500 characters, so a refusal phrase past that point became a 502 that hopped to another provider. Cyber-policy, context-length and client-closed verdicts read from the message now outrank the caller's code. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- src/bridge/errors.ts | 6 +++++- ...ponses-combo-run-turn-classified-failure.test.ts | 13 +++++++++++++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/src/bridge/errors.ts b/src/bridge/errors.ts index fd2016fc6e7..3b51c85b6e2 100644 --- a/src/bridge/errors.ts +++ b/src/bridge/errors.ts @@ -17,6 +17,8 @@ import { type OcxErrorPayload, } from "../lib/errors"; +const MESSAGE_VERDICT_CODES = new Set([CYBER_POLICY_ERROR_CODE, "context_length_exceeded", "client_closed_request"]); + export function formatErrorResponse( status: number, type: string, @@ -29,7 +31,9 @@ export function formatErrorResponse( // and a 400 modality refusal into invalid_request_error, so the combo loop cooled and hopped // on the wrong evidence. The child body reaches only that loop, whose consumer re-applies // this formatter's allowlist before anything reaches a client. The overrides below still win. - if (options?.classified && options.code) { + // A verdict read from the full message outranks the caller's code: the consumer sees only the + // first 500 characters, so a generic code replacing it would turn a late cyber refusal into a hop. + if (options?.classified && options.code && !MESSAGE_VERDICT_CODES.has(error.code ?? "")) { error.type = type; error.code = options.code; } diff --git a/tests/responses/responses-combo-run-turn-classified-failure.test.ts b/tests/responses/responses-combo-run-turn-classified-failure.test.ts index 0c4a10bfc60..bfe81e6311f 100644 --- a/tests/responses/responses-combo-run-turn-classified-failure.test.ts +++ b/tests/responses/responses-combo-run-turn-classified-failure.test.ts @@ -136,6 +136,19 @@ for (const stream of [false, true]) { expect(response.status).toBe(429); expect(JSON.parse(response.text).error).toMatchObject({ type: "insufficient_quota", code: "insufficient_quota" }); }); + + // The consumer truncates the child message to 500 characters, so a refusal phrase past that + // point survives only as the child's own cyber_policy code. + test(`a cyber-policy refusal read from a long message still stops the combo (stream=${stream})`, async () => { + childFailure = { + type: "error", status: 502, errorType: "upstream_error", code: "upstream_error", + message: `${"Agent transcript detail. ".repeat(30)}This request was flagged for possible cybersecurity risk.`, + }; + const response = await respond(config([{ provider: "cursor", model: MODEL }, { provider: "fallback", model: MODEL }]), stream); + expect(response.status).toBe(400); + expect(JSON.parse(response.text).error.code).toBe("cyber_policy"); + expect(fallbackCalls).toBe(0); + }); } test("a classified child failure keeps its verdict without displacing the formatter's own refusals", async () => { From cf27ffde7b3f031e5e12249c322ca045e4d648bf Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 21:42:50 +0900 Subject: [PATCH 7/9] fix(bridge): let a classified child code fill in only what the message leaves open The previous guard pinned only cyber-policy, context and client-closed verdicts, so a coding-agent CLI's "Account quota exhausted" reported as a generic 502 upstream_error still lost its insufficient_quota verdict and cooled one target instead of the provider. The caller's code now applies only when classifying the message adds nothing to the status and type, so every message-read verdict (quota, cyber policy, context overflow, catalog size) outranks a generic adapter code. Cyber-policy and client-closed stay pinned. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- src/bridge/errors.ts | 11 +++++++---- ...s-combo-run-turn-classified-failure.test.ts | 18 +++++++++++++++++- 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/src/bridge/errors.ts b/src/bridge/errors.ts index 3b51c85b6e2..5a32e5877a0 100644 --- a/src/bridge/errors.ts +++ b/src/bridge/errors.ts @@ -17,7 +17,7 @@ import { type OcxErrorPayload, } from "../lib/errors"; -const MESSAGE_VERDICT_CODES = new Set([CYBER_POLICY_ERROR_CODE, "context_length_exceeded", "client_closed_request"]); +const PINNED_VERDICT_CODES = new Set([CYBER_POLICY_ERROR_CODE, "client_closed_request"]); export function formatErrorResponse( status: number, @@ -31,9 +31,12 @@ export function formatErrorResponse( // and a 400 modality refusal into invalid_request_error, so the combo loop cooled and hopped // on the wrong evidence. The child body reaches only that loop, whose consumer re-applies // this formatter's allowlist before anything reaches a client. The overrides below still win. - // A verdict read from the full message outranks the caller's code: the consumer sees only the - // first 500 characters, so a generic code replacing it would turn a late cyber refusal into a hop. - if (options?.classified && options.code && !MESSAGE_VERDICT_CODES.has(error.code ?? "")) { + // The caller's code fills in only where the message adds nothing to the status and type. A + // verdict read from the full message (quota, cyber policy, context overflow, catalog size) + // outranks a generic adapter code: the consumer sees only the first 500 characters, so + // replacing it turned a late cyber refusal into a hop and spent quota into a target cooldown. + const messageVerdict = error.code !== classifyError(status, type, "").code; + if (options?.classified && options.code && !messageVerdict && !PINNED_VERDICT_CODES.has(error.code ?? "")) { error.type = type; error.code = options.code; } diff --git a/tests/responses/responses-combo-run-turn-classified-failure.test.ts b/tests/responses/responses-combo-run-turn-classified-failure.test.ts index bfe81e6311f..f9eca5a01e1 100644 --- a/tests/responses/responses-combo-run-turn-classified-failure.test.ts +++ b/tests/responses/responses-combo-run-turn-classified-failure.test.ts @@ -13,6 +13,7 @@ import { acquireOwnedSpendHome } from "../helpers/owned-spend-home"; import { removeTreeWithRetry } from "../helpers/remove-tree"; const MODEL = "claude-opus-5-5"; +const SIBLING_MODEL = "claude-sonnet-5-5"; let childFailure: AdapterEvent | undefined; let runTurnCalls = 0; let fallbackCalls = 0; @@ -73,7 +74,7 @@ function config(targets: Array<{ provider: string; model: string }>): OcxConfig port: 0, defaultProvider: "cursor", providers: { - cursor: { adapter: "cursor", baseUrl: "https://api2.cursor.sh", authMode: "oauth", models: [MODEL] }, + cursor: { adapter: "cursor", baseUrl: "https://api2.cursor.sh", authMode: "oauth", models: [MODEL, SIBLING_MODEL] }, fallback: { adapter: "anthropic", baseUrl: "https://fallback.test", authMode: "key", apiKey: "fixture-fallback-key", models: [MODEL], @@ -149,6 +150,21 @@ for (const stream of [false, true]) { expect(JSON.parse(response.text).error.code).toBe("cyber_policy"); expect(fallbackCalls).toBe(0); }); + + // A coding-agent CLI reports spent account quota as a generic 502 upstream_error; only the + // message says quota. That verdict cools the whole provider, so the sibling model is skipped. + test(`a quota verdict read from the message outranks a generic child code (stream=${stream})`, async () => { + childFailure = { + type: "error", status: 502, errorType: "upstream_error", code: "upstream_error", + message: `${"Agent transcript detail. ".repeat(30)}Account quota exhausted.`, + }; + const response = await respond(config([ + { provider: "cursor", model: MODEL }, { provider: "cursor", model: SIBLING_MODEL }, { provider: "fallback", model: MODEL }, + ]), stream); + expect(response.status).toBe(200); + expect(response.text).toContain("fallback answered"); + expect(runTurnCalls).toBe(1); + }); } test("a classified child failure keeps its verdict without displacing the formatter's own refusals", async () => { From 85c1cd99cca5ff5240a01d8018f3b61e0d730a3e Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 21:43:01 +0900 Subject: [PATCH 8/9] docs(structure): describe when a classified combo-child code applies Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- structure/transports/responses-failover.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/structure/transports/responses-failover.md b/structure/transports/responses-failover.md index a45b3234e74..f9ef7971c9c 100644 --- a/structure/transports/responses-failover.md +++ b/structure/transports/responses-failover.md @@ -1,6 +1,6 @@ # Responses Failover And Replay -Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`, and `formatErrorResponse`'s `classified` option keeps that verdict in the child body instead of re-deriving it from the status; spend-ledger, cyber-policy and replay-refusal overrides still win, and `consumeComboFailure` re-applies the client allowlist. Spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response, a child's `insufficient_quota`, and the cool-free hop on `unsupported_input_modality` in `src/combos/failover.ts`. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` and `tests/responses/responses-combo-run-turn-classified-failure.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. +Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`, and `formatErrorResponse`'s `classified` option keeps that verdict in the child body instead of re-deriving it from the status, wherever the message itself adds nothing to the status and type; a verdict read from the message (quota, cyber policy, context overflow, catalog size) still outranks a generic adapter code, and spend-ledger, cyber-policy and replay-refusal overrides still win, and `consumeComboFailure` re-applies the client allowlist. Spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response, a child's `insufficient_quota`, and the cool-free hop on `unsupported_input_modality` in `src/combos/failover.ts`. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` and `tests/responses/responses-combo-run-turn-classified-failure.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. Classified 429 admission follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#classified-429-admission). Classified account-entitlement 403 recovery follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#account-entitlement-refusal-recovery). `src/server/responses/compaction-recovery-policy.ts` is a pure eligibility policy, not a dispatcher. It requires explicit configuration and normalized attempt evidence, preserves ordinary requests, From fbab4ff9d6c858a7c753c5302f1699c4e7d93c18 Mon Sep 17 00:00:00 2001 From: GOAT Date: Thu, 15 Oct 2026 21:47:59 +0900 Subject: [PATCH 9/9] fix(bridge): compare a classified child code against the status alone An untyped adapter event reaches the formatter with a type adapterFailureFromEvent already inferred from its message, so comparing against an empty message with that same type hid the message's verdict: a 502 upstream_error ending in "Authentication failed." lost invalid_api_key and cooled one target instead of the provider. The baseline now uses a neutral type, so a type or message verdict outranks a generic adapter code. Co-authored-by: 001005HS <99410048+001005HS@users.noreply.github.com> --- src/bridge/errors.ts | 12 +++++++----- structure/transports/responses-failover.md | 2 +- ...nses-combo-run-turn-classified-failure.test.ts | 15 +++++++++++++++ 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/src/bridge/errors.ts b/src/bridge/errors.ts index 5a32e5877a0..9d43a953e26 100644 --- a/src/bridge/errors.ts +++ b/src/bridge/errors.ts @@ -31,11 +31,13 @@ export function formatErrorResponse( // and a 400 modality refusal into invalid_request_error, so the combo loop cooled and hopped // on the wrong evidence. The child body reaches only that loop, whose consumer re-applies // this formatter's allowlist before anything reaches a client. The overrides below still win. - // The caller's code fills in only where the message adds nothing to the status and type. A - // verdict read from the full message (quota, cyber policy, context overflow, catalog size) - // outranks a generic adapter code: the consumer sees only the first 500 characters, so - // replacing it turned a late cyber refusal into a hop and spent quota into a target cooldown. - const messageVerdict = error.code !== classifyError(status, type, "").code; + // The caller's code fills in only where its type and message add nothing to what the status + // alone says. A verdict read from the full message (quota, credentials, cyber policy, context + // overflow, catalog size) outranks a generic adapter code: the consumer sees only the first + // 500 characters, so replacing it turned a late cyber refusal into a hop and spent quota into a + // target cooldown. The baseline uses a neutral type because an untyped adapter event arrives + // with a type adapterFailureFromEvent already inferred from that same message. + const messageVerdict = error.code !== classifyError(status, "upstream_error", "").code; if (options?.classified && options.code && !messageVerdict && !PINNED_VERDICT_CODES.has(error.code ?? "")) { error.type = type; error.code = options.code; diff --git a/structure/transports/responses-failover.md b/structure/transports/responses-failover.md index f9ef7971c9c..a1d9a82eed8 100644 --- a/structure/transports/responses-failover.md +++ b/structure/transports/responses-failover.md @@ -1,6 +1,6 @@ # Responses Failover And Replay -Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`, and `formatErrorResponse`'s `classified` option keeps that verdict in the child body instead of re-deriving it from the status, wherever the message itself adds nothing to the status and type; a verdict read from the message (quota, cyber policy, context overflow, catalog size) still outranks a generic adapter code, and spend-ledger, cyber-policy and replay-refusal overrides still win, and `consumeComboFailure` re-applies the client allowlist. Spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response, a child's `insufficient_quota`, and the cool-free hop on `unsupported_input_modality` in `src/combos/failover.ts`. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` and `tests/responses/responses-combo-run-turn-classified-failure.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. +Combo `metered` eligibility follows the [runtime consent contract](../runtime.md#request-local-target-compatibility), including recovery selections. In `src/server/responses/run-turn-execution.ts`, streaming and buffered pre-output failures retain structured adapter status/type/code via `adapterFailureFromEvent`, and `formatErrorResponse`'s `classified` option keeps that verdict in the child body instead of re-deriving it from the status, wherever its type and message add nothing to what the status alone says; a verdict read from the message (quota, credentials, cyber policy, context overflow, catalog size) still outranks a generic adapter code, and spend-ledger, cyber-policy and replay-refusal overrides still win, and `consumeComboFailure` re-applies the client allowlist. Spend-ledger storage refusal remains 503. This preserves an exhausted subscription chain’s 429 response, a child's `insufficient_quota`, and the cool-free hop on `unsupported_input_modality` in `src/combos/failover.ts`. Coverage: `tests/adapters/anthropic/anthropic-subscription-chain.test.ts` and `tests/responses/responses-combo-run-turn-classified-failure.test.ts`. `src/server/responses/request-transport.ts` resolves the final Anthropic model ID once for each enabled-pool request and holds its model route through admission. The three account-refusal retry sites in `adapter-dispatch.ts`, `adapter-continuation.ts` and `sidecar-execution.ts` use the same route, keep the original refusal if no replacement exists inside it, and preserve existing send and output replay limits. A local cooldown returns 429 with the earliest known Retry-After among accounts the route can use when they recover, and answers a usage limit when every held account is on spent quota ([quota exhaustion answer](../providers/anthropic-account-pool.md#quota-exhaustion-answer-and-fresh-read-recovery)); explicit fallback includes usable ordinary-pool accounts even when the saved route IDs have been removed, while a strict route stays route-scoped. Selection, refusal and 429-rotation logs use the rule’s 1-based `route:#` position, never its name. Classified 429 admission follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#classified-429-admission). Classified account-entitlement 403 recovery follows the [Anthropic account-pool contract](../providers/anthropic-account-pool.md#account-entitlement-refusal-recovery). `src/server/responses/compaction-recovery-policy.ts` is a pure eligibility policy, not a dispatcher. It requires explicit configuration and normalized attempt evidence, preserves ordinary requests, diff --git a/tests/responses/responses-combo-run-turn-classified-failure.test.ts b/tests/responses/responses-combo-run-turn-classified-failure.test.ts index f9eca5a01e1..934124521f3 100644 --- a/tests/responses/responses-combo-run-turn-classified-failure.test.ts +++ b/tests/responses/responses-combo-run-turn-classified-failure.test.ts @@ -165,6 +165,21 @@ for (const stream of [false, true]) { expect(response.text).toContain("fallback answered"); expect(runTurnCalls).toBe(1); }); + + // Without an errorType the adapter infers authentication_error from the message, so that + // inferred type must not make the message look as if it added nothing. + test(`a credential verdict inferred from an untyped child still cools the provider (stream=${stream})`, async () => { + childFailure = { + type: "error", status: 502, code: "upstream_error", + message: `${"Agent transcript detail. ".repeat(30)}Authentication failed.`, + }; + const response = await respond(config([ + { provider: "cursor", model: MODEL }, { provider: "cursor", model: SIBLING_MODEL }, { provider: "fallback", model: MODEL }, + ]), stream); + expect(response.status).toBe(200); + expect(response.text).toContain("fallback answered"); + expect(runTurnCalls).toBe(1); + }); } test("a classified child failure keeps its verdict without displacing the formatter's own refusals", async () => {