Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions docs-site/src/content/docs/guides/routing-profile-editor.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The **Models → Routing** tab in the OpenCodex dashboard can manage `config.rou
2. Select **Create profile**.
3. Enter an `id`. The canonical model id is `policy/<id>`.
4. Add one or more explicit provider/model candidates.
5. Configure optional requirements, scoring weights, cost limits, and unknown-evidence behavior.
5. Configure optional requirements, scoring weights, cost limits (`maxEstimatedCostUsd`, optional `onUnknownCost`), and unknown-evidence behavior.
6. Save the profile.

Profile ids are immutable after creation. To use a different id, create a new profile and remove the old one after updating callers.
Expand All @@ -28,6 +28,14 @@ The dashboard sends the same profile object used by `config.routingProfiles` to

A successful save persists the profile through the normal config writer, reconciles live state, and refreshes the model catalog. Validation failures leave the previous configuration unchanged and are shown in the editor.

When `limits.maxEstimatedCostUsd` is configured, `limits.onUnknownCost` defaults to `"allow"`: an unknown cost estimate does not get a
cap-specific exclusion, and dry-run / live route-decision traces stamp
`cost.capOutcome: "unknown-allowed"` so operators can tell the cap was not proven. Set `"exclude"`
when the ceiling must fail closed (`cost-limit-unknown`, with
`cost.capOutcome: "unknown-excluded"`). Configuring `onUnknownCost` alone is inert and does not emit a cap outcome. This is separate from
`unknownEvidence.cost`, which can still exclude or penalize unknown prices independently of the
cap outcome.

## Dry-run a saved profile

Select a saved profile and use **Dry-run evaluation** to add request evidence such as context-window size, tool use, image input, or structured output. Dry-run evaluates eligibility and scoring but never sends an upstream model request.
Expand Down Expand Up @@ -57,7 +65,7 @@ Example save payload:
],
"require": { "tools": true, "minContextWindow": 128000 },
"optimize": { "latency": 0.55, "health": 0.25, "cost": 0.1, "quota": 0.1 },
"limits": { "maxEstimatedCostUsd": 0.5 },
"limits": { "maxEstimatedCostUsd": 0.5, "onUnknownCost": "allow" },
"unknownEvidence": {
"capability": "exclude",
"health": "penalize",
Expand Down
12 changes: 8 additions & 4 deletions docs-site/src/content/docs/reference/configuration/routing.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ namespace, or reserved bare native families (`gpt-*`, `o1-*`, `o3-*`, `o4-*`, `c
| `alias?` | `string` | — | Optional public model id in place of `policy/<id>`. |
| `require?` | object | `{}` | Hard capability requirements evaluated before scoring (see below). |
| `optimize?` | object | latency 0.55, health 0.25, cost 0.10, quota 0.10 | Scoring weights, normalized deterministically. `health`, `quota`, and `cost` have score dimensions; the configured-priority share is `1 - health - quota - cost` (default 0.55), and `latency` folds into that priority share rather than scoring independently. |
| `limits?` | object | — | Hard limits, e.g. `maxEstimatedCostUsd` (enforced by the dry-run evaluator when candidate cost evidence is known). |
| `limits?` | object | — | Hard limits. `maxEstimatedCostUsd` excludes a candidate when its estimated cost is known and above the cap. When that cap is set, `onUnknownCost` (`"allow"` default, or `"exclude"`) controls unknown estimates: allow prevents a cap-specific exclusion and records `cost.capOutcome: "unknown-allowed"`; exclude emits `cost-limit-unknown` and `capOutcome: "unknown-excluded"`. `onUnknownCost` alone (no cap) is inert. Separate from `unknownEvidence.cost`, which can still exclude or penalize unknown prices via `unknown-price` / scoring. |
| `unknownEvidence?` | object | capability `exclude`, health/quota/cost `penalize` | How unknown evidence is treated per dimension: `allow`, `penalize`, or `exclude`. Unknown never becomes zero. |

`require` supports: `minContextWindow` (positive integer), `minQuotaHeadroom` (0..1 fraction),
Expand Down Expand Up @@ -145,7 +145,7 @@ candidate evidence is provided through the API (`POST /api/routing-profiles/dry-
],
"require": { "tools": true, "minContextWindow": 128000 },
"optimize": { "latency": 0.55, "health": 0.25, "cost": 0.10, "quota": 0.10 },
"limits": { "maxEstimatedCostUsd": 0.50 },
"limits": { "maxEstimatedCostUsd": 0.50, "onUnknownCost": "allow" },
"unknownEvidence": {
"capability": "exclude",
"health": "penalize",
Expand Down Expand Up @@ -185,8 +185,12 @@ Both are virtual namespaces with aliases and collision validation; they differ i
is chosen. Profile scoring combines the configured-priority component with the health (RI-06),
quota (RI-07), and cost (RI-08) score dimensions where evidence is present; the `latency` weight
folds into the priority share rather than scoring independently. Cost is also enforced through the
`limits.maxEstimatedCostUsd` cap (a candidate whose estimated cost exceeds the cap is excluded).
Per-request route-decision traces are recorded when a policy profile executes.
`limits.maxEstimatedCostUsd` cap: a candidate whose estimated cost is known and exceeds the cap is
excluded (`cost-limit`). When a cap is configured and the estimate is unknown, the default `limits.onUnknownCost: "allow"`
records `cost.capOutcome: "unknown-allowed"` on the route-decision trace without a cap exclusion;
set `onUnknownCost: "exclude"` for a fail-closed ceiling (`cost-limit-unknown`). Cap outcome is not
overall eligibility — `unknownEvidence.cost: "exclude"` can still add `unknown-price` and mark the
candidate ineligible. Per-request route-decision traces are recorded when a policy profile executes.

### Catalog eligibility

Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ export const de: Record<TKey, string> = {
"routing.candidate": "Kandidat",
"routing.eligible": "Geeignet",
"routing.exclusions": "Ausschlüsse",
"routing.costCap": "Kostenobergrenze",
"routing.capOutcome.satisfied": "innerhalb des Limits",
"routing.capOutcome.exceeded": "über dem Limit",
"routing.capOutcome.unknown-allowed": "unbekannt (erlaubt)",
"routing.capOutcome.unknown-excluded": "unbekannt (ausgeschlossen)",
"routing.exclusion.capability-unsatisfied": "Anforderung nicht erfüllt",
"routing.exclusion.unknown-capability": "unbekannte Fähigkeit",
"routing.exclusion.cost-limit": "über Kostenobergrenze",
"routing.exclusion.cost-limit-unknown": "unbekannte Kosten unter Obergrenze",
"routing.exclusion.cooldown": "Abklingzeit",
"routing.exclusion.unknown-health": "unbekannter Zustand",
"routing.exclusion.unknown-quota": "unbekanntes Kontingent",
"routing.exclusion.unknown-price": "unbekannter Preis",
"routing.exclusion.other": "Ausschluss: {code}",
"routing.score": "Punktzahl",
"routing.selected": "ausgewählt",
"routing.yes": "ja",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,20 @@ export const en = {
"routing.candidate": "Candidate",
"routing.eligible": "Eligible",
"routing.exclusions": "Exclusions",
"routing.costCap": "Cost cap",
"routing.capOutcome.satisfied": "within limit",
"routing.capOutcome.exceeded": "over limit",
"routing.capOutcome.unknown-allowed": "unknown (allowed)",
"routing.capOutcome.unknown-excluded": "unknown (excluded)",
"routing.exclusion.capability-unsatisfied": "capability not met",
"routing.exclusion.unknown-capability": "unknown capability",
"routing.exclusion.cost-limit": "over cost cap",
"routing.exclusion.cost-limit-unknown": "unknown cost under cap",
"routing.exclusion.cooldown": "cooldown",
"routing.exclusion.unknown-health": "unknown health",
"routing.exclusion.unknown-quota": "unknown quota",
"routing.exclusion.unknown-price": "unknown price",
"routing.exclusion.other": "exclusion: {code}",
"routing.score": "Score",
"routing.selected": "selected",
"routing.yes": "yes",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/ja.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ export const ja: Record<TKey, string> = {
"routing.candidate": "候補",
"routing.eligible": "対象",
"routing.exclusions": "除外",
"routing.costCap": "コスト上限",
"routing.capOutcome.satisfied": "上限内",
"routing.capOutcome.exceeded": "上限超過",
"routing.capOutcome.unknown-allowed": "不明(許可)",
"routing.capOutcome.unknown-excluded": "不明(除外)",
"routing.exclusion.capability-unsatisfied": "能力要件未達",
"routing.exclusion.unknown-capability": "能力不明",
"routing.exclusion.cost-limit": "コスト上限超過",
"routing.exclusion.cost-limit-unknown": "上限下でコスト不明",
"routing.exclusion.cooldown": "クールダウン",
"routing.exclusion.unknown-health": "健全性不明",
"routing.exclusion.unknown-quota": "割当不明",
"routing.exclusion.unknown-price": "価格不明",
"routing.exclusion.other": "除外: {code}",
"routing.score": "スコア",
"routing.selected": "選択済み",
"routing.yes": "はい",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/ko.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ export const ko: Record<TKey, string> = {
"routing.candidate": "후보",
"routing.eligible": "적격",
"routing.exclusions": "제외",
"routing.costCap": "비용 상한",
"routing.capOutcome.satisfied": "한도 이내",
"routing.capOutcome.exceeded": "한도 초과",
"routing.capOutcome.unknown-allowed": "알 수 없음(허용)",
"routing.capOutcome.unknown-excluded": "알 수 없음(제외)",
"routing.exclusion.capability-unsatisfied": "기능 미충족",
"routing.exclusion.unknown-capability": "알 수 없는 기능",
"routing.exclusion.cost-limit": "비용 상한 초과",
"routing.exclusion.cost-limit-unknown": "상한 이하 비용 불명",
"routing.exclusion.cooldown": "쿨다운",
"routing.exclusion.unknown-health": "상태 불명",
"routing.exclusion.unknown-quota": "할당량 불명",
"routing.exclusion.unknown-price": "가격 불명",
"routing.exclusion.other": "제외: {code}",
"routing.score": "점수",
"routing.selected": "선택됨",
"routing.yes": "예",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/ru.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ export const ru: Record<TKey, string> = {
"routing.candidate": "Кандидат",
"routing.eligible": "Допустим",
"routing.exclusions": "Исключения",
"routing.costCap": "Лимит стоимости",
"routing.capOutcome.satisfied": "в пределах лимита",
"routing.capOutcome.exceeded": "сверх лимита",
"routing.capOutcome.unknown-allowed": "неизвестно (разрешено)",
"routing.capOutcome.unknown-excluded": "неизвестно (исключено)",
"routing.exclusion.capability-unsatisfied": "требование не выполнено",
"routing.exclusion.unknown-capability": "неизвестная возможность",
"routing.exclusion.cost-limit": "сверх лимита стоимости",
"routing.exclusion.cost-limit-unknown": "неизвестная стоимость при лимите",
"routing.exclusion.cooldown": "пауза",
"routing.exclusion.unknown-health": "неизвестное состояние",
"routing.exclusion.unknown-quota": "неизвестная квота",
"routing.exclusion.unknown-price": "неизвестная цена",
"routing.exclusion.other": "исключение: {code}",
"routing.score": "Оценка",
"routing.selected": "выбран",
"routing.yes": "да",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/tr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,20 @@ export const tr: Record<TKey, string> = {
"routing.candidate": "Aday",
"routing.eligible": "Uygun",
"routing.exclusions": "Hariç Tutulanlar",
"routing.costCap": "Maliyet tavanı",
"routing.capOutcome.satisfied": "limit içinde",
"routing.capOutcome.exceeded": "limit aşıldı",
"routing.capOutcome.unknown-allowed": "bilinmiyor (izinli)",
"routing.capOutcome.unknown-excluded": "bilinmiyor (hariç tutuldu)",
"routing.exclusion.capability-unsatisfied": "yetenek karşılanmadı",
"routing.exclusion.unknown-capability": "bilinmeyen yetenek",
"routing.exclusion.cost-limit": "maliyet tavanı aşıldı",
"routing.exclusion.cost-limit-unknown": "tavan altında bilinmeyen maliyet",
"routing.exclusion.cooldown": "soğuma süresi",
"routing.exclusion.unknown-health": "bilinmeyen sağlık",
"routing.exclusion.unknown-quota": "bilinmeyen kota",
"routing.exclusion.unknown-price": "bilinmeyen fiyat",
"routing.exclusion.other": "hariç tutma: {code}",
"routing.score": "Puan",
"routing.selected": "seçildi",
"routing.yes": "evet",
Expand Down
14 changes: 14 additions & 0 deletions gui/src/i18n/zh.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ export const zh: Record<TKey, string> = {
"routing.candidate": "候选",
"routing.eligible": "合格",
"routing.exclusions": "排除项",
"routing.costCap": "成本上限",
"routing.capOutcome.satisfied": "未超限",
"routing.capOutcome.exceeded": "已超限",
"routing.capOutcome.unknown-allowed": "未知(允许)",
"routing.capOutcome.unknown-excluded": "未知(排除)",
"routing.exclusion.capability-unsatisfied": "能力未满足",
"routing.exclusion.unknown-capability": "能力未知",
"routing.exclusion.cost-limit": "超出成本上限",
"routing.exclusion.cost-limit-unknown": "上限下成本未知",
"routing.exclusion.cooldown": "冷却中",
"routing.exclusion.unknown-health": "健康状态未知",
"routing.exclusion.unknown-quota": "配额未知",
"routing.exclusion.unknown-price": "价格未知",
"routing.exclusion.other": "排除项:{code}",
"routing.score": "分数",
"routing.selected": "已选择",
"routing.yes": "是",
Expand Down
69 changes: 67 additions & 2 deletions gui/src/pages/RoutingProfiles.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import {
type OptionalBoolean,
type RoutingProfileDraft,
type RoutingProfileDto,
type UnknownCostCapMode,
type UnknownEvidenceMode,
} from "../routing-profile-editor-data";
import { readJsonIfOk } from "../fetch-json";
Expand All @@ -23,6 +24,7 @@ type DryRunCandidate = {
eligible: boolean;
exclusions: Array<{ code: string; detail?: string }>;
score?: { total: number; components: Record<string, number | undefined> };
cost?: { capOutcome?: string; estimatedUsd?: number; incomplete?: boolean; limitUsd?: number };
};

type Analytics = {
Expand Down Expand Up @@ -69,6 +71,7 @@ const NUMERIC_REQUIREMENTS = Object.keys(NUMERIC_REQUIREMENT_SPEC) as Array<keyo
const OPTIMIZE_KEYS = ["latency", "health", "cost", "quota"] as const;
const UNKNOWN_EVIDENCE_KEYS = ["capability", "health", "quota", "cost"] as const;
const UNKNOWN_EVIDENCE_OPTIONS: UnknownEvidenceMode[] = ["allow", "penalize", "exclude"];
const UNKNOWN_COST_CAP_OPTIONS: UnknownCostCapMode[] = ["allow", "exclude"];

function fmtMs(value: number | undefined, unavailable: string): string {
return value === undefined ? unavailable : `${Math.round(value)}ms`;
Expand All @@ -78,6 +81,48 @@ function fmtRate(value: number | null | undefined, unavailable: string): string
return value === null || value === undefined ? unavailable : `${Math.round(value * 100)}%`;
}

function fmtCapOutcome(
value: string | undefined,
t: ReturnType<typeof useT>,
unavailable: string,
): string {
switch (value) {
case "satisfied":
return t("routing.capOutcome.satisfied");
case "exceeded":
return t("routing.capOutcome.exceeded");
case "unknown-allowed":
return t("routing.capOutcome.unknown-allowed");
case "unknown-excluded":
return t("routing.capOutcome.unknown-excluded");
default:
return unavailable;
}
}

function fmtExclusion(code: string, t: ReturnType<typeof useT>): string {
switch (code) {
case "capability-unsatisfied":
return t("routing.exclusion.capability-unsatisfied");
case "unknown-capability":
return t("routing.exclusion.unknown-capability");
case "cost-limit":
return t("routing.exclusion.cost-limit");
case "cost-limit-unknown":
return t("routing.exclusion.cost-limit-unknown");
case "cooldown":
return t("routing.exclusion.cooldown");
case "unknown-health":
return t("routing.exclusion.unknown-health");
case "unknown-quota":
return t("routing.exclusion.unknown-quota");
case "unknown-price":
return t("routing.exclusion.unknown-price");
default:
return t("routing.exclusion.other", { code });
}
}

function isPlainObject(value: unknown): value is Record<string, unknown> {
return !!value && typeof value === "object" && !Array.isArray(value);
}
Expand Down Expand Up @@ -709,10 +754,28 @@ export default function RoutingProfiles({
value={draft.limits.maxEstimatedCostUsd}
onChange={event => setDraft(current => current ? {
...current,
limits: { maxEstimatedCostUsd: event.target.value },
limits: { ...current.limits, maxEstimatedCostUsd: event.target.value },
} : current)}
/>
</label>
<label className="field-label">
<code>onUnknownCost</code>
<select
className="input"
value={draft.limits.onUnknownCost}
onChange={event => setDraft(current => current ? {
...current,
limits: {
...current.limits,
onUnknownCost: event.target.value as UnknownCostCapMode,
},
} : current)}
>
{UNKNOWN_COST_CAP_OPTIONS.map(mode => (
<option key={mode} value={mode}>{t(`routing.unknownEvidence.${mode}` as const)}</option>
))}
</select>
</label>
</fieldset>

<fieldset style={{ border: 0, padding: 0, margin: 0 }}>
Expand Down Expand Up @@ -817,6 +880,7 @@ export default function RoutingProfiles({
<th>{t("routing.candidate")}</th>
<th>{t("routing.eligible")}</th>
<th>{t("routing.exclusions")}</th>
<th>{t("routing.costCap")}</th>
<th>{t("routing.score")}</th>
</tr>
</thead>
Expand All @@ -828,7 +892,8 @@ export default function RoutingProfiles({
{index === dryRunResult.selectedIndex ? ` ✓ (${t("routing.selected")})` : ""}
</td>
<td>{candidate.eligible ? t("routing.yes") : t("routing.no")}</td>
<td>{candidate.exclusions.map(exclusion => exclusion.code).join(", ") || t("routing.none")}</td>
<td>{candidate.exclusions.map(exclusion => fmtExclusion(exclusion.code, t)).join(", ") || t("routing.none")}</td>
<td>{fmtCapOutcome(candidate.cost?.capOutcome, t, unavailable)}</td>
Comment thread
coderabbitai[bot] marked this conversation as resolved.
<td>{candidate.score ? candidate.score.total.toFixed(3) : unavailable}</td>
</tr>
))}
Expand Down
Loading
Loading