Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions router_inference/config/nadir-caliper-2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"pipeline_params": {
"router_name": "nadir-caliper-2",
"router_cls_name": "NadirCaliper2Router",
"models": [
"qwen/qwen3-235b-a22b-2507",
"google/gemini-3.1-flash-lite",
"grok-4.3",
"deepseek/deepseek-v3.2"
],
"description": "Nadir-Caliper-2, from Nadir Lab. Supersedes the pending nadir-caliper submission (PR #173) and the nadir-tumbler leaderboard entry; please keep a single Nadir Lab row. Identical consensus cascade to nadir-caliper with one branch changed, selected on our RouterBench-only analysis before this submission: two cheap probes (qwen3-235b, gemini-3.1-flash-lite) answer at temperature 0; agreement ships qwen; disagreement is arbitrated by grok-4.3 (low reasoning effort), shipping the cheapest confirmed model. Where the arbiter confirms neither probe, the router now ships qwen rather than the arbiter: on our RouterBench labels that branch selects questions all models fail (grok correct 12%, qwen 15%), so the premium answer bought no accuracy at 25x the price. grok-4.3 is therefore consult-only and never shipped. All fitting on RouterBench only, frozen before routing; responses self-generated with recorded token usage; rows generated 2026-08-05 carry per-row provenance (requested_model, model_used, provider, invoked_at). deepseek-v3.2 substitutes as probe-B on 44 rate-limited rows. The robustness split is a genuine re-run of the identical rule on all 420 perturbed prompts.",
"router_homepage": "https://getnadir.com",
"contact_email": "info@getnadir.com",
"affiliation": "Nadir Lab",
"github": "https://github.com/NadirRouter"
}
}

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions router_inference/predictions/nadir-caliper-2.json

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions universal_model_names.py
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@
# xAI models
"grok-4",
"grok-4-1-fast-reasoning",
"grok-4.3",
# R2-Router
"qwen/qwen3-235b-a22b-2507",
"qwen/qwen3-next-80b-a3b-instruct",
Expand Down
Loading