Skip to content

Update CrossRouter Submission - #163

Merged
yl231 merged 1 commit into
RouteWorks:mainfrom
JiaHg:add-cross-router-final
Jul 14, 2026
Merged

Update CrossRouter Submission#163
yl231 merged 1 commit into
RouteWorks:mainfrom
JiaHg:add-cross-router-final

Conversation

@JiaHg

@JiaHg JiaHg commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Update CrossRouter Submission

@JiaHg

JiaHg commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

/evaluate

@github-actions

Copy link
Copy Markdown

Router Evaluation Results

Router: cross-router
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.7612
Accuracy 78.14%
Total Cost $2.557320
Avg Cost per Query $0.000304
Avg Cost per 1K Queries $0.3044
Number of Queries 8400
Abnormal Entries 0
Robustness Score 0.6714

Optimality Metrics

Metric Value
Opt.Sel (Optimal Selection) 0.1766
Opt.Cost (Cost Efficiency) 0.4549
Opt.Acc (Accuracy vs Optimal) 0.9031

Evaluation completed by RouterArena automated workflow

@JiaHg

JiaHg commented Jul 10, 2026

Copy link
Copy Markdown
Contributor Author

@yl231 Please update our results(CrossRouter) to leaderboard. Thanks!

@JiaHg

JiaHg commented Jul 12, 2026

Copy link
Copy Markdown
Contributor Author

@yl231 Please update our latest results(CrossRouter) to leaderboard. Thanks!

@yl231 yl231 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer review — approving. Genuine improvement over the merged Cross-Router (routing pool expanded 3→7 models). Faithfulness verified: sub_10 gap is normal (+5.5%, below the trusted-router average) and Opt.Sel is low (0.18) — i.e. not oracle/label-fit. Official /evaluate: Accuracy 78.14% · Arena 0.7612 · $0.3044 per 1K · Robustness 0.6714. All checks green. Thanks @JiaHg — merging; this supersedes the current Cross-Router leaderboard row.

@yl231
yl231 merged commit 5ae22bf into RouteWorks:main Jul 14, 2026
6 checks passed
yl231 added a commit that referenced this pull request Jul 14, 2026
…er (#159); Cross-Router update to #1 (#163) (#165)

Adds three newly-merged router submissions and refreshes the Cross-Router
row from its updated submission. Official /evaluate metrics:

- Cross-Router (#163):   Arena 76.12 · Acc 78.14% · $0.30/1K · Robust 67.14  -> now #1
- Nadir-Tumbler (#159):  Arena 75.17 · Acc 75.34% · $0.08/1K · Robust 66.43  -> #4
- chuzom-solo-v32 (#161): Arena 70.61 · Acc 70.59% · $0.10/1K · Robust 100.00 -> #12
- BARouter (#157):       Arena 67.09 · Acc 68.80% · $0.63/1K · Robust 52.38  -> #15

Manifest entries added for the three new routers so the website sync
regenerates their derived data.

Co-authored-by: yl231 <jamesbond19925@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@yl231

yl231 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Heads-up on a leaderboard correction affecting Cross-Router (#172, closes #166) — flagging transparently.

xAI retired the grok-4-1-fast-reasoning slug on 2026-05-15 and silently redirects it to grok-4.3 (stronger, ~5–6× pricier). Cross-Router routed 217 calls to that slug via OpenRouter, and your artifacts confirm they were served grok-4.3 (model_used=x-ai/grok-4.3) — but they had been billed at the retired slug's old $0.20/$0.50 price. We've corrected the accounting to bill the model that actually answered.

Impact: Arena 76.12 → 75.75, cost $0.30 → $0.40/1K — you remain #1. No action needed on your end. Thanks for the submission!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants