Relay anthropic-beta tokens on the native Anthropic Messages routes - #1069
PierreLeGuen wants to merge 1 commit into
Conversation
The native Messages routes validated every anthropic-beta token against a static allowlist. Each client release adds dated tokens, so the first unknown one 400s the whole request and the native lane stops serving current clients. This is the third recurrence in three weeks (#970, #988, #1068) and the allowlist buys nothing: Anthropic already answers a name it does not recognise with its own 400, and that response relays unbilled. Treat the header as transport instead. Tokens are syntax-checked (ASCII letters, digits, '-', '_', '.'; 128 bytes per token, 64 distinct tokens), deduplicated in order, and forwarded verbatim. Billing and product policy stay where they already live — the body gates in reject_unsupported_features reject speed=fast, non-standard service tiers, inference_geo, mcp_servers, container, server-side fallbacks, typed Anthropic tools and one-hour cache TTL — so a beta token alone cannot unlock a premium or server-side feature. ANTHROPIC_ALLOWED_BETAS becomes ANTHROPIC_DENIED_BETAS: an operator kill-switch for a future header-only premium beta, settable without a release. The retired variable is ignored with a startup warning so the deploy window where it is still set explains itself. Fixes #988 Fixes #1068
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Review: relay
|
Review · Status🟩 CompletedIronLoop completed the review and posted it to GitHub. ResultRun detailsAutomatic trigger · attempt 1 of 3 · completed in 3m 33s |
|
✅ OpenCodeReview: Review partially complete: 0 finding(s); 1 of 3 selected item(s) failed. |
lloydmak99
left a comment
lloydmak99
left a comment
There was a problem hiding this comment.
Clean, well-contained change: the static anthropic-beta allowlist is replaced with syntax-validation + verbatim relay plus an operator denylist kill-switch, and ANTHROPIC_ALLOWED_BETAS → ANTHROPIC_DENIED_BETAS is renamed cleanly. Core relay logic checks out; approving.
Non-blocking follow-up:
normalized_beta_header(crates/api/src/routes/anthropic.rs:855-903) relays any Anthropic-accepted beta that reshapes theusageobject, which can 502 the non-streaming path (anthropic.rs:995-1004) after upstream has already billed the work. This is a documented tradeoff mitigated only by theANTHROPIC_DENIED_BETASkill-switch — fine to keep, worth watching.
Local checks: git diff --check passed; upstream header construction, non-2xx billing gate, body-gate enforcement, and config rename/parse verified by cross-reference. Focused cargo test (routes::anthropic, config) could not run locally (no cc linker), but CI lint/unit/integration/E2E jobs passed.
![ironloopai[bot]](https://avatars.githubusercontent.com/in/4116490?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review · Summary
🟢 No actionable findings
No actionable defects found in the pull request.
Validation
- ✅ Change inspection — Reviewed the changed beta-token validation, denial configuration, upstream relay path, documentation, and cache-walk defaults.
- ✅ Diff hygiene — The changed diff has no whitespace errors.
Review details
- Run:
694938d7-9110-4837-b249-32d5d3048cba - Attempts: 1
2 participants
Summary
anthropic-betatokens against a static allowlist on/v1/messagesand/v1/messages/count_tokens; syntax-check them and relay them to Anthropic verbatimANTHROPIC_ALLOWED_BETASwithANTHROPIC_DENIED_BETAS, an operator kill-switch that refuses a named token without a releaseFixes #988. Fixes #1068. Same class as #970.
Why
Each client release adds dated beta tokens. The allowlist rejects the first unknown one, which fails the whole request, so the native lane stops serving current clients until someone edits an env var. That has now happened three times in three weeks (#970 on 08-23, #988 on 08-28, #1068 today), and the reporter's production cutover is blocked on the class rather than on any one token.
The allowlist was never the protection it looked like:
invalid_request_error— "Unexpected value(s)…for theanthropic-betaheader" — for an invalid name or one the organization cannot access. Relaying an unknown token produces that same 400, from the party that actually knows, and a non-2xx upstream response relays without billing.reject_unsupported_featuresalready refusesspeed: "fast",service_tierother thanstandard_only,inference_geo,mcp_servers,container, server-sidefallbacks, every typed Anthropic tool and one-hour cache TTL. Every premium beta is inert without one of those fields, so the header alone cannot unlock one. A test now pins that:fast-mode-2026-02-01relays, whilespeed: "fast"in the body is still rejected.x-api-keyand never forwardsAuthorization, so anoauth-*token is a no-op; endpoint-scoped betas name paths this router does not serve.What relaying admits
Tokens are still bounded: ASCII letters, digits,
-,_and., at most 128 bytes each and 64 distinct tokens per request, deduplicated in order. The bounds are roughly triple and five times the largest values Anthropic has published, so this endpoint is never the reason a plausible future token fails. Rejections carry their own wording rather than the old "not supported by the native Anthropic endpoint yet", so an operator denial or malformed input is distinguishable from the retired gate in triage.Residual risk
A future header-only premium beta — one that raises Anthropic's price without a body field — would be relayed and billed upstream at premium while we meter at catalog rates.
ANTHROPIC_DENIED_BETAScovers that in an env change plus a rolling update, no release. The only such toggle to date,context-1m-2025-08-07, is a no-op on current models (#979), and the provider's recent premium features are all body-gated and already refused.Separately and unchanged by this PR:
anthropic/claude-sonnet-4-5is in the catalog at 200K, and on that model the long-context beta is premium-priced above 200K input while we bill flat. That token has been admitted since the #988 mitigation and current clients send it for 1M-suffixed models, so it cannot go in a global denylist; I will file the per-model follow-up separately.Rollout
ANTHROPIC_ALLOWED_BETASstays set in staging and production until this image is live; the new image ignores it and logs a one-line startup warning, and the old image ignoresANTHROPIC_DENIED_BETAS, so neither deploy order breaks. nearai/cvm-ansible-playbooks#683 admits today's two tokens as the runtime mitigation in the meantime; the variable is removed from the playbooks one promote cycle after this reaches production, so a rollback target is also a relaying image.Validation
cargo fmt --all -- --checkcargo clippy -p api -p config --all-targets -- -D warningscargo test -p api --lib routes::anthropic(23 passed)cargo test -p config --lib(61 passed)git diff --checkspeed: "fast"body gate to follow on the deployed image