Repository navigation
fix(anthropic): give each Claude model its real 128K output maximum #6329
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,9 +19,38 @@ export const ANTHROPIC_MODEL_INPUT_MODALITIES: Record<string, string[]> = Object | |
| ANTHROPIC_MODELS.map(id => [id, ["text", "image"]]), | ||
| ); | ||
| // Every current Claude family accepts at least 64k output tokens (Haiku 4.5 / Sonnet 4.x | ||
| // through Opus 5 and Fable 5). Anthropic caps max_tokens per model server-side, so a | ||
| // larger request never over-allocates; it only stops the 8192 truncation. | ||
| // through Opus 5 and Fable 5), so this stays the floor for a model id not listed in | ||
| // ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS below. | ||
| export const ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS = 64_000; | ||
| /** | ||
| * Per-model synchronous Messages API output maxima, each read from that model's own overview | ||
| * page on platform.claude.com (`/docs/en/models/<slug>/overview`), which states the number in | ||
| * its header line and again in the Capabilities table. Deliberately NOT the 300K Message | ||
| * Batches extended-output beta, which needs `output-300k-2026-03-24` and does not apply to the | ||
| * synchronous route these providers use. | ||
| * | ||
| * Every seeded Claude model except Haiku 4.5 is 128K. Haiku 4.5 is genuinely 64K and is left | ||
| * out so it keeps inheriting ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS. | ||
| * | ||
| * Without this the 64000 default stood in for all of them, which understates the real ceiling | ||
| * by half on every Opus, Sonnet and Fable model. `resolveOutputCeiling` is the value the | ||
| * adapter budgets against when a caller declares no `max_tokens`, so a 128K-capable model | ||
| * stopped at 64000 with `stop_reason: max_tokens`; the same figure feeds the combo admission | ||
| * reserve in `checkComboTargetInputAdmission`, where understating a target's output makes its | ||
| * headroom arithmetic wrong. | ||
| */ | ||
| export const ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS: Record<string, number> = { | ||
| "claude-fable-5-1": 128_000, | ||
|
Comment on lines
+42
to
+43
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a request uses a concrete Anthropic ID such as Useful? React with 👍 / 👎. |
||
| "claude-fable-5": 128_000, | ||
| "claude-sonnet-5-5": 128_000, | ||
| "claude-sonnet-5": 128_000, | ||
| "claude-opus-5-5": 128_000, | ||
| "claude-opus-5": 128_000, | ||
| "claude-opus-4-8": 128_000, | ||
| "claude-opus-4-7": 128_000, | ||
| "claude-opus-4-6": 128_000, | ||
| "claude-sonnet-4-6": 128_000, | ||
| }; | ||
| /** | ||
| * The effort rungs opencodex exposes for native Anthropic models. Without this the | ||
| * providers advertised no ladder at all, so every client that keys its effort control off | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| import { describe, expect, test } from "bun:test"; | ||
| import { providerConfigSeed } from "../../../src/providers/derive"; | ||
| import { getProviderRegistryEntry } from "../../../src/providers/registry"; | ||
| import { | ||
| ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS, | ||
| ANTHROPIC_MODELS, | ||
| ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS, | ||
| } from "../../../src/providers/registry/model-seeds"; | ||
| import { resolveModelPolicy } from "../../../src/providers/resolved-model-policy"; | ||
| import { resolveOutputCeiling } from "../../../src/server/responses/input-admission"; | ||
|
|
||
| // Anthropic's per-model overview pages (platform.claude.com/docs/en/models/<slug>/overview, | ||
| // read 2026-09-30) put the synchronous Messages API maximum at 128K for every seeded Claude | ||
| // model except Haiku 4.5, which is 64K. The 300K figure on those pages belongs to the Message | ||
| // Batches extended-output beta and does not apply to the synchronous route. | ||
| const HIGH_OUTPUT = 128_000; | ||
| const HAIKU = "claude-haiku-4-5"; | ||
| const PROVIDERS = ["anthropic", "anthropic-apikey"] as const; | ||
|
|
||
| const entry = (name: string) => getProviderRegistryEntry(name)!; | ||
|
|
||
| function policy(providerName: string, modelId: string) { | ||
| const registryEntry = entry(providerName); | ||
| const authMode = providerName === "anthropic" ? "oauth" : "key"; | ||
| return resolveModelPolicy({ | ||
| providerName, | ||
| modelId, | ||
| provider: { adapter: registryEntry.adapter, baseUrl: registryEntry.baseUrl, authMode }, | ||
| registryEntry, | ||
| transportMatchedRegistry: true, | ||
| effectiveAuth: { authMode }, | ||
| }); | ||
| } | ||
|
|
||
| describe("Anthropic per-model output maxima", () => { | ||
| test("the table covers every seeded model except the one that really is 64K", () => { | ||
| expect(Object.keys(ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS).sort()) | ||
| .toEqual(ANTHROPIC_MODELS.filter(id => id !== HAIKU).sort()); | ||
| for (const maximum of Object.values(ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS)) { | ||
| expect(maximum).toBe(HIGH_OUTPUT); | ||
| } | ||
| // Haiku 4.5 carries no entry on purpose, so it must still resolve the default and not 128K. | ||
| expect(ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS[HAIKU]).toBeUndefined(); | ||
| expect(ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS).toBe(64_000); | ||
| }); | ||
|
|
||
| for (const providerName of PROVIDERS) { | ||
| test(`${providerName} resolves each model's real ceiling rather than the 64K default`, () => { | ||
| expect(entry(providerName).modelMaxOutputTokens).toEqual(ANTHROPIC_MODEL_MAX_OUTPUT_TOKENS); | ||
| const provider = providerConfigSeed(entry(providerName)); | ||
| for (const modelId of ANTHROPIC_MODELS) { | ||
| const expected = modelId === HAIKU ? ANTHROPIC_DEFAULT_MAX_OUTPUT_TOKENS : HIGH_OUTPUT; | ||
| expect(policy(providerName, modelId).model.maxOutputTokens, modelId).toBe(expected); | ||
| // The admission path reads the ceiling through its own resolver, so assert that too: | ||
| // it is what the adapter budgets against when a caller declares no max_tokens, and what | ||
| // the combo reserve is computed from. | ||
| expect(resolveOutputCeiling(provider, providerName, modelId)).toBe(expected); | ||
| } | ||
| }); | ||
| } | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This registry field changes the effective Anthropic wire behavior for every omitted
max_tokensrequest, but the commit does not update the mappedstructure/contract documentation, such as the provider registry/Anthropic adapter description instructure/providers-and-adapters.md. Record the new per-model output-budget contract in the same change as required for ownedsrc/providers/changes.AGENTS.md reference: AGENTS.md:L38-L46
Useful? React with 👍 / 👎.