Repository navigation
Conversation
331f5aa strips reasoning_effort / reasoning / enable_thinking / thinking from every outbound request, which correctly protects clients from the degenerate "reasoning eats max_tokens, content never generated" stream. The tradeoff is that it also locks out callers that do provide an adequate budget. How that plays out in practice: this repo was synced into a private deployment on 2026-10-08. Across that deployment's session logs, the provider's reasoning output went from 1,339,875 characters on 09-18 and 3,031,411 on 09-20 to exactly 0 on 10-08, 10-09 and 10-10 -- three days of traffic with thinking silently gone. No client-side spelling could reach the thinking variant again (reasoningEffort, enableThinking, chat_template_kwargs, extra_body and thinking_budget were all tried against the live gateway and all returned zero reasoning_content). The failure being guarded against is narrower than "thinking is unsafe": it is "thinking toggle + too-small max_tokens". Reproduced against the real upstream with a hard prompt: max_tokens=2048 + toggle -> reasoning 4589 chars, content 0, finish=length max_tokens=8192 + toggle -> reasoning 20305 chars, content 0, finish=length max_tokens=2048, no toggle -> content 2994 chars, finish=stop With a large enough budget the reasoning and the answer are produced in sequence, which is what the pre-331f5aa traffic did at max_tokens=60000. So this keeps stripping as the default and adds an explicit opt-in: CNB2API_ALLOW_THINKING=1 When set, thinking toggles pass through unchanged and the operator owns the budget decision. Requests without a toggle are unaffected either way. The Anthropic adapter is covered too. fromAnthropicRequest() previously dropped body.thinking unconditionally, so the flag alone would not have helped /v1/messages clients; it now maps thinking:{type:"enabled"} to the upstream shape (thinking:{type:"enabled"} plus reasoning_effort when the caller supplies one). budget_tokens is not mapped -- the upstream has no corresponding field. The flag is passed in as an option rather than imported, keeping anthropic.mjs dependency-free. Tests: - opt-in forwards reasoning_effort and thinking, other fields untouched - Anthropic adapter maps thinking when enabled, honours type:"disabled", and leaves plain requests alone - the existing default-strip test is unchanged and still passes Full suite: 73/73 pass. Note for Windows contributors: test/proxy.test.mjs uses new URL('..', import.meta.url).pathname for the child cwd, which yields "/D:/..." on Windows and makes every spawn fail with "port not up". That is pre-existing on main and unrelated to this change -- I verified it by running the suite on an unmodified checkout. Left alone here so the diff stays focused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
Adds
CNB2API_ALLOW_THINKING=1as an explicit opt-in that lets client thinking toggles reach the upstream thinking variant, instead of the current unconditional strip. Stripping stays the default — this only adds an escape hatch for deployments that supply a largemax_tokens.Why
331f5aastripsreasoning_effort/reasoning/enable_thinking/thinkingat the single outbound choke point. The reasoning behind it is sound and I reproduced the failure it prevents against the real upstream with a hard prompt:max_tokens=2048+ togglelengthmax_tokens=8192+ togglelengthmax_tokens=2048, no togglestopSo the degenerate stream is real, and the guard is justified.
The problem is that the guard is absolute. It also excludes callers that do provide an adequate budget, and for them thinking works correctly — the pre-
331f5aatraffic did exactly this atmax_tokens=60000.Concretely: this repo was synced into a private deployment on 2026-10-08. Reasoning output across that deployment's session logs went from 1,339,875 characters on 09-18 and 3,031,411 on 09-20 to exactly 0 on 10-08, 10-09 and 10-10 — three days of traffic where thinking silently disappeared. I also confirmed there is no client-side workaround against the live gateway:
reasoningEffort(camelCase),enableThinking,chat_template_kwargs,extra_body, andthinking_budgetall returned zeroreasoning_content, because the upstream recognises only the four stripped spellings.The change
Three lines of behaviour, plus an Anthropic path fix:
src/config.mjs—allowThinking: process.env.CNB2API_ALLOW_THINKING === '1'src/proxy.mjs—stripReasoningTriggers()returns the payload untouched when the flag is setsrc/anthropic.mjs—fromAnthropicRequest()previously droppedbody.thinkingunconditionally, so the flag alone would not have helped/v1/messagesclients (Claude Code). It now mapsthinking:{type:"enabled"}to the upstream shape, plusreasoning_effortwhen the caller supplies one.budget_tokensis not mapped — there is no upstream field for it. The flag is passed in as an option rather than imported, soanthropic.mjsstays dependency-free..env.example,README.md,README.zh-CN.mdSafety
1enables it (0/true/yes/ empty all keep stripping).Tests
New coverage: opt-in forwards
reasoning_effortandthinkingwith other fields untouched; Anthropic adapter mapsthinkingwhen enabled, honourstype:"disabled", and leaves plain requests alone.One unrelated note
test/proxy.test.mjsderives the childcwdfromnew URL('..', import.meta.url).pathname, which yields/D:/...on Windows, so every spawn fails withport not up. This is pre-existing onmain— I verified it on an unmodified checkout — and I left it alone to keep this diff focused. Happy to send it as a separate PR if useful (fileURLToPath(new URL('..', import.meta.url))fixes it).