Repository navigation
Conversation
331f5aa strips reasoning_effort / reasoning / enable_thinking / thinking from every outbound request, which correctly protects clients from the degenerate "reasoning eats max_tokens, content never generated" stream. The tradeoff is that it also locks out callers that do provide an adequate budget. How that plays out in practice: this repo was synced into a private deployment on 2026-10-08. Across that deployment's session logs, the provider's reasoning output went from 1,339,875 characters on 09-18 and 3,031,411 on 09-20 to exactly 0 on 10-08, 10-09 and 10-10 -- three days of traffic with thinking silently gone. No client-side spelling could reach the thinking variant again (reasoningEffort, enableThinking, chat_template_kwargs, extra_body and thinking_budget were all tried against the live gateway and all returned zero reasoning_content). The failure being guarded against is narrower than "thinking is unsafe": it is "thinking toggle + too-small max_tokens". Reproduced against the real upstream with a hard prompt: max_tokens=2048 + toggle -> reasoning 4589 chars, content 0, finish=length max_tokens=8192 + toggle -> reasoning 20305 chars, content 0, finish=length max_tokens=2048, no toggle -> content 2994 chars, finish=stop With a large enough budget the reasoning and the answer are produced in sequence, which is what the pre-331f5aa traffic did at max_tokens=60000. So this keeps stripping as the default and adds an explicit opt-in: CNB2API_ALLOW_THINKING=1 When set, thinking toggles pass through unchanged and the operator owns the budget decision. Requests without a toggle are unaffected either way. The Anthropic adapter is covered too. fromAnthropicRequest() previously dropped body.thinking unconditionally, so the flag alone would not have helped /v1/messages clients; it now maps thinking:{type:"enabled"} to the upstream shape (thinking:{type:"enabled"} plus reasoning_effort when the caller supplies one). budget_tokens is not mapped -- the upstream has no corresponding field. The flag is passed in as an option rather than imported, keeping anthropic.mjs dependency-free. Tests: - opt-in forwards reasoning_effort and thinking, other fields untouched - Anthropic adapter maps thinking when enabled, honours type:"disabled", and leaves plain requests alone - the existing default-strip test is unchanged and still passes Full suite: 73/73 pass. Note for Windows contributors: test/proxy.test.mjs uses new URL('..', import.meta.url).pathname for the child cwd, which yields "/D:/..." on Windows and makes every spawn fail with "port not up". That is pre-existing on main and unrelated to this change -- I verified it by running the suite on an unmodified checkout. Left alone here so the diff stays focused.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on upstream PR #8 (commit
7d2f842). This is an internal draft in the fork, not a submission to the original project.Changes: backwards-compatible off/guarded/passthrough Thinking modes, configurable total token preflight (not a true upstream reasoning cap), correct Anthropic/OpenAI stream error signalling, Windows test cwd fix, regression tests.
CNB live integration is intentionally deferred. CI tests exercise mocks only.
Please don't merge until testing and review are complete.