Skip to content

feat: add CNB2API_ALLOW_THINKING opt-in for the reasoning-toggle strip - #8

Open
A8Chann wants to merge 1 commit into
dengyie:mainfrom
A8Chann:feat/allow-thinking-opt-in
Open

A8Chann wants to merge 1 commit into
dengyie:mainfrom
A8Chann:feat/allow-thinking-opt-in

Conversation

@A8Chann

@A8Chann A8Chann commented Oct 12, 2026

Copy link
Copy Markdown

What this does

Adds CNB2API_ALLOW_THINKING=1 as an explicit opt-in that lets client thinking toggles reach the upstream thinking variant, instead of the current unconditional strip. Stripping stays the default — this only adds an escape hatch for deployments that supply a large max_tokens.

Why

331f5aa strips reasoning_effort / reasoning / enable_thinking / thinking at the single outbound choke point. The reasoning behind it is sound and I reproduced the failure it prevents against the real upstream with a hard prompt:

request reasoning content finish
max_tokens=2048 + toggle 4,589 chars 0 length
max_tokens=8192 + toggle 20,305 chars 0 length
max_tokens=2048, no toggle 0 2,994 chars stop

So the degenerate stream is real, and the guard is justified.

The problem is that the guard is absolute. It also excludes callers that do provide an adequate budget, and for them thinking works correctly — the pre-331f5aa traffic did exactly this at max_tokens=60000.

Concretely: this repo was synced into a private deployment on 2026-10-08. Reasoning output across that deployment's session logs went from 1,339,875 characters on 09-18 and 3,031,411 on 09-20 to exactly 0 on 10-08, 10-09 and 10-10 — three days of traffic where thinking silently disappeared. I also confirmed there is no client-side workaround against the live gateway: reasoningEffort (camelCase), enableThinking, chat_template_kwargs, extra_body, and thinking_budget all returned zero reasoning_content, because the upstream recognises only the four stripped spellings.

The change

Three lines of behaviour, plus an Anthropic path fix:

  • src/config.mjs — allowThinking: process.env.CNB2API_ALLOW_THINKING === '1'
  • src/proxy.mjs — stripReasoningTriggers() returns the payload untouched when the flag is set
  • src/anthropic.mjs — fromAnthropicRequest() previously dropped body.thinking unconditionally, so the flag alone would not have helped /v1/messages clients (Claude Code). It now maps thinking:{type:"enabled"} to the upstream shape, plus reasoning_effort when the caller supplies one. budget_tokens is not mapped — there is no upstream field for it. The flag is passed in as an option rather than imported, so anthropic.mjs stays dependency-free.
  • Docs: .env.example, README.md, README.zh-CN.md

Safety

  • Default behaviour is unchanged; the existing strip test passes untouched.
  • Strict opt-in: only the exact string 1 enables it (0 / true / yes / empty all keep stripping).
  • Requests without a thinking toggle behave identically either way.

Tests

ℹ tests 73
ℹ pass 73
ℹ fail 0

New coverage: opt-in forwards reasoning_effort and thinking with other fields untouched; Anthropic adapter maps thinking when enabled, honours type:"disabled", and leaves plain requests alone.

One unrelated note

test/proxy.test.mjs derives the child cwd from new URL('..', import.meta.url).pathname, which yields /D:/... on Windows, so every spawn fails with port not up. This is pre-existing on main — I verified it on an unmodified checkout — and I left it alone to keep this diff focused. Happy to send it as a separate PR if useful (fileURLToPath(new URL('..', import.meta.url)) fixes it).

331f5aa strips reasoning_effort / reasoning / enable_thinking / thinking from
every outbound request, which correctly protects clients from the degenerate
"reasoning eats max_tokens, content never generated" stream. The tradeoff is
that it also locks out callers that do provide an adequate budget.

How that plays out in practice: this repo was synced into a private deployment
on 2026-10-08. Across that deployment's session logs, the provider's reasoning
output went from 1,339,875 characters on 09-18 and 3,031,411 on 09-20 to
exactly 0 on 10-08, 10-09 and 10-10 -- three days of traffic with thinking
silently gone. No client-side spelling could reach the thinking variant again
(reasoningEffort, enableThinking, chat_template_kwargs, extra_body and
thinking_budget were all tried against the live gateway and all returned zero
reasoning_content).

The failure being guarded against is narrower than "thinking is unsafe": it is
"thinking toggle + too-small max_tokens". Reproduced against the real upstream
with a hard prompt:

  max_tokens=2048 + toggle -> reasoning 4589 chars, content 0, finish=length
  max_tokens=8192 + toggle -> reasoning 20305 chars, content 0, finish=length
  max_tokens=2048, no toggle -> content 2994 chars, finish=stop

With a large enough budget the reasoning and the answer are produced in
sequence, which is what the pre-331f5aa traffic did at max_tokens=60000.

So this keeps stripping as the default and adds an explicit opt-in:

  CNB2API_ALLOW_THINKING=1

When set, thinking toggles pass through unchanged and the operator owns the
budget decision. Requests without a toggle are unaffected either way.

The Anthropic adapter is covered too. fromAnthropicRequest() previously
dropped body.thinking unconditionally, so the flag alone would not have helped
/v1/messages clients; it now maps thinking:{type:"enabled"} to the upstream
shape (thinking:{type:"enabled"} plus reasoning_effort when the caller supplies
one). budget_tokens is not mapped -- the upstream has no corresponding field.
The flag is passed in as an option rather than imported, keeping anthropic.mjs
dependency-free.

Tests:
  - opt-in forwards reasoning_effort and thinking, other fields untouched
  - Anthropic adapter maps thinking when enabled, honours type:"disabled",
    and leaves plain requests alone
  - the existing default-strip test is unchanged and still passes

Full suite: 73/73 pass.

Note for Windows contributors: test/proxy.test.mjs uses
new URL('..', import.meta.url).pathname for the child cwd, which yields
"/D:/..." on Windows and makes every spawn fail with "port not up". That is
pre-existing on main and unrelated to this change -- I verified it by running
the suite on an unmodified checkout. Left alone here so the diff stays focused.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant