Repository navigation
feat: opt-in thinking with max_tokens budget floor (refs dengyie/cnb2api#9) - #6
Merged
Merged
Conversation
- config: CNB2API_ALLOW_THINKING / CNB2API_THINKING_MIN_MAX_TOKENS + fail-fast
- proxy: passthrough when allowThinking, apply budget floor (raise-only)
- anthropic: fromAnthropicRequest(body, {allowThinking}) maps thinking:{type:enabled}
- server: pass allowThinking through
- tests: fix Windows cwd bug (fileURLToPath); +6 cases (71 -> 77 green)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
对应上游 issue dengyie#9(最新版双协议均无法思考)。
根因不是"上游不支持思考",而是上游的 thinking 变体会把推理阶段计入
max_tokens:预算不足时推理吃穿预算 →
finish_reason=length、content全空,答案永不生成。为此
src/proxy.mjs在唯一出站点无条件剥除了reasoning_effort/reasoning/enable_thinking/thinking,锁死非 thinking 变体;Anthropic 侧的fromAnthropicRequest()本身也不透传顶层thinking—— 两道关卡叠加即 issue 所述"双重失效"。复现证据(对真实上游实测):
8192 那次的 reasoning 产出约 20,305 字符,正好撞在 8192 token 的墙上 ——
推理工作正常,只是预算不够它写完。所以思考是被
max_tokens饿死的,解法必须是放开开关 + 同时给足预算,两者缺一不可。
本 PR 做了什么
在上游 PR dengyie#8 思路(开关 opt-in)基础上补上了缺失的预算兜底 —— 只放开开关而不给预算,
仍然会退化成空答案:
src/config.mjs— 新增allowThinking(CNB2API_ALLOW_THINKING === '1')与thinkingMinMaxTokens(CNB2API_THINKING_MIN_MAX_TOKENS,0 = 不兜底);新增 fail-fast 校验(NaN / 负数启动即拒),并对"设了兜底却没开开关"给启动警告。
src/proxy.mjs—stripReasoningTriggers()在开启时原样放行;新增applyThinkingBudgetFloor(),只抬高、不压低max_tokens,与剥除共用同一出站点。src/anthropic.mjs—fromAnthropicRequest(body, { allowThinking })新增第二参数;开启时把
thinking:{type:'enabled'}映射为上游形态(type:'disabled'与缺省不加字段);budget_tokens在上游无对应字段,不映射。src/server.mjs— 调用处传入{ allowThinking: config.allowThinking }。.env.example— 新增配置段,说明为什么默认剥除。默认行为零改动
不设
CNB2API_ALLOW_THINKING时,所有行为与补丁前逐字节一致(原有的剥除测试原样通过)。顺手修了一个既有 bug
test/proxy.test.mjs里cwd: new URL('..', import.meta.url).pathname在 Windows 上会得到/D:/...(前导斜杠),导致 23 个用例全部port not up。改用fileURLToPath修复。测试
(原始 71 + 新增 6:开关透传(流式/非流式)、预算兜底只抬不压、不设兜底时交由客户端控制、
非法兜底值 fail-fast、默认关闭时兜底不生效、Anthropic thinking 映射。)
端到端冒烟(mock 上游模拟两种变体)