Skip to content

fix(stream): send full disable signals when thinking is off - #40

Merged
simonsmh merged 1 commit into
simonsmh:mainfrom
fredtanxo:fix/thinking-off
Oct 22, 2026
Merged

simonsmh merged 1 commit into
simonsmh:mainfrom
fredtanxo:fix/thinking-off

Conversation

@fredtanxo

Copy link
Copy Markdown
Contributor

Problem

Turning thinking off still leaves the model reasoning. The off path only sent parameters.enable_thinking: false, but the request kept model_config.is_reasoning: true (and chat_context.extra.modelConfig.is_reasoning), so Qoder's gateway still routed the model as reasoning-enabled.

@qoder-ai/qodercli disables thinking with three signals together (reverse-engineered from its published bundle):

t.parameters   = { ..., reasoning_effort: V, enable_thinking: V !== "none" }
t.model_config = { ..., is_reasoning: V !== "none" }

i.e. enable_thinking: false + reasoning_effort: "none" + is_reasoning: false.

Fix

  • Normalize every host's "off" signal into one explicitlyOff flag: pi and OMP both resolve level "off" to reasoning: undefined; OMP's fork additionally sets disableReasoning / forceReasoningOff; false / "off" cover older hosts.
  • Skip clamping when off, so an off request is never snapped back up to a concrete level (previously clampThinkingLevel could lift "off" to the model's lowest effort).
  • Emit the three disable signals whenever no concrete level survives (reasoningLevel === undefined): enable_thinking: false, (reasoning_effort: "none", is_reasoning: false (both chat_context and model_config).

Tests

Three regression cases assert the request body when thinking is off: reasoning: 'off", disableReasoning: true (OMP), and omitted reasoning (pi's off signal).

Notes

  • disableReasoning / forceReasoningOff are OMP-fork fields not present in @earendil-works/pi-ai's SimpleStreamOptions; they are read through a type assertion, same as the existing reasoning !== false && reasoning !== "off" guard.
  • For models whose catalog lacks a disabled branch (Kimi-K3, GLM-5.3, Sonus, Cantus, ...), off now still sends reasoning_effort: "none". qodercli hides the "none" option for those models, so the gateway may accept or ignore it - but the request is at least self-consistent now, no longer "disable + is_reasoning: true".

The off path only sent `enable_thinking: false`, but goder's gateway
still routes the reasoning model as reasoning-enabled because
`model_config.is_reasoning` stayed `true`. qodercli disables thinking by
emitting three signals together: `enable_thinking: false`, `reasoning
effort: "none"`, and `is_reasoning: false`.

Normalize every host's "off" signal into one `explicitlyOff` flag pi and
OMP both omit `reasoning` (resolve "off" to `undefined`), OMP's fork
additionally sets `disableReasoning` / `forceReasoningOff` - so off is
never clamped back up to a concrete level, then emit all three disable
signals consistently keyed off `reasoningLevel`.
@simonsmh
simonsmh merged commit 255e07e into simonsmh:main Oct 22, 2026
simonsmh added a commit that referenced this pull request Oct 22, 2026
…ed branch

Live verification of #40 against the real gateway: models whose catalog
has no disabled branch (e.g. Sonus, backed by an upstream that only
accepts low/medium/high/xhigh/max) reject reasoning_effort "none" with
a 400, and the omitted-reasoning path hits the same branch. Only send
the effort signal when the catalog does not positively show the branch
is missing (thinking_config efforts without disabled, or
thinkingLevelMap off: null); enable_thinking:false and is_reasoning:
false still go out, and models with a disabled branch are unchanged.

Adds two regression tests seeding the model cache for both catalog
shapes; Sonus no longer errors live and Ultimate still fully disables.
@simonsmh

Copy link
Copy Markdown
Owner

Hi @fredtanxo — thanks for this PR! It has been merged as-is (squash: 255e07e). The diagnosis was spot on: enable_thinking: false alone really was contradicted by is_reasoning: true, and the clamp snapping "off" back up to a concrete level for models without a disabled branch was a nice catch.

Before cutting a release, I ran your change against the real Qoder gateway (a reasoning-bait prompt, thinking high as a control vs off). Sharing one edge case that turned up, since you flagged it yourself in the PR notes:

  • Models with a disabled branch (e.g. Ultimate): perfect. The three signals go out together and the response comes back with zero reasoning content. ✅
  • Models without one (e.g. Sonus): the upstream rejects reasoning_effort: "none" with a 400 — Unsupported value: 'none' is not supported with the 'gpt-6-astra-2026-09-03' model. Supported values are: 'low', 'medium', 'high', 'xhigh', and 'max'. Because the omitted-reasoning path lands in the same branch, those models would have failed on any request that doesn't carry a concrete level. (For the record, Kimi-K3 and GLM-5.3 accept "none" and do stop reasoning — it's specifically the strict-enum upstreams that error.)

I've landed a small follow-up on main (030a1fb) that keeps your behavior intact except for this case: when the catalog positively shows there is no disabled branch (an effort-based thinking_config without disabled, or thinkingLevelMap pinning off to null), the effort signal is omitted while enable_thinking: false and is_reasoning: false still go out. With no catalog signal either way, "none" is still sent, as in your PR. Verified live afterwards: Sonus no longer errors, Ultimate still fully disables. The trade-off is the same one qodercli makes — for those strict upstreams there is no real "off" to offer, so thinking-off stays best-effort there rather than breaking the request.

Two regression tests seeding the model cache for both catalog shapes were added alongside; the suite is at 367 passing and CI is green. This will ship in v0.6.3.

Thanks again for the careful reverse-engineering and the thorough PR description — it made the live verification straightforward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants