Skip to content

fix(local): disable thinking on Ollama's OpenAI-compat endpoint (hybrid reasoning models) - #358

Open
alekseikondratenko wants to merge 2 commits into
JerryZLiu:mainfrom
alekseikondratenko:fix/ollama-disable-thinking-on-hybrid-models
Open

alekseikondratenko wants to merge 2 commits into
JerryZLiu:mainfrom
alekseikondratenko:fix/ollama-disable-thinking-on-hybrid-models

Conversation

@alekseikondratenko

Copy link
Copy Markdown

Problem

Local-engine analysis silently degrades ~5-10x when the configured Ollama model is a hybrid reasoning model (any of the Qwen3.5 family, and other recent releases that ship with thinking enabled).

Dayflow talks to Ollama through the OpenAI-compatible /v1/chat/completions endpoint, and on that endpoint Ollama enables the thinking stream by default for these models. There is no model-level or server-level way to turn it off on the Ollama side:

Since ChatRequest is a fixed struct, users have no workaround short of switching models.

Measured on an M4 MacBook Air (24 GB), qwen3.5:9b, real Dayflow screenshots:

Request Completion tokens Wall time per frame
Dayflow's current request shape 602-603 (mostly reasoning) 57-105 s
Same + reasoning_effort: "none" 33-72 ~25 s

At 15 sampled frames per batch, that is the difference between a batch finishing in ~6 minutes and a batch taking longer than the 15-minute window it covers - the analysis queue falls behind permanently and the machine stays hot.

Fix

Send reasoning_effort: "none" when (and only when) the local engine is Ollama. Ollama's OpenAI-compat layer honors this field (verified against Ollama 0.32.14); non-thinking models such as the recommended qwen3-vl:4b preset simply ignore it.

The field is an optional on ChatRequest that stays nil for LM Studio and custom endpoints, and JSONEncoder never encodes it in that case - so those request bodies are byte-identical to today's. LM Studio has its own per-model thinking controls, and some OpenAI-compatible providers reject "none", which is why the field is gated to the Ollama engine rather than sent unconditionally.

Testing

  • Verified reasoning_effort: "none" disables the reasoning stream and answer quality is unchanged, via direct /v1/chat/completions calls against Ollama 0.32.14 with qwen3.5:9b on real Dayflow screenshot batches (numbers above).
  • Verified plain-JSON encoding: with the field nil, the encoded body is unchanged from the current one.
  • Note: I authored this patch on a machine without Xcode, so it has not been compiled locally - the change is confined to one struct field and one guarded assignment in makeChatURLRequest. Happy to adjust if CI or a maintainer build turns anything up.

🤖 Generated with Claude Code

alekseikondratenko and others added 2 commits August 17, 2026 16:33
Hybrid reasoning models (Qwen3.5 family) emit a full thinking stream by
default on Ollama's /v1/chat/completions, and Ollama has no model-level
or server-level way to turn it off. Each describe_frame call then burns
500-900 reasoning tokens before the 1-2 sentence answer, taking a batch
from ~6 minutes to 15+ minutes on Apple Silicon - the analysis queue
never catches up.

Send reasoning_effort: "none" when the engine is Ollama. Non-thinking
models ignore the field; nil is never encoded for LM Studio or custom
endpoints, which have their own thinking controls and in some cases
reject "none".

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verified LM Studio 0.4.21 honors reasoning_effort: "none" for hybrid
models (Qwen3.5-9B MLX, 26s/70 tokens vs 120s/1800+ thinking tokens),
while its template kwargs are not honored via the server API - so the
same guard applies to both local engines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alekseikondratenko

Copy link
Copy Markdown
Author

Updated: extended the same guard to the LM Studio engine after verifying LM Studio 0.4.21 also honors reasoning_effort: "none" (Qwen3.5-9B MLX: 26s / 70 completion tokens with the field vs 120s / 1800+ thinking tokens without; chat_template_kwargs is not honored via the server API, so the request field is the only client-side lever there as well). Custom endpoints remain untouched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant