Skip to content

hf: use the reasoning parser a card's SGLang command names - #58

Merged
DevelopmentCats merged 2 commits into
mainfrom
devcats/card-reasoning-parser
Nov 5, 2026
Merged

DevelopmentCats merged 2 commits into
mainfrom
devcats/card-reasoning-parser

Conversation

@DevelopmentCats

@DevelopmentCats DevelopmentCats commented Nov 5, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #23.

What I found on the Sparks

spark-1 still has the profile from the original report: vllm-helper new wrote EXTRA_ARGS="--reasoning-parser deepseek_r1" for Qwen/Qwen3-0.6B on 2026-08-25, and the vllm.service journal shows vLLM 0.27.1 serving it with reasoning_parser: 'deepseek_r1' while qwen3 was registered.

The issue's measurement ("reasoning_content is empty") was a field-name mix-up. vLLM 0.27.1 returns reasoning in reasoning and no longer sends reasoning_content. The real failure is narrower. I served Qwen3-0.6B on spark-1 with each parser and sent the same request four ways:

Parser Thinking on Thinking on, streamed Thinking off Thinking off, streamed
deepseek_r1 correct correct content: null, answer in reasoning correct
qwen3 correct correct correct correct

With thinking off, Qwen3's template puts an empty <think></think> in the prompt, so the model's output has no tags, and deepseek_r1 treats untagged output as reasoning.

Fix

Qwen3's hybrid cards name deepseek_r1 in the vllm serve line and qwen3 in the SGLang line. When a card's SGLang command names a reasoning parser that the installed vLLM registers under exactly that name, that parser replaces the vLLM line's, and inspect, new and the panel say so. Nothing is guessed from similar names.

Cards whose lines agree are unchanged. I checked 32 reasoning-model cards: the Qwen3 hybrid models change to qwen3, while the Thinking-2507 and Next-Thinking cards (deepseek_r1 and deepseek-r1) and GLM-4.5/4.7 (glm45 on both lines) stay as they were.

Existing profiles are not rewritten. One can be fixed with llm-helper set <profile> EXTRA_ARGS=....

The issue's other two suggestions already work: --enable-reasoning is dropped with a warning that names it, and inspect shows the result before anything is pulled.

Verification

  • inspect on spark-1 against vLLM 0.27.1, v0.6.0 and this branch side by side:
    • Qwen3-0.6B and Qwen3-8B: deepseek_r1 before, qwen3 with the change, plus a changed: line saying why.
    • Qwen3-30B-A3B-Thinking-2507 and GLM-4.5: identical output from both builds.
  • The live serving test above, with a memory watchdog. GLM on spark-1 stayed healthy (/health 200 before and after).
  • Unit tests for reading the SGLang parser from a card (including the real Qwen3-0.6B card) and for the replacement rules. Replacing into deepseek_v3 keeps its --default-chat-template-kwargs {"thinking":true}, and replacing away from it drops that default.
  • go vet (Linux and macOS), go test -race ./..., gofmt, prettier, gendocs (no diff), recipediff (unchanged).

Qwen3's hybrid cards name --reasoning-parser deepseek_r1 in their vllm
serve line and qwen3 in their SGLang line. Served with deepseek_r1 on
vLLM 0.27.1, a non-streaming request with enable_thinking false returns
content null and the whole answer in reasoning; qwen3 returns it in
content. With thinking on, and when streaming, both parsers behave the
same. Measured on a DGX Spark with Qwen3-0.6B.

When the SGLang command names a parser the installed vLLM registers
under that exact name, use it in place of the vllm serve line's, and say
so in inspect, new and the panel. Cards whose lines agree, such as the
Thinking-2507 ones (deepseek_r1 and deepseek-r1) and GLM-4.5 (glm45),
are unchanged.

Fixes #23
Copilot AI balanced review requested due to automatic review settings November 5, 2026 05:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Replacing a parser with deepseek_v3 or poolside_v1 omits the required thinking default, potentially making the parser a no-op and leaking raw thinking tags into content.

1 open issue
What changed in this PR

This PR addresses stale reasoning-parser recommendations by using the parser named in a model card’s SGLang command when the installed vLLM registers that exact name.

Changes:

  • Extracts SGLang reasoning-parser hints and applies them during inspection and profile creation.
  • Reports parser replacements in the CLI and web panel without rewriting existing profiles.
  • Adds parser-selection tests, a Qwen3 card fixture, and user documentation.
File Description
internal/​web/​static/​models.js Displays parser-replacement notes.
internal/​vllm/​cardparser.go Selects registered SGLang parser replacements.
internal/​vllm/​cardparser_internal_test.go Tests replacement rules and unchanged cases.
internal/​hf/​testdata/​README_qwen3_hybrid.md Adds a Qwen3 hybrid card fixture.
internal/​hf/​snippet.go Extracts parser names from SGLang commands.
internal/​hf/​snippet_internal_test.go Tests SGLang snippet parsing.
internal/​hf/​hf.go Includes the SGLang parser in detected flags.
internal/​hf/​hf_internal_test.go Tests detection using the card fixture.
internal/​cli/​search.go Prints parser-replacement explanations.
internal/​catalog/​catalog.go Applies replacements and propagates notes.
docs/​models.md Explains parser selection and Qwen3 behavior.
CHANGELOG.md Records the fix and existing-profile guidance.

Review effort: Balanced

Comment thread internal/vllm/cardparser.go Resolved
Replacing the vllm serve line's --reasoning-parser after the snippet was
parsed skipped the --default-chat-template-kwargs {"thinking":true} that
deepseek_v3 and poolside_v1 need. DetectFlags now rebuilds the card's flags
with the SGLang parser substituted, so those defaults follow the parser that
is actually used.
@DevelopmentCats
DevelopmentCats merged commit f6c1697 into main Nov 5, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reasoning-parser detection copies a stale model card line, leaving reasoning_content empty

2 participants