Repository navigation
hf: use the reasoning parser a card's SGLang command names - #58
Merged
Merged
Conversation
Qwen3's hybrid cards name --reasoning-parser deepseek_r1 in their vllm serve line and qwen3 in their SGLang line. Served with deepseek_r1 on vLLM 0.27.1, a non-streaming request with enable_thinking false returns content null and the whole answer in reasoning; qwen3 returns it in content. With thinking on, and when streaming, both parsers behave the same. Measured on a DGX Spark with Qwen3-0.6B. When the SGLang command names a parser the installed vLLM registers under that exact name, use it in place of the vllm serve line's, and say so in inspect, new and the panel. Cards whose lines agree, such as the Thinking-2507 ones (deepseek_r1 and deepseek-r1) and GLM-4.5 (glm45), are unchanged. Fixes #23
There was a problem hiding this comment.
🟡 Changes recommended
Replacing a parser with deepseek_v3 or poolside_v1 omits the required thinking default, potentially making the parser a no-op and leaking raw thinking tags into content.
1 open issue
What changed in this PR
This PR addresses stale reasoning-parser recommendations by using the parser named in a model card’s SGLang command when the installed vLLM registers that exact name.
Changes:
- Extracts SGLang reasoning-parser hints and applies them during inspection and profile creation.
- Reports parser replacements in the CLI and web panel without rewriting existing profiles.
- Adds parser-selection tests, a Qwen3 card fixture, and user documentation.
| File | Description |
|---|---|
internal/web/static/models.js |
Displays parser-replacement notes. |
internal/vllm/cardparser.go |
Selects registered SGLang parser replacements. |
internal/vllm/cardparser_internal_test.go |
Tests replacement rules and unchanged cases. |
internal/hf/testdata/README_qwen3_hybrid.md |
Adds a Qwen3 hybrid card fixture. |
internal/hf/snippet.go |
Extracts parser names from SGLang commands. |
internal/hf/snippet_internal_test.go |
Tests SGLang snippet parsing. |
internal/hf/hf.go |
Includes the SGLang parser in detected flags. |
internal/hf/hf_internal_test.go |
Tests detection using the card fixture. |
internal/cli/search.go |
Prints parser-replacement explanations. |
internal/catalog/catalog.go |
Applies replacements and propagates notes. |
docs/models.md |
Explains parser selection and Qwen3 behavior. |
CHANGELOG.md |
Records the fix and existing-profile guidance. |
Review effort: Balanced
Replacing the vllm serve line's --reasoning-parser after the snippet was
parsed skipped the --default-chat-template-kwargs {"thinking":true} that
deepseek_v3 and poolside_v1 need. DetectFlags now rebuilds the card's flags
with the SGLang parser substituted, so those defaults follow the parser that
is actually used.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Fixes #23.
What I found on the Sparks
spark-1 still has the profile from the original report:
vllm-helper newwroteEXTRA_ARGS="--reasoning-parser deepseek_r1"forQwen/Qwen3-0.6Bon 2026-08-25, and the vllm.service journal shows vLLM 0.27.1 serving it withreasoning_parser: 'deepseek_r1'whileqwen3was registered.The issue's measurement ("reasoning_content is empty") was a field-name mix-up. vLLM 0.27.1 returns reasoning in
reasoningand no longer sendsreasoning_content. The real failure is narrower. I served Qwen3-0.6B on spark-1 with each parser and sent the same request four ways:deepseek_r1content: null, answer inreasoningqwen3With thinking off, Qwen3's template puts an empty
<think></think>in the prompt, so the model's output has no tags, anddeepseek_r1treats untagged output as reasoning.Fix
Qwen3's hybrid cards name
deepseek_r1in thevllm serveline andqwen3in the SGLang line. When a card's SGLang command names a reasoning parser that the installed vLLM registers under exactly that name, that parser replaces the vLLM line's, andinspect,newand the panel say so. Nothing is guessed from similar names.Cards whose lines agree are unchanged. I checked 32 reasoning-model cards: the Qwen3 hybrid models change to
qwen3, while the Thinking-2507 and Next-Thinking cards (deepseek_r1anddeepseek-r1) and GLM-4.5/4.7 (glm45on both lines) stay as they were.Existing profiles are not rewritten. One can be fixed with
llm-helper set <profile> EXTRA_ARGS=....The issue's other two suggestions already work:
--enable-reasoningis dropped with a warning that names it, andinspectshows the result before anything is pulled.Verification
inspecton spark-1 against vLLM 0.27.1, v0.6.0 and this branch side by side:deepseek_r1before,qwen3with the change, plus achanged:line saying why./health200 before and after).deepseek_v3keeps its--default-chat-template-kwargs {"thinking":true}, and replacing away from it drops that default.go vet(Linux and macOS),go test -race ./..., gofmt, prettier, gendocs (no diff), recipediff (unchanged).