Skip to content

common: set Muse Glimmer thinking tags in common_chat_params - #27475

Open
paralin wants to merge 1 commit into
ggml-org:masterfrom
paralin:muse-glimmer-thinking-tags
Open

common: set Muse Glimmer thinking tags in common_chat_params#27475
paralin wants to merge 1 commit into
ggml-org:masterfrom
paralin:muse-glimmer-thinking-tags

Conversation

@paralin

@paralin paralin commented Aug 21, 2026

Copy link
Copy Markdown

Overview

Set thinking_start_tag and thinking_end_tags for muse glimmer.

Fixes thinking / reasoning budget not being applied properly.

Additional information

   data.thinking_start_tag = " to=self<|message|>";
   data.thinking_end_tags  = { "<|eom|>" };

After the change, /v1/chat/completions with reasoning_budget_tokens: 32 and max_tokens: 384:

  • finish_reason=stop (not length)
  • reasoning contains the continuation message set in reasoning-budget-message

Requirements

@paralin
paralin requested a review from a team as a code owner August 21, 2026 08:11
@paralin paralin changed the title model: set Muse Glimmer thinking tags in metadata common: set Muse Glimmer thinking tags in common_chat_params Aug 21, 2026
@ggml-gh-bot

ggml-gh-bot Bot commented Aug 21, 2026

Copy link
Copy Markdown

Hi @paralin, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

  • PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.

@ggml-gh-bot ggml-gh-bot Bot added the draft PR will be changed to draft by github-actions bot label Aug 21, 2026
@github-actions
github-actions Bot marked this pull request as draft August 21, 2026 08:23
@github-actions github-actions Bot removed the draft PR will be changed to draft by github-actions bot label Aug 21, 2026
@pwilkin
pwilkin marked this pull request as ready for review August 21, 2026 08:34

@pwilkin pwilkin left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, @aldehir wanna take a peek?

Set thinking_start_tag and thinking_end_tags for muse glimmer.

Fixes thinking / reasoning budget not being applied properly.

```
   data.thinking_start_tag = " to=self<|message|>";
   data.thinking_end_tags  = { "<|eom|>" };
```

After the change, /v1/chat/completions with reasoning_budget_tokens: 32 and max_tokens: 384:

 - finish_reason=stop (not length)
 - reasoning contains the continuation message set in reasoning-budget-message

Signed-off-by: Christian Stewart <christian@aperture.us>
@paralin
paralin force-pushed the muse-glimmer-thinking-tags branch from 87e1e3a to 79e3448 Compare August 21, 2026 08:36
@paralin

paralin commented Aug 21, 2026

Copy link
Copy Markdown
Author

@pwilkin My only question is if I should have defined it as a const string THINK_START? like in common_chat_params_init_cohere2moe - but this seems more straightforward

@paralin

paralin commented Aug 21, 2026

Copy link
Copy Markdown
Author

Double-checked and it works, before the commit, it would not respect reasoning budget, now:

  • finish_reason=stop
  • reasoning contains the budget continuation
  • final content is the inspection plan
  • 356 completion tokens, not a 384-token reasoning stall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants