Repository navigation
fix(llm): strip a spurious leading markdown rule from model output #144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -652,6 +652,51 @@ def strip_reasoning(content: str) -> str: | |
| return cleaned | ||
|
|
||
|
|
||
| # Markdown horizontal-rule line - 3+ of -, *, or _ alone on a line. Some | ||
| # community merges open a reply with one as a spurious section divider | ||
| # (observed: an abliterated Gemma 4 merge emitting a lone "---" before the | ||
| # actual reply, or as the entire reply). Never load-bearing at the start of a | ||
| # user-facing message. | ||
| _LEADING_DIVIDER = re.compile(r"^\s*(?:[-*_]\s*){3,}(?:\n|$)") | ||
|
|
||
|
|
||
| def strip_leading_divider(content: str) -> str: | ||
| """ | ||
| Drop a spurious markdown horizontal rule from the start of model output. | ||
|
|
||
| Community merges sometimes open a reply with a lone ``---`` (or ``***`` / | ||
| ``___``) divider line - markdown-structure leakage, never intended as | ||
| content. Strips any such leading divider lines plus the whitespace around | ||
| them. | ||
|
|
||
| Only touches the *start* of the content, so an intentional internal rule | ||
| survives. Model-agnostic and a no-op when the content does not start with | ||
| a divider, so it is safe to apply unconditionally. | ||
| """ | ||
| if not content: | ||
| return content | ||
| cleaned = content | ||
| while True: | ||
| m = _LEADING_DIVIDER.match(cleaned) | ||
| if not m: | ||
| break | ||
| cleaned = cleaned[m.end():] | ||
| cleaned = cleaned.lstrip() | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Preserve clean content when no divider exists. At Line 684, 🤖 Prompt for AI Agents |
||
| if cleaned == content: | ||
| return content | ||
| if not cleaned: | ||
| logger.warning( | ||
| "strip_leading_divider: model output was entirely divider lines, " | ||
| "nothing left after strip (%d chars removed)", len(content), | ||
| ) | ||
| return "" | ||
| logger.info( | ||
| "strip_leading_divider: removed %d chars of leading divider", | ||
| len(content) - len(cleaned), | ||
| ) | ||
| return cleaned | ||
|
|
||
|
|
||
| _PROVIDER_ALIASES: dict[str, str] = { | ||
| "openai_chat_completions_endpoint": "openai-chat-completions-endpoint", | ||
| "openai_codex": "openai-codex", | ||
|
|
@@ -1511,7 +1556,7 @@ async def _do_gemini_completion(): | |
| async def _do_chat_completion(): | ||
| response = await client.chat.completions.create(**payload) | ||
| message = response.choices[0].message | ||
| content = strip_reasoning(message.content or "") | ||
| content = strip_leading_divider(strip_reasoning(message.content or "")) | ||
| tool_calls = _openai_tool_calls(message.tool_calls or []) | ||
| return {"content": content, "tool_calls": tool_calls, "raw": response} | ||
|
|
||
|
|
@@ -1853,7 +1898,7 @@ async def _do_stream_completion(): | |
| args = {} | ||
| tool_calls.append({"id": tc["id"], "name": tc["name"], "arguments": args}) | ||
| return { | ||
| "content": strip_reasoning("".join(content_parts)), | ||
| "content": strip_leading_divider(strip_reasoning("".join(content_parts))), | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift Clean the streamed prefix before delivering text deltas. At Line 1901, cleanup runs only after the stream finishes. The callback at Lines 1852-1858 already forwards raw chunks, and the downstream 🤖 Prompt for AI Agents |
||
| "tool_calls": tool_calls, | ||
| "raw": None, | ||
| } | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Restrict the pattern to actual divider runs.
(?:[-*_]\s*){3,}accepts mixed markers such as-*-, and\s*can cross line breaks. The helper can therefore remove content that is not one of the supported---,***, or___forms. Match one repeated marker with horizontal whitespace only.Proposed pattern
📝 Committable suggestion
🤖 Prompt for AI Agents