Skip to content

qwen3_coder non-streaming drops the text between tool calls #89

Open

Description

@gdevenyi

FunctionCallParser.parse_non_stream keeps only the text before the first <tool_call> (stripped) plus the text after the last </tool_call>. Prose between two calls disappears from non-streaming content, while streaming emits it. Streaming's own whitespace rule (suppress_ws) also depends on chunking: a whitespace-only gap after a call is dropped only when it arrives as a separate chunk. An empty <tool_call></tool_call> block is hidden by streaming but left in non-streaming content.

from freetoken.server.function_call_parser import FunctionCallParser
tools = [{"type": "function", "function": {"name": "read", "parameters": {"type": "object", "properties": {"p": {"type": "string"}}}}}]
blk = lambda v: f"<tool_call>\n<function=read>\n<parameter=p>\n{v}\n</parameter>\n</function>\n</tool_call>"
r = FunctionCallParser(tools, "qwen3_coder").parse_non_stream("Hi" + blk("a") + "\nThen more.\n" + blk("b"))
print(repr(r.normal_text), len(r.calls))   # 'Hi' 2  (streaming shows 'Hi\nThen more.\n')

Fixed on 5080deploy by 7339156 (merged in 1542d52); stream/non-stream parity tests are in tests/server/test_streaming_model_matrix.py.

Activity

  1. gdevenyi commented on Oct 16, 2026

    @gdevenyi
    OwnerAuthor

    Upstream: FlashML-org#713

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions