Skip to content

Add Kimi K2 native tool calling for chat.json models - #50

Open
c85199917-sudo wants to merge 3 commits into
sqliteai:mainfrom
c85199917-sudo:kimi-k2-tools
Open

Add Kimi K2 native tool calling for chat.json models#50
c85199917-sudo wants to merge 3 commits into
sqliteai:mainfrom
c85199917-sudo:kimi-k2-tools

Conversation

@c85199917-sudo

Copy link
Copy Markdown

Summary

Add native Kimi K2 tool-calling support for models served through
chat.json.

Kimi K2 tokenizers can carry native tool-call control tokens even when
chat.json itself only describes ordinary conversation turns. This change
discovers those tokens when available and uses them to support
OpenAI-compatible tool declarations, assistant tool calls, tool-result
continuations, and streaming tool-call responses.

Changes

  • discover Kimi K2 native tool markers from the tokenizer
  • render tool declarations in Kimi K2's native format
  • render assistant tool-call and tool-result conversation turns
  • parse generated Kimi K2 tool calls into OpenAI-compatible tool_calls
  • support streaming tool-call deltas
  • return finish_reason: "tool_calls" when appropriate
  • preserve tool refusal for chat.json containers without the required
    native tool markers
  • report conversation + native tools at server startup when the
    capability is detected
  • support DeepSeek expert names in the container verifier

Validation

  • full test suite: 228 tests passed, 2 skipped
  • git diff --check clean
  • tested against the real Kimi K2 WASTE container
  • verified non-streaming native tool-call generation
  • verified OpenAI-compatible tool_calls
  • verified finish_reason: "tool_calls"
  • verified tool-result continuation to a final assistant response

Example live response:

{
  "message": {
    "role": "assistant",
    "content": "I'll check the weather for Paris for you.",
    "tool_calls": [
      {
        "id": "call_0",
        "type": "function",
        "function": {
          "name": "get_weather",
          "arguments": "{\"city\": \"Paris\"}"
        }
      }
    ]
  },
  "finish_reason": "tool_calls"
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant