Skip to content

feat(api): add unified Anthropic Messages protocol adapter - #5389

Open
m199369309 wants to merge 2 commits into
xorbitsai:mainfrom
m199369309:feat/anthropic-messages-adapter
Open

feat(api): add unified Anthropic Messages protocol adapter#5389
m199369309 wants to merge 2 commits into
xorbitsai:mainfrom
m199369309:feat/anthropic-messages-adapter

Conversation

@m199369309

Copy link
Copy Markdown
Collaborator

Summary

  • add a dependency-free Anthropic Messages protocol adapter for request, response, error, and SSE conversion
  • expose both the standard /v1/messages endpoint and the backward-compatible /anthropic/v1/messages endpoint
  • translate Anthropic system/text/tool-use/tool-result inputs into the existing OpenAI-compatible chat backend contract
  • translate text, reasoning, tool calls, finish reasons, usage, and streaming events back to Anthropic format
  • accept x-api-key as an authentication carrier only on Anthropic Messages routes

Protocol behavior

  • /v1/messages requires anthropic-version: 2023-06-01
  • /anthropic/v1/messages remains compatible with existing clients that omit the version header
  • text and tool blocks are supported
  • unsupported multimodal and extended-thinking history blocks return Anthropic-shaped invalid_request_error responses
  • streamed content blocks are emitted sequentially, including fragmented or interleaved OpenAI tool-call arguments

Relationship to #5051

#5051 contains model/chat-template-specific compatibility work for Qwen tool flows. This PR keeps that backend concern separate and implements the public Anthropic protocol boundary consistently for physical models. It does not include the Qwen template normalization from #5051.

A follow-up PR will reuse this adapter to route Anthropic Messages requests through Token Router virtual models.

Tests

  • pytest -q xinference/api/tests/test_anthropic_protocol.py xinference/api/tests/test_anthropic_auth.py xinference/api/tests/test_anthropic_messages.py
  • pytest -q xinference/core/tests/test_restful_api.py -k anthropic
  • pre-commit run --files xinference/api/protocols/__init__.py xinference/api/protocols/anthropic.py xinference/api/restful_api.py xinference/api/routers/llm.py xinference/api/oauth2/advanced/auth_service.py xinference/api/tests/test_anthropic_protocol.py xinference/api/tests/test_anthropic_auth.py xinference/api/tests/test_anthropic_messages.py

@XprobeBot XprobeBot added this to the v3.x milestone Aug 19, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for the Anthropic Messages protocol in Xinference, allowing clients to interact with models using Anthropic-compatible endpoints. Key changes include request parsing, response mapping, and streaming event conversion between OpenAI and Anthropic formats, along with support for x-api-key authentication. The review feedback highlights several critical improvements: explicitly closing the inner generator in anthropic_stream_events to prevent active serve count leaks, gracefully handling malformed tool argument JSON in openai_to_anthropic instead of raising a 500 error, and using .endswith() for path checks in auth_service.py and restful_api.py to ensure compatibility when deployed behind reverse proxies or sub-paths.

Comment thread xinference/api/protocols/anthropic.py Outdated
Comment thread xinference/api/protocols/anthropic.py Outdated
Comment thread xinference/api/oauth2/advanced/auth_service.py Outdated
Comment thread xinference/api/restful_api.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants