feat(api): add unified Anthropic Messages protocol adapter - #5389
feat(api): add unified Anthropic Messages protocol adapter#5389m199369309 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces support for the Anthropic Messages protocol in Xinference, allowing clients to interact with models using Anthropic-compatible endpoints. Key changes include request parsing, response mapping, and streaming event conversion between OpenAI and Anthropic formats, along with support for x-api-key authentication. The review feedback highlights several critical improvements: explicitly closing the inner generator in anthropic_stream_events to prevent active serve count leaks, gracefully handling malformed tool argument JSON in openai_to_anthropic instead of raising a 500 error, and using .endswith() for path checks in auth_service.py and restful_api.py to ensure compatibility when deployed behind reverse proxies or sub-paths.
Summary
/v1/messagesendpoint and the backward-compatible/anthropic/v1/messagesendpointx-api-keyas an authentication carrier only on Anthropic Messages routesProtocol behavior
/v1/messagesrequiresanthropic-version: 2023-06-01/anthropic/v1/messagesremains compatible with existing clients that omit the version headerinvalid_request_errorresponsesRelationship to #5051
#5051 contains model/chat-template-specific compatibility work for Qwen tool flows. This PR keeps that backend concern separate and implements the public Anthropic protocol boundary consistently for physical models. It does not include the Qwen template normalization from #5051.
A follow-up PR will reuse this adapter to route Anthropic Messages requests through Token Router virtual models.
Tests
pytest -q xinference/api/tests/test_anthropic_protocol.py xinference/api/tests/test_anthropic_auth.py xinference/api/tests/test_anthropic_messages.pypytest -q xinference/core/tests/test_restful_api.py -k anthropicpre-commit run --files xinference/api/protocols/__init__.py xinference/api/protocols/anthropic.py xinference/api/restful_api.py xinference/api/routers/llm.py xinference/api/oauth2/advanced/auth_service.py xinference/api/tests/test_anthropic_protocol.py xinference/api/tests/test_anthropic_auth.py xinference/api/tests/test_anthropic_messages.py