Skip to content

fix: include input_tokens in message_delta events for Anthropic compatibility - #21

Open
dracpet wants to merge 1 commit into
AMAP-ML:mainfrom
dracpet:fix/anthropic-message-delta-input-tokens
Open

fix: include input_tokens in message_delta events for Anthropic compatibility#21
dracpet wants to merge 1 commit into
AMAP-ML:mainfrom
dracpet:fix/anthropic-message-delta-input-tokens

Conversation

@dracpet

@dracpet dracpet commented Apr 27, 2026

Copy link
Copy Markdown

Problem

Some Anthropic-compatible clients (e.g., AgentScope/QwenPaw) expect input_tokens to be present in the message_delta event's usage object. Currently, stream_from_openai_result only includes output_tokens, which causes TypeError: '>' not supported between instances of 'NoneType' and 'int' in frameworks that parse usage.input_tokens from the final delta event.

Solution

Add input_tokens to the usage dict in the message_delta event, matching the value from message_start.

Changed

# Before
"usage": {"output_tokens": usage.get("completion_tokens", 0)},

# After  
"usage": {
    "input_tokens": usage.get("prompt_tokens", 0),
    "output_tokens": usage.get("completion_tokens", 0),
},

Verification

Tested with:

  • ✅ Simple streaming & non-streaming
  • ✅ Tool definitions & tool results
  • ✅ Multi-turn conversations (3 turns)
  • ✅ Large system prompts (skill injection simulation)
  • ✅ Minimal responses (edge case)
  • ✅ Stop sequences
  • ✅ Concurrent requests (3 simultaneous)
  • ✅ AgentScope/QwenPaw integration

All 10 test scenarios pass without regression.

@Upper9527
Upper9527 requested a review from StoneHanaMori April 27, 2026 12:11
@Upper9527

Copy link
Copy Markdown
Collaborator

The compatibility fix still looks relevant, but this branch now conflicts with the substantially updated Anthropic protocol adapter. Please rebase onto current main and add a regression test asserting that the final message_delta.usage contains both input_tokens and output_tokens. Once rebased, this should be a small change to review.

@dracpet
dracpet force-pushed the fix/anthropic-message-delta-input-tokens branch from 32ac797 to 584b54d Compare August 21, 2026 08:44
@dracpet

dracpet commented Aug 21, 2026

Copy link
Copy Markdown
Author

Rebased onto current main (3938f75) and added the regression test.

  • Final message_delta now emits "usage": {**anthropic_usage} — same shape as message_start: input_tokens + output_tokens (+ cache_read_input_tokens when prompt caching is reported).
  • New test test_streaming_final_message_delta_usage_includes_input_and_output_tokens asserts the final message_delta.usage carries both fields.
  • pytest tests/test_anthropic_messages.py → 19 passed, ruff check clean.

Note: the full suite has 17 pre-existing failures in test_responses_native.py (identical on clean main, so unrelated to this diff); left untouched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants