[bot] PromptTokensDetails missing image_tokens and text_tokens from OpenAI Responses API input_token_details

[braintrust-bot]

Summary

PromptTokensDetails in src/types.rs does not have image_tokens or text_tokens fields. The OpenAI Responses API's input_token_details object includes both fields, making it impossible to capture per-modality token cost breakdown for vision/multimodal calls — even after the key alias fix in #68 is applied.

What is missing

The OpenAI Responses API usage.input_token_details object has four breakdown fields:

{
  "usage": {
    "input_tokens": 428,
    "input_token_details": {
      "text_tokens": 228,
      "audio_tokens": 0,
      "image_tokens": 200,
      "cached_tokens": 0
    },
    "output_tokens": 356,
    "total_tokens": 784
  }
}

The current PromptTokensDetails struct (src/types.rs:509-518) only covers three of these:

pub struct PromptTokensDetails {
    audio_tokens: Option<u32>,       // ✓ present
    cached_tokens: Option<u32>,      // ✓ present
    cache_creation_tokens: Option<u32>, // ✓ present (Anthropic-specific)
    // image_tokens: missing          // ✗ image input tokens
    // text_tokens: missing           // ✗ text input tokens
}

parse_prompt_tokens_details in src/extractors.rs reads from the details object using explicit field names (.get("audio_tokens"), .get("cached_tokens"), .get("cache_creation_tokens")). Any field not explicitly fetched is silently dropped.

Why image_tokens matters

For vision/multimodal calls with the Responses API (e.g. GPT-4o with image inputs), image_tokens is the only way to see the image-specific token cost. A single high-detail image can cost 765+ tokens. Without image_tokens in PromptTokensDetails, users cannot:

• See the image vs. text cost split for multimodal prompts in the Braintrust dashboard
• Compare image token usage across model versions or detail settings
• Debug unexpected token cost spikes from image-heavy prompts

Relationship to #68

Issue #68 adds the missing key alias input_token_details (singular) so parse_prompt_tokens_details is called with the correct key. This issue is orthogonal: even after #68 lands, image_tokens and text_tokens will still be silently dropped because the struct fields do not exist. Both fixes are needed.

CompletionTokensDetails gap (image generation)

For image-generating models (gpt-image-1 via the Responses API), output_token_details can also return image_tokens and text_tokens for the output side:

{
  "output_tokens_details": {
    "image_tokens": 1500,
    "text_tokens": 0
  }
}

CompletionTokensDetails (src/types.rs:546-557) has audio_tokens, reasoning_tokens, accepted_prediction_tokens, rejected_prediction_tokens — but no image_tokens or text_tokens. Image generation token costs are therefore completely invisible.

Braintrust docs status

unclear — Braintrust documents capturing token usage for OpenAI calls. The Responses API specifically is not mentioned in the Rust SDK docs, and image_tokens is not called out as a captured metric for any SDK. Braintrust's proxy does understand multimodal cost, so adding image_tokens to spans would allow cost attribution to work correctly.

• https://www.braintrust.dev/docs/integrations/ai-providers/openai

Upstream sources

• OpenAI Responses API response object — usage.input_token_details with text_tokens, image_tokens, audio_tokens, cached_tokens: https://platform.openai.com/docs/api-reference/responses/object
• OpenAI token counting guide (image tokens, detail levels): https://platform.openai.com/docs/guides/token-counting
• OpenAI Responses API overview (multimodal input, usage fields): https://platform.openai.com/docs/guides/responses
• Community discussion on Responses API image token usage: https://community.openai.com/t/responses-api-image-generation-token-usage/1277438

Local files inspected

• src/types.rs:509-543 — PromptTokensDetails struct; has audio_tokens, cached_tokens, cache_creation_tokens; no image_tokens or text_tokens
• src/types.rs:546-588 — CompletionTokensDetails struct; has audio_tokens, reasoning_tokens, accepted_prediction_tokens, rejected_prediction_tokens; no image_tokens or text_tokens
• src/extractors.rs:190-213 — parse_prompt_tokens_details reads only audio_tokens, cached_tokens, cache_creation_tokens from the details object; unknown fields are silently dropped
• src/extractors.rs:216-243 — parse_completion_tokens_details reads only audio_tokens, reasoning_tokens, accepted_prediction_tokens, rejected_prediction_tokens; no image_tokens
• src/stream.rs:764-789 — local PromptTokensDetails/CompletionTokensDetails structs for streaming also lack image_tokens and text_tokens
• Full codebase grep for image_tokens, text_tokens — zero results