Skip to content

fix(openai): report prompt cache write tokens in usage - #331

Open
1stRunciter wants to merge 1 commit into
charmbracelet:mainfrom
1stRunciter:fix/openai-cache-write-usage
Open

fix(openai): report prompt cache write tokens in usage#331
1stRunciter wants to merge 1 commit into
charmbracelet:mainfrom
1stRunciter:fix/openai-cache-write-usage

Conversation

@1stRunciter

Copy link
Copy Markdown

Current behavior

OpenAI Responses cache writes remain inside InputTokens, while CacheCreationTokens stays zero.

Expected behavior

Subtract provider-reported cached reads and cache writes from normalized input tokens, and expose writes through the existing CacheCreationTokens field. Normalized input is clamped at zero, while TotalTokens retains the existing raw input-plus-output aggregate.

Compatibility

Absent or zero cache-write values preserve existing behavior. This makes no public API changes.

Path coverage

The shared mapping covers generate, response.completed and response.incomplete streams, successful object generation, both object-stream terminal events, and no-text object-generation errors.

Evidence

OpenAI documents cache_write_tokens and cached_tokens under Responses usage.input_tokens_details in the prompt caching guide.

Case input_tokens cached_tokens cache_write_tokens
cold write 3438 0 3435
warm read 3438 3423 12
below cache minimum 14 0 0

Scope

Responses only. Chat Completions and proxy-provider behavior remain unchanged.

Validation

  • gofumpt -w providers/openai/responses_language_model.go providers/openai/openai_test.go
  • golangci-lint run
  • go test ./... -count=1 -timeout=30m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant