Skip to content

fix(gateway): UsageCapture.finish() disjoint input + DeepSeek cache-hit spelling (rant 2026-08-23T14:05:02) - #130

Merged
argszero merged 1 commit into
mainfrom
fix/usage-capture-disjoint
Aug 23, 2026
Merged

argszero merged 1 commit into
mainfrom
fix/usage-capture-disjoint

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

Fix incomplete P0 cache-billing fix from PR #128 (rant 2026-08-23T08:20:38): PR #128 only patched src/sse.rs (cross-protocol conversion path), but the same-protocol passthrough path — UsageCapture::finish() in src/gateway.rs — was missed. Per host's rant 2026-08-23T14:05:02, cached tokens were double-counted into the total (observed: per-call uncached 241,776 instead of 112, total ≈2× on dev).

Root cause (gateway.rs openai branch of finish())

  • input was set to the full prompt_tokens (includes cache-hit portion) — not disjoint
  • cached only recognized OpenAI's prompt_tokens_details.cached_tokens spelling — DeepSeek's native top-level prompt_cache_hit_tokens was dropped (which made the miss even worse on DeepSeek responses)

Changes

  1. UsageCapture::finish() openai branch: cached extraction now three-spelling compatible with the same priority as extract_cache_read_tokens in sse.rs — DeepSeek native prompt_cache_hit_tokens → OpenAI prompt_tokens_details.cached_tokens
  2. Disjoint input at return (covers openai + anthropic branches uniformly): (input - cached).max(0.0) — matches the already-fixed logic in sse.rs; max guards underflow
  3. Tests (extended usage_capture_parses_openai_and_anthropic):
    • DeepSeek spelling: {"prompt_tokens":100,"prompt_cache_hit_tokens":90,"completion_tokens":50} → input=10, cached=90
    • OpenAI spelling: prompt_tokens_details.cached_tokens=90 → input=10, cached=90
    • Anthropic: cache_read_input_tokens=70 → input=10, cached=70
    • Original no-cached cases still pass (10-0=10, 80-0=80)

Expected effect (correct behavior, not a bug)

  • Points drop sharply: input billed at ~100 uncached + ~241k cached rate (0.05/1M) instead of 241k full rate
  • Transaction record: 输入(非缓存)≈100, 输入(缓存)≈24万, 总 Token≈24万 — consistent with real EMRG usage

Related

Tests

  • cargo test 145/145 pass (incl. new cached cases)
  • cargo fmt --check clean
  • cargo clippy --all-targets clean

@argszero
argszero merged commit c9a78b2 into main Aug 23, 2026
1 check passed
@argszero
argszero deleted the fix/usage-capture-disjoint branch August 23, 2026 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant