Skip to content

fix(gateway): DeepSeek cache-hit tokens billing + disjoint input (rant 2026-08-23T08:20:38) - #128

Merged
argszero merged 1 commit into
mainfrom
fix/cache-billing-deepseek
Aug 23, 2026
Merged

argszero merged 1 commit into
mainfrom
fix/cache-billing-deepseek

Conversation

@argszero

Copy link
Copy Markdown
Owner

Summary

Fixes the P0 cache-billing bug (rant 2026-08-23T08:20:38): DeepSeek's native top-level prompt_cache_hit_tokens spelling was silently dropped by serde, so cache hits were billed at the full cache-miss price (~30x overcharge), and input tokens were not disjointed (double-billing risk).

Changes

  1. StreamUsage (src/sse.rs): added #[serde(default)] prompt_cache_hit_tokens: u32 — DeepSeek's native top-level spelling (prompt_tokens = prompt_cache_hit_tokens + prompt_cache_miss_tokens).
  2. extract_cache_read_tokens: now supports all three spellings with priority DeepSeek → OpenAI → Anthropic (prompt_cache_hit_tokens → prompt_tokens_details.cached_tokens → cache_read_input_tokens).
  3. All 6 record_usage call sites: input is now disjoint — prompt_tokens.saturating_sub(cached) (or .max(0.0) for the f64 paths) — so cached tokens are never billed twice (miss price + hit price).
  4. Downstream forwarding (build_anthropic_usage_json): already disjoints (input_tokens = prompt_tokens - cached - cache_creation) and is now effective since cached is actually extracted for DeepSeek.

Tests

Added 3 spelling tests, each asserting the (input, cached, output) triple:

  • deepseek_cache_hit_tokens_spelling — {prompt_tokens:100, prompt_cache_hit_tokens:90, completion_tokens:50} → (10, 90, 50); also verifies downstream input_tokens:10 + cache_read_input_tokens:90
  • openai_cached_tokens_spelling_disjoint — OpenAI prompt_tokens_details.cached_tokens → (10, 90, 50)
  • anthropic_cache_read_spelling_disjoint — Anthropic cache_read_input_tokens → (10, 90, 50)

Related

Rant 2026-08-23T08:20:38 — 缓存计费 bug(P0,多收 ~30 倍)

Verification

  • cargo test — 145/145 pass (142 baseline + 3 new)
  • cargo fmt --check clean
  • cargo clippy --all-targets -- -D warnings clean

Explicitly NOT changed (per rant)

  • Price table untouched (values match official CNY pricing)
  • No new DB fields/migrations (reuses existing cached_tokens column)
  • No reasoning-token / output-billing changes

…t 2026-08-23T08:20:38)

P0 cache-billing fix:
- StreamUsage: add top-level prompt_cache_hit_tokens (DeepSeek native spelling)
- extract_cache_read_tokens priority: DeepSeek -> OpenAI -> Anthropic
- record_usage input = prompt_tokens - cached (saturating) at all 6 sites
- forward disjoint input_tokens downstream (build_anthropic_usage_json now effective)
- tests: 3 spelling tests (deepseek/openai/anthropic) assert (input, cached, output)
@argszero
argszero merged commit 326d00f into main Aug 23, 2026
1 check passed
@argszero
argszero deleted the fix/cache-billing-deepseek branch August 23, 2026 04:51
@argszero argszero mentioned this pull request Aug 23, 2026
4 tasks done
argszero added a commit that referenced this pull request Aug 23, 2026
…0 / 2026-08-23T08:20:38) (#129)

- Cargo.toml/Cargo.lock 0.7.9 -> 0.7.10
- ui/index.html cache-bust 20260822-9 -> 20260823-1 (5 refs)
- CHANGELOG: v0.7.10 entry (70MB body limit #127 + P0 cache-billing fix #128)

Gates: 145/145 cargo test + fmt + clippy + node --check all green.

Co-authored-by: argszero <argszero@argszerodeMac-mini.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant