Count Groq prompt cache hits instead of reporting every request as a miss

[DavertMik]

Problem

Every Groq request was recorded as a full-price cache miss. Across the last ~1200 generations in Langfuse, Groq accounted for 95% of all input tokens (11.7M) with exactly zero cached tokens reported on 537 calls, pinning the overall cache hit rate at 2.26%.

provider	model	calls	input tokens	cached	hit
groq	openai/gpt-oss-120b	257	6,445,271	0	0.0%
groq	openai/gpt-oss-20b	280	5,320,826	0	0.0%
openrouter	openai/gpt-5.6-luna	142	615,048	279,317	45.4%

This was a reporting bug, not a prompt problem. Diffing consecutive tester calls in Langfuse showed a 100% stable prefix (11312/11313, 17706/17707, 38851/38852 shared chars) — the prompts were perfectly cacheable the whole time.

Cause

Groq returns prompt_tokens_details.cached_tokens, but @ai-sdk/groq@4.0.0 parses that field in its Zod response schema (dist/index.js:729,768) and never assigns it. Usage always came back as noCacheTokens = the full prompt.

Downstream, extractCachedTokens() in src/ai/provider.ts:52 reads inputTokenDetails.cacheReadTokens / cachedInputTokens / raw — all undefined — and returns 0. The AI SDK OTEL exporter emitted no cached-token attribute, so Groq spans in Langfuse carried only input_tokens and output_tokens, while OpenRouter spans carried the cache keys.

@ai-sdk/openai does this mapping correctly at its dist/index.js:203; groq simply lacked the equivalent.

Fix

Bump to 4.0.34, which adds convertGroqUsage() mapping cached_tokens → inputTokens.cacheRead. package.json already allowed it under ^4.0; only the lockfile pinned 4.0.0. No source changes needed — the existing extractCachedTokens() reads exactly the key the new version populates.

Verification

Fetch interceptor around generateText, 12 identical 28k-token prompts — raw hits 3/12, SDK-reported 3/12, exact match:

raw.cached_tokens=27904  →  {"noCacheTokens":175,"cacheReadTokens":27904}

End-to-end through the same LangfuseSpanProcessor + NodeSDK + registerTelemetry wiring the app uses, 14 calls → 5 raw hits → 5 generations in Langfuse carrying cache_read.input_tokens: 27904. First time Groq spans have carried that key.

Tests: bun run test:unit 1126 pass / 0 fail — bun test tests/integration 84 pass / 0 fail.

Notes

• Groq's cache is per-node, so the hit rate stays uneven by nature: probes suggest roughly a quarter of calls hit, but each hit covers ~99% of the prompt. This PR makes the number visible; it does not raise it.
• The lockfile also picks up the chalk entry missing from its workspace manifest block. chalk is already declared in package.json on main — this just syncs the record.

🤖 Generated with Claude Code

https://claude.ai/code/session_014xHSmzm2ZczcH2ox3WQAC3