Repository navigation
Grok local scan: signals.json undercounts by 54.5x, and costUsdTicks is a usable real-spend source (divisor verified as 1e10) #3345
Description
Grok local scan: signals.json undercounts by 54.5x, and costUsdTicks is a usable real-spend source (divisor verified as 1e10)
Context
Not a PR proposal. #3135 and #3236 are already rewriting the Grok local scanner and are in review; I'm not opening a competing branch. I measured an independent corpus and got two results worth putting on the record — one corroborating #3135, one that I think argues against a design choice all three open PRs currently share.
Corpus: 51 session directories under ~/.grok/sessions, 934 completed turns, 2026-07-22 → 2026-08-25, models grok-4.6-build and grok-4.5-build.
1. main still reads signals.json — 54.5x undercount
docs/grok.md:115 and GrokLocalSessionScanner.swift on current main aggregate totalTokensBeforeCompaction, contextTokensUsed, and modelsUsed. Those are context-window occupancy snapshots, not per-turn consumption.
| Source | Tokens |
|---|---|
signals.json (totalTokensBeforeCompaction + contextTokensUsed) |
44.9M |
turn_completed in updates.jsonl (934 turns) |
2.45B (2.44B in / 7.9M out) |
54.5x. #3135 reports 653K vs 54.1M (83x) on its own corpus — different ratio, same failure mode, so this is not specific to one usage pattern.
Mechanism, stated explicitly because it explains why the ratio varies: totalTokensBeforeCompaction counts each compaction event's context snapshot once, while billed input is the full context re-sent every turn. My corpus is 97.6% cachedReadTokens (2.38B of 2.44B) — cache reads are real billed input that signals.json has no field to represent. The ratio therefore scales with turns-per-compaction, which is why my 54.5x and #3135's 83x differ while both being correct.
2. costUsdTicks / 1e10 == USD — verified, and it is the only real-spend source
All three open PRs avoid costUsdTicks (#2407: "missing ticks are not estimated"; #3135: models.dev list price labeled an estimate; #3236: token-only, no dollars), and ccusage documents ignoring it because the scale is undocumented. I want to submit evidence that the scale is determinable, because the consequence is material.
Method. Price each turn from the published xAI rates and compare against costUsdTicks / 1e10. Two structural facts must be modeled or the check fails:
- Long-context tiering. At 200k tokens the entire request bills at 2x ($2/$0.50/$6 → $4/$1.00/$12 for 4.6).
- A promotional rate from 2026-08-15, at 17% of list. (xAI shipped Grok 4.6 on 2026-08-12 with a launch promotion for Grok Build users.)
Taking the best of {tier1, tier2} × {list, 17%} per turn:
| Model | turns | median error | within 1% |
|---|---|---|---|
| grok-4.6-build | 645 | 0.0000% | 82.2% (530/645) |
| grok-4.5-build | 286 | 0.0000% | 79.4% (227/286) |
The promotional rate is not a fitted parameter — it falls out of the data as a clean date boundary:
| Date | turns priced at list | turns priced at 17% |
|---|---|---|
| 07-22 → 08-14 (8 active days) | 445 | 0 |
| 08-15 → 08-25 (6 active days) | 9 | 477 |
Zero discount-priced turns before 08-15; 9 list-priced turns after. A fitted constant does not produce that.
A caution on my own result: the 200k tier does not key off inputTokens as recorded. 212 turns with inputTokens >= 200k price at tier 1. So the tier predicate is something else (uncached portion, or per-modelCalls context). This does not affect the conclusion — it means the tier must be read from costUsdTicks rather than recomputed, which is precisely the argument for using the field.
Why this matters for #3135 and #3236
List-price reconstruction cannot see a promotional rate. On my corpus:
| Period | Real spend (costUsdTicks) |
List-price reconstruction | Overstatement |
|---|---|---|---|
| through 08-14 (list-rate period) | $697.56 | $822.92 | 1.18x |
| 08-15 onward (promo period) | $264.82 | $1,762.74 | 6.66x |
| total | $962.38 | $2,585.66 | 2.69x |
A user inside the promo window would see about 6.7x their actual spend. #3135's "list-price estimate, explicitly labeled" framing is honest about being an estimate, but a 6.7x gap is not a rounding artifact — it is the difference between a bill and a sticker price during any promotional period, and xAI runs these at model launches. The residual 1.18x in the list-rate period comes from the tier predicate noted above: reconstructing the 200k tier from inputTokens over-applies the 2x rate.
costUsdTicks already encodes tier and promo. Suggested handling: use it when present, fall back to list price only when absent, and label which one produced each figure. Absence is rare — 2 of 934 turns (0.21%) had costUsdTicks == 0, both early in the corpus.
I'd also gently flag that this cuts against #3236's "do not invent dollar/API-rate estimates for subscription credits" rationale: reading costUsdTicks is not inventing an estimate, it is reading the number the CLI recorded. Whether to display dollars for subscription users stays a product call — but the data is there and it is accurate.
3. Two bounds notes for the scanners under review
costUsdTickscoverage is high: 2/934 turns (0.21%) missing. Add Grok local token cost tracking and main-menu usage chart #2407's "missing ticks are not estimated" stays correct but will rarely trigger on a mid-2026 corpus.- 7 of 51 session directories have no
turn_completedrows at all (aborted/short sessions). Keying offturn_completedpresence rather than directory existence avoids carrying empty entries.
Environment
- CodexBar
mainas of 2026-09-01 (docs/grok.md:115,GrokLocalSessionScanner.swift) - Grok Build: 51 session dirs, 934 turns, 2026-07-22 → 2026-08-25
- macOS 15.6 (Darwin 25.6.0)
Ask
Nothing beyond recording this, and specifically that the costUsdTicks scale question be treated as answered rather than open. Happy to re-run any query against my corpus if it would settle a review question — field distributions, per-version behavior, specific edge-case records. Not planning a competing PR.
Activity
clawsweeper commentedon Sep 1, 2026
Codex review: keeping this open for maintainer follow-up; there is still a little grit to resolve. Reviewed September 1, 2026, 1:08 AM ET / 05:08 UTC.
Summary
Keep open: current main still derives Grok local tokens solely from signals snapshots, while the related open PRs take incompatible approaches to completed-turn logs and reported spend. The reported-tick display policy needs maintainer sign-off before one path can become canonical.
Reproducibility: no. high-confidence local reproduction is available: the source clearly shows the snapshot-only path, but confirming the reported ratio needs representative updates.jsonl records from a Grok corpus.
Ways to help us reproduce this
Maintainer decision needed
Why: This changes a user-visible provider-cost contract and the available open implementations intentionally make different choices; VISION.md requires sign-off for new features and behavior changes affecting provider data.
Next step
Use this evidence during review of the existing open Grok implementations; a competing repair PR should not be dispatched while their product semantics remain unresolved.
Review details
Best possible solution:
Adopt one completed-turn scanner and, if approved, render costUsdTicks as clearly labeled reported CLI spend only when present while keeping subscription-credit quota separate.
Do we have a high-confidence way to reproduce the issue?
No high-confidence local reproduction is available: the source clearly shows the snapshot-only path, but confirming the reported ratio needs representative updates.jsonl records from a Grok corpus.
Is this the best way to solve the issue?
Unclear; completed-turn parsing is the maintainable correction for the token path, but the choice to display reported ticks remains a provider-cost product decision.
AGENTS.md: found and applied where relevant.
Remaining risk / open question:
Codex review notes: model internal, reasoning high; reviewed against 8a732e743564.
Label changes
Label changes:
P2: The current fallback can materially misrepresent Grok local usage, but it is limited to the optional Grok cost surface.impact:other: The report concerns inaccurate local usage and spend presentation, which does not fit a more specific owned impact category.issue-rating: 🐚 platinum hermit: Current issue advisory state selects this label.clawsweeper:needs-live-repro: Current issue advisory state selects this label.clawsweeper:no-new-fix-pr: Current issue advisory state selects this label.clawsweeper:needs-product-decision: Current issue advisory state selects this label.Label justifications:
P2: The current fallback can materially misrepresent Grok local usage, but it is limited to the optional Grok cost surface.impact:other: The report concerns inaccurate local usage and spend presentation, which does not fit a more specific owned impact category.Evidence reviewed
What I checked:
Sources/CodexBarCore/Providers/Grok/GrokLocalSessionScanner.swift:112, 8a732e743564)Sources/CodexBarCore/Providers/Grok/GrokLocalSessionScanner.swift:47, 8a732e743564)docs/grok.md:114, 8a732e743564)CHANGELOG.md:192, 3bfbffdcea58)qa/maturity-scores.yaml)Likely related people:
How this review workflow works
@clawsweeper re-reviewcomments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.@clawsweeper re-reviewor@clawsweeper re-runon an open PR or issue to request a fresh review only.@clawsweeper reviewto request a fresh review only.@clawsweeper autofix,@clawsweeper automerge,@clawsweeper fix ci, or@clawsweeper address review.@clawsweeper explainto ask for more context, or@clawsweeper stopto stop active automation.initH271 commentedon Sep 1, 2026
Re:
clawsweeper:needs-live-repro— here is a self-contained reproduction. I've provided a runnable script rather than a screenshot, because a screenshot of my numbers can't be verified, whereas anyone with a Grok corpus can run this and get their own.Everything below is redacted: session UUIDs and prompt IDs are zeroed, project paths are mapped to
project-a…project-e. I verified the redaction by enumerating every string value in the attached JSON — there are exactly ten, all protocol constants or placeholders. No prompts, paths, or usernames are included, and nochat_history.jsonl/events.jsonlcontent is touched (that is where conversation text lives; these records contain only counters).Verification script
Reads only token counters and
costUsdTicks. Prints no paths or identifiers, so its output is safe to paste.Expected vs actual on my corpus
Expected: the local scanner reports Grok token totals of the same order as actual consumption.
Actual:
mainreports ~44.9M where completed turns account for ~2.45B.Per-session ratios — the part I'd weigh most
A single aggregate can be dismissed as one odd corpus. Every session in mine deviates in the same direction:
Range 45.5x–96.2x across 42 sessions. Note the 0-compaction sessions ratio highest (96.2x) — that rules out compaction as the cause. The mechanism is per-turn context re-send:
contextTokensUsedis a single end-state snapshot, while every turn re-sends (and is billed for) the whole context. My corpus is 97.6%cachedReadTokens— real billed input with no representation insignals.json.Sample records
Three redacted
turn_completedrecords, so the arithmetic can be checked by hand:[ { "timestamp": 1785812739, "method": "_x.ai/session/update", "params": { "sessionId": "00000000-0000-0000-0000-000000000000", "update": { "sessionUpdate": "turn_completed", "prompt_id": "00000000-0000-0000-0000-000000000000", "stop_reason": "end_turn", "usage": { "inputTokens": 183477, "outputTokens": 4424, "totalTokens": 187901, "cachedReadTokens": 153856, "cacheCreationTokens": 0, "reasoningTokens": 2142, "modelCalls": 6, "apiDurationMs": 63611, "costUsdTicks": 1319428000, "modelUsage": { "grok-4.5-build": { "inputTokens": 183477, "outputTokens": 4424, "totalTokens": 187901, "cachedReadTokens": 153856, "cacheCreationTokens": 0, "reasoningTokens": 2142, "modelCalls": 6, "apiDurationMs": 63611, "costUsdTicks": 1319428000 } }, "numTurns": 6 } }, "_meta": { "eventId": "00000000-0000-0000-0000-000000000000-000", "agentTimestampMs": 1785812739144 } } }, { "timestamp": 1785812921, "method": "_x.ai/session/update", "params": { "sessionId": "00000000-0000-0000-0000-000000000000", "update": { "sessionUpdate": "turn_completed", "prompt_id": "00000000-0000-0000-0000-000000000000", "stop_reason": "end_turn", "usage": { "inputTokens": 290945, "outputTokens": 4505, "totalTokens": 295450, "cachedReadTokens": 273664, "cacheCreationTokens": 0, "reasoningTokens": 1108, "modelCalls": 6, "apiDurationMs": 57905, "costUsdTicks": 1436912000, "modelUsage": { "grok-4.5-build": { "inputTokens": 290945, "outputTokens": 4505, "totalTokens": 295450, "cachedReadTokens": 273664, "cacheCreationTokens": 0, "reasoningTokens": 1108, "modelCalls": 6, "apiDurationMs": 57905, "costUsdTicks": 1436912000 } }, "numTurns": 6 } }, "_meta": { "eventId": "00000000-0000-0000-0000-000000000000-000", "agentTimestampMs": 1785812921270 } } }, { "timestamp": 1785813080, "method": "_x.ai/session/update", "params": { "sessionId": "00000000-0000-0000-0000-000000000000", "update": { "sessionUpdate": "turn_completed", "prompt_id": "00000000-0000-0000-0000-000000000000", "stop_reason": "end_turn", "usage": { "inputTokens": 362789, "outputTokens": 6155, "totalTokens": 368944, "cachedReadTokens": 337920, "cacheCreationTokens": 0, "reasoningTokens": 1211, "modelCalls": 5, "apiDurationMs": 88217, "costUsdTicks": 1880440000, "modelUsage": { "grok-4.5-build": { "inputTokens": 362789, "outputTokens": 6155, "totalTokens": 368944, "cachedReadTokens": 337920, "cacheCreationTokens": 0, "reasoningTokens": 1211, "modelCalls": 5, "apiDurationMs": 88217, "costUsdTicks": 1880440000 } }, "numTurns": 5 } }, "_meta": { "eventId": "00000000-0000-0000-0000-000000000000-000", "agentTimestampMs": 1785813080288 } } } ]Taking the first record (
grok-4.5-build, published rate $2.00 / $0.30 / $6.00 per M, tier 1):Exact to six decimal places.
On the stability risk you raised
Agreed, and that is an argument for provenance labelling rather than avoidance — the same caveat applies to
updates.jsonl's schema generally, which all three open PRs already depend on. What I can add is empirical bounds on the current contract:grok-4.5-build,grok-4.6-build) and four distinct rate regimes (standard/long-context × list/promo), median error 0.0000% in each.A displayed figure that reads "reported by Grok CLI" degrades safely if xAI changes the field's meaning — the number stops matching list price and the discrepancy is visible. A reconstructed list-price figure has the opposite failure mode: it looks plausible while being 6.7x wrong during a promo, with nothing to signal it.
One robustness note for the scanners under review
13
turn_completedrows in my corpus carry nousageobject at all (my first parser crashed onKeyError: 'usage'). Distinct from the 2 rows that haveusagewithcostUsdTicks == 0. Worth guarding both cases separately.Happy to run further queries against this corpus if it helps settle the product decision.
olddonkey commentedon Sep 1, 2026
Your divisor result reproduces exactly on an independent corpus, and it corrects a conclusion of mine that is currently load-bearing in #3135. Thank you for measuring it.
I am the author of #3135. On 2026-08-21 I backtested
costUsdTickson 25 turns and concluded thatcostUsdTicks / 1e9was USD, because that quotient came out to exactly1.7 ×the public grok-4.6 card with per-call 200k tiering. I read the 1.7 as an unexplained internal multiplier and treated the field as not directly usable, which is part of why #3135 prices from the public card instead.Re-running the arithmetic on my own corpus against your model:
Both ratios land on four decimals. My "unexplained 1.7×" is your 17% rate seen through a divisor that was off by 10, and my sample is timestamped 2026-08-22 — inside the promotional window your table starts at 08-15. So the two measurements never disagreed; my divisor was wrong, and I am correcting it here rather than leaving it in the record.
The consequence for #3135 is concrete rather than theoretical. Those six turns are exactly what its gated live proof prices, and the number it publishes is the
$2.181282above. Real recorded spend for the same turns is$0.370818. That is 5.9× on a real user's card, which matches the 6.66× your promo-period table reports and is not something an "estimate, not a bill" label carries.I am changing #3135 to read
costUsdTickswhen present and fall back to the public card only when it is absent, labeling which source produced each figure. Two notes from your write-up that I am taking as given, and will credit: the 1e10 divisor, and that the tier predicate must be read from the field rather than recomputed, since your 212 turns atinputTokens >= 200kpriced at tier 1 — my own list-side reconstruction has that same over-application baked in.One point I would not want lost: an owner ruling on 2026-08-21 chose the public card over
costUsdTicksfor Grok, and my incorrect divisor was part of the evidence behind it. That ruling deserves to be revisited on your numbers, not silently overridden, so I am flagging it on #3135 as well.Price Grok turns from the spend the CLI recorded
initH271 commentedon Sep 2, 2026
Reproduced your six turns exactly:
Your read of why the two measurements never disagreed is right, and worth stating plainly for anyone reading later: a 25-turn sample dated 2026-08-22 sits entirely inside the promotional window, so the promo rate and a 10x divisor error are not separable within it. Both show up as one constant. My corpus only resolves them because it straddles 08-15 — 445 list-priced turns before, 477 discounted after. That is a sampling-coverage accident on my side, not better method on my part.
One number worth reconciling, since it looks like a discrepancy and is not. You report 5.9x; my promo-period table says 6.66x:
5.88x is the pure-discount ratio, and your six turns are all tier 1, so that is exactly what they should show. My 6.66x is the discount compounded with over-application of the 200k tier — the same over-application you note is baked into your list-side reconstruction. So the two figures agree; they are measuring the promo alone versus the promo plus the tier predicate error.
That also sharpens the tier point: the two error sources are multiplicative, and only one of them disappears when a promo ends. A list-price path carries the tier error permanently.
On the 2026-08-21 owner ruling — agreed it should be revisited rather than overridden, and thank you for flagging it rather than quietly re-deciding. If it helps that review, the divisor question is separable from the display question: whether
costUsdTicks / 1e10is USD is now an empirical matter (two independent corpora, two model generations, four rate regimes, median error 0.0000%), while whether to surface dollars for subscription users remains entirely a product call that this evidence does not touch. The ruling could be reaffirmed on product grounds without the incorrect divisor propping it up.Offer stands: happy to run queries against my corpus if anything would help settle it — per-version behavior, tier-predicate probes, or specific record shapes.
Price Grok turns from the spend the CLI recorded
Price Grok turns from the spend the CLI recorded