Repository navigation
feat: add a stacked spend-over-time chart to the usage tab - #17159
Conversation
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
|
👋 Hi! This PR is currently in draft status. Workflow runs won't be auto-approved until it's marked as ready for review. When you're ready, click "Ready for review" and the workflows will be approved on the next cycle automatically. |
ca8b4ac to
fdddaf6
Compare
Design Review (Opus 5.5, fork) — ✅ PASSDesign-level review of Design-Verdict: PASS Check that turns run on the KAS relay really write a non-zero [DESIGN-REVIEWED] e0b5a03 Description read[DESCRIPTION-READ] 8220384c2f8357fa62ebf2a83ffcf1abf06b38f605f8ca69459324fe5e397a82 That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
UX Review (Opus 5.5, fork) — 🟡 CONCERNSUX-level review of UX-Verdict: CONCERNS When the window is truncated, the chart draws the dropped days as zero spend, with nothing on the plot to mark them. No blind reader has checked any of the controls. Watch
Suggestions
[UX-REVIEWED] e0b5a03 Description read[DESCRIPTION-READ] 7f7c807d01bfdee3ae1217bfa3d7af5db4688af20a3cd68e7a50943576e92c04 That sha256 names everything this verdict read: the PR description, plus the 24 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5, fork) — 🟡 CONCERNSPremise-level review of First-Principles-Verdict: CONCERNS The Cumulative toggle, on by default, is a second view the Goal never asks for, and no harm is named for it. Not justified as shipped
What this change shipsInventory (10 items) — 9 justifiedIntent: let an operator see what is consuming their credits over 30 days (by channel, agent, model or session start week). ADDITION.
[FIRST-PRINCIPLES-REVIEWED] e0b5a03 Description read[DESCRIPTION-READ] 74ab31dd11cd539e46f1347809feba148fba3eb7caa2a4a950b4ec8f5f1ac33c That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review (fork) — ✅ no blocking findingsReviewed |
GPT 6.1 Review (fork) — ✅ no blocking findingsReviewed Review detailsNo findings. |
0049ad8 to
12cc5ab
Compare
12cc5ab to
7234d36
Compare
540dd5b to
c6f16d2
Compare
c6f16d2 to
f21348b
Compare
|
Resolved. The Integration failure below cleared after #18224 ( Original note (2026-10-08)Maintainers: the one red on this PR is That is #17964 / #18170 (umask 022 on the GitHub-hosted runners fork PRs use; same-repo PRs pass on the self-hosted ones), not this change: the diff touches no spool, autonomy or boot code, and the same test is the only failure in 10 of the last 50 completed Integration jobs. A rerun will not clear it on a hosted runner, so I have not asked for one. I will rebase as soon as a fix for #18170 lands. Everything else on the latest head is green: Fast Gate 15/15, every backend and frontend shard, E2E (the |
f21348b to
b355cf2
Compare
|
This PR adds a new usage_series backend endpoint (487 lines) and a stacked spend-over-time chart on the Usage tab, for +2160 lines in total. I read the description and checked the lane verdicts on b355cf2: GPT and Opus have no findings, and Design, UX and First Principles are all CONCERNS. The exact hold reasons: this is a big new feature and a UX decision, and it is well over the ~800 production-line limit. The open lane items should be answered first. On a backend that records tokens but no credits, the credits-only card shows an empty "No spend recorded yet" card. The first-load error shows only the raw "series unavailable" text with no title and no next step. metric=tokens has no consumer (the only caller hard-codes 'credits'), so First Principles asks to drop it. A maintainer should review the feature and its placement. — crew-pr-reviewer |
b355cf2 to
1a39a48
Compare
|
Acted on all three lane items in
Placement and whether the feature belongs are yours to call; the open-lane items are answered. |
1a39a48 to
ffeef95
Compare
ffeef95 to
f9e9d07
Compare
|
Round 11,
|
Add GET /api/usage/series, a credits-only stacked time series over the per-turn usage shards layered by channel, agent, model, or session start week (top-N plus "other", with an explicit "unattributed" layer for turns that did not record the field), and a "Spend over time" card on the Overview Usage tab that renders it as a d3 stacked area with per-day and cumulative views. The channel comes from the session key, never the row's own surface field, for the reason cost_breakdown documents. The parsed-row cache is bounded on rows and on every retained string; a window past the cap keeps its newest rows and the payload says how many of the oldest were left out. App callers are refused with an audited 404. The plot is a keyboard slider over the day axis, the tooltip lists every layer, and the axis top is rounded so the ticks read cleanly.
iamwhatever
left a comment
There was a problem hiding this comment.
This PR adds a dashboard-only GET /api/usage/series endpoint and a stacked credits-over-time card on the Usage tab, so a user can see what each day's spend is made of (by channel, agent, model or session start week), which the per-day totals cannot show. I read the whole of dashboard/handlers/usage_series.py, TokenStackedAreaChart.tsx, the UsageTab.tsx mount, reportsCredits and the client type on head e0b5a03. Our earlier hold items are all met: the tokens metric is gone, the card mounts only for a credits-billing harness (kiro-cli/KAS), and the first-load error has a title and a Try again button. I tried to break the endpoint. An app token gets a SEL-audited 404, using the same request.get("app") convention as the other handlers. An unknown by is a 400. Rows are capped at 100k with every string field clipped to 256 chars, and a non-finite or oversized credits value, or a sum that overflows, can never reach the JSON as Infinity. A corrupt shard costs only its own day, and complete_from stops at it. GPT and Opus have no findings, Design PASS, checks green, no change request open. The UX and First Principles items left open are advisory and fine as follow-ups: shade the days before complete_from when the window is truncated (this only happens past 100k rows), add a Try again button to the refresh-failed notice, add title={label} to the truncated legend and tooltip names, and decide in a later PR whether Cumulative should stay the default. — crew-pr-reviewer (deep re-check)
Problem / Motivation
Goal: Show where spend went over the last 30 days -- by surface, agent, model, or the week a session started -- as a stacked area chart on the Usage tab.
Today the Usage tab answers "how much did I spend?" with totals and a flat daily series. It cannot answer "what is spending it?" or "is this a new session or an old one that keeps running?": the per-turn usage shards already carry the surface, agent, model and session of every turn, but nothing on the dashboard breaks spend down along those axes or over time.
Why it matters
Spend without attribution is hard to act on. Seeing the stack by surface tells you whether crons or chat sessions dominate; by agent, which crewmate costs the most; by model, how much the default routing is worth; by session start week, whether long-running sessions are still spending weeks after they began. All of that is latent in data the gateway already keeps, so it costs no new collection. When the window holds more turns than the reader retains, the oldest days are the ones undercounted; a chart that says only "the totals undercount" leaves every day suspect, so it must also name the first day it counted in full.
Not a goal
/api/usagealready serves daily tokens per model.What changed (motivation -> approach -> change)
The daily credits series already reads the per-turn shards (
<data home>/usage/tokens/YYYY-MM-DD.jsonl); this is a second read keeping the dimension columns, with bucketing, folding and cohort assignment on the server and only summing, stacking and drawing client-side: aggregating in the browser would re-download tens of thousands of rows per view.Backend.
GET /api/usage/series?by=channel|agent|model|cohort(src/kiro_crew/dashboard/handlers/usage_series.py, registered inroutes/system.py), stacking each row'scredits. It admits rows by the daily series' guard and answers a dense day axis plus layers in stack order: the top seven values largest-first,__other__with its member count,__unattributed__. The channel comes from the row's session key viatelemetry_channel_of, never itssurfacefield (the reasoncost_breakdowndocuments), so the chart shares the Spend panel's vocabulary; the model is canonicalised across provider migrations.cohortis the session's first-seen ISO week inside the window, keyed by its Monday, oldest-first; every accumulation is finite-guarded. Parsed rows are cached on the shard fingerprint, bounded on both axes (row count, string length); the cap applies as each shard streams, newest first, and the payload discloses the shortening (truncated,dropped_rows,complete_from: the oldest day kept whole). Dashboard-only: an app token gets a SEL-audited 404.Frontend. A "Spend over time (credits)" card on the Usage tab (
website/src/pages/overview/TokenStackedAreaChart.tsx), above Session activity, mounted only once a loaded config names a default harness that bills in credits (reportsCreditsinwebsite/src/api/acpBackend.ts); a config read that fails with nothing cached puts anErrorNotice(with the request's report and a Try again button) where the card would stand, since a silent absence reads as "no credits". The title names the unit. A d3stack/areachart with a "Layer by" select (Channel, Agent, Model, Session start week) and a "Cumulative" toggle, on by default; off, the same stack per day. Channel ids show as words; unrecognised session keys are labelled apart from the Other fold. Hover and a keyboard slider read out each day's layers. Changing the dimension keeps the previous stack on screen, dimmed andaria-busywith a "Loading…" cue, until the new answer arrives. At 320 px a long legend label truncates, and the tooltip slides within the plot (left-aligned on the first day, right-aligned on the last), so neither crosses the card's edge. Captions explain the Unattributed and session-start-week layers; the oldest cohort drawn is labelled "Started by" its Sunday, not "Week of", because a cohort is the week a session was first seen here, so that layer also holds sessions that began before the window. A truncated window shows a warning-toned line: the totals undercount, and from which day they are whole. A failed first load shows a titledErrorNotice, the server's text on its own line, with Try again; a failed refresh over a drawn chart shows it above the stack; a zero-spend window shows an empty state, and a failed refresh over it the first-load notice. Preferences persist inlocalStorage;usageSeries(by)inwebsite/src/api/client/telemetry.tsuses the sharedgettransport. 24 strings underwebsite/src/i18n/locales/in every locale and the pseudolocale, with translator context inen.context.json.Backwards compatibility
Compatible: one new route and one new card; no existing endpoint, response shape, config key, or stored format changes, and the
apiclient surface keeps every existing method.Removes nothing: the change is purely additive -- the Usage tab's existing cards, the daily series, and the shard format are untouched.
Tests
Backend (
test/test_usage_series.py, 34 tests; the shard-reading tests run on one frozenManualClockinstalled on both the shard reader and the series module, so no expectation reads the live clock):__other__with its member count; an empty dimension value becomes an explicit__unattributed__layer; both reserved layers are omitted when empty.Infinity; folded and total sums cannot overflow either; a credits figure wider than a double counts zero instead of failing the read.surface: dashboardbuckets as Telegram); a row with no key stays unattributed rather than taking the classifier's "unknown".truncatedwith the dropped count; the shard dated the day before the axis opens (the shard window's cutoff is inclusive) is never read, so it neither spends the budget nor counts as refused; a shard wider than the remaining budget is never held whole (peak live rows are counted while it streams); the cap is silent when the window fits; a dropped count passed to the aggregator is disclosed along withcomplete_from, which is the oldest whole day (a day with no shard counts as whole; an unreadable shard stops it),nullwhen the cap tripped inside the newest shard, and withheld when nothing was dropped.byis a 400; an app token gets 404 and the refusal is written to the SEL audit log; the cohort dimension over the wire.Frontend (
website/src/test/TokenStackedAreaChart.test.tsx, 32 tests):weekMondayOfkeys every weekday, month and year rollovers to the server's Monday andstartedByDaynames the week's Sunday clamped to the axis; prefix sums keep the window total; pointer-to-day mapping clamps at the edges; the axis top rounds to a nice number; channel ids render as words and the unrecognised channel is labelled apart from the fold; stacking is bottom-first against the rounded top; no division by zero on an all-zero window; the default view draws one path per layer with a legend and the Unattributed caption, which is absent when every turn is attributed; a legend item can shrink below its content (min-w-0) with its swatch and amount held, so a 64-character label truncates instead of the card clipping it (fails on the pre-fix component, whose item could not shrink; jsdom has no layout, so the test pins the mechanism as the Drive page's narrow-row test does); the tooltip is capped at the plot's width and slides with the day line, hanging from the plot's right edge on the last day, each amount held (fails on the pre-fix component, which centred an uncapped tooltip on a clamp sized for a 10 rem box); a truncated payload shows, in the warning tone, how many turns were not counted and from which day every turn is counted, falling back to the count alone when no day is; session-start-week layers are labelled "Week of ..." except the oldest drawn, "Started by" its Sunday, also in the slider's value text (fails on the pre-fix component: "Week of Sep 28"), and a window opening on a quiet weekend puts it on the first layer drawn, clamped to the axis's last day (fails when keyed on the window's first week); the caption; other dimensions show none; the server's text sits on its own line under the title on a failed first load and on a failed refresh over a drawn chart, which keeps the stack; the cumulative switch persists and re-stacks without refetching; changing the dimension keeps the previous stack on screen, dimmed and busy, still labelled as its own dimension and with the Loading… live-region cue (which stays mounted and empties afterwards), until the next answer lands (fails on the pre-fix component: the skeleton replaces the chart); the hover tooltip lists all nine layers of a maximal payload; the keyboard slider steps days, clamps at the edges, jumps with Home/End, clears with Escape and reads the day out as its value text; the empty state; a failed refresh over the empty state reads like a failed first load (no "last values" title beside "no spend"); a failed first load names what could not be loaded and its Try again button refetches.UsageTab.spendChartHarness.test.tsx(6 tests): the card mounts for kiro-cli, an unset key and KAS, never for a tokens-or-dollars harness (which also never requests the series), and not on a guess before the config loads -- each absence is asserted only after a probe on the same config query has rendered the answer; the last two fail on the pre-fix tab, which rendered the card unconditionally; a config read that fails renders the notice where the card would stand, with no chart and no series request (fails on the pre-fix tab: no notice, the chart simply absent), holds it through the retry Try again starts (fails onisError) and clears it when the read succeeds; nothing shows while the first read is pending (fails ondata === undefined).acpBackend.test.tspinsreportsCredits(false unloaded, true for kiro-cli/unset/KAS, false for every other harness). The four existing UsageTab suites mock the config and series queries, and the API-client surface pin includes the new method.Manual verification
Ran a local gateway against a data home seeded with generated sample shards (72 sessions over 30 days across six channels, ten agents, three models, with background and some cron turns recording no model). Checked all four layer modes, per-day and cumulative, the hover tooltip at several days, the
Other (n more)fold on the agent view, the Unattributed layer and its caption on the model view, the caption on the session-start-week view, the empty state on a data home with no shards, a 390 px viewport, a 320 px viewport on the agent view with a 64-character agent name (the legend truncates the name; the capture measures every legend item inside the card), the hover tooltip at the first, a middle and the last day, keyboard stepping through the days with the focus ring showing, the loading state before the first answer, a failed first load (title, the server's text under it, Try again), a failed refresh over a drawn chart (the same notice above the stack it keeps showing), the config read failing (the notice and Try again in the card's place, no series request), the "Started by" first layer on the session-start-week view, the dimmed stand-in with its Loading… cue while a newly chosen dimension loads, the light theme on the model view, and that preferences survive a reload; browser console clean. Also checked, on further generated data homes: the per-day tooltip, a channel view with more channels than the chart keeps apart plus sessions whose key shape the classifier does not recognise, a window with more turns than the row cap retains (the caption names the count left out and the first fully counted day, which matches where the drawn area begins), a single day alone past the cap (the caption gives the count with no date), and a failed refresh while the empty state is showing (the titled error and Try again replace the empty line). Confirmed the endpoint answers 400 for an unknown dimension and 404 for an app token, and that withagent.acp_backendset to a non-credits harness the card is absent and no series request is made.Screenshots / video
Dark theme unless noted, 30-day window of generated sample data.
Channel, cumulative (default view):
Channel, per day:
Model view with the Unattributed layer and its caption:
Session start week: the oldest layer is "Started by" its Sunday, the rest "Week of", with the caption:
Hover tooltip, cumulative ("Total to date"):
Hover tooltip, per day ("That day"):
Keyboard focus on the plot (ring, day line and tooltip four days back):
Loading, before the first answer:
The first load failing: a titled notice with the server's text on its own line, and a Try again button under the controls, no chart:
Layer by changed to Agent while that answer is still loading: the channel stack stands in, dimmed, with the Loading… cue beside the controls:
Light theme, model view, with the Unattributed layer:
Channel view with the "Unrecognized sessions" layer beside the "Other (4 more)" fold:
Truncated window: more turns than the chart retains, with the warning naming how many were not counted and from which day every turn is counted:
A refresh failing over a chart already drawn: the "Can't refresh" notice, server text below its title, above the stack it keeps showing:
Truncated window where the cap tripped inside today's shard: no day is whole, so the warning gives the count without a date:
The agent backend setting could not be read: a notice and Try again in the card's place instead of a silent absence:
Empty state:
A failed refresh over the empty state, read like a failed first load (no "last values" title):
Agent view with the "Other" fold, 390 and 320 px viewports
Related Issues
None.
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)Contribution License Agreement
N/A -- no CLA text has been supplied yet.