Skip to content

feat(metrics): show engine generation speed - #273

Merged
outofcoffee merged 7 commits into
mainfrom
feat/engine-generation-speed
Oct 28, 2026
Merged

outofcoffee merged 7 commits into
mainfrom
feat/engine-generation-speed

Conversation

@outofcoffee

Copy link
Copy Markdown
Collaborator

Show the rate the engine reports for itself — tokens per second — on every metrics surface.

Closes #271

Summary

  • The daemon parses the instantaneous tg figure from llama.cpp's print_timing lines out of the engine's captured output, and reports it as an optional speed reading on /v1/metrics plus per-tick s points in the retained history. Engines that print no rate, stopped engines and engines quiet for a sampling interval simply omit the figure — absent, not zero.
  • Bar, combined (serve view), gauge, table and dashboard surfaces draw a Speed row: the t/s figure and, where history is drawn, a sparkline whose glyph heights scale to the window's own high watermark and whose points all draw in the terminal's default colour — the 80/90% thresholds belong to utilisation percentages, and a rate has no ceiling to cross.
  • The GPU utilisation series is relabelled: GPU util/GPU N util become GPU/GPU N (memory keeps GPU mem).
  • Backward/forward compatible: older daemons and the cloud control plane omit the fields; older clients ignore them.

Implementation details

  • A lineTee wraps the supervisor's log-file capture sink: raw bytes reach the log file untouched, and complete lines are additionally handed to an injected OnOutput hook. One tee covers both capture paths (stderr and the pty pump's normalised stdout), so the parser reads lines where they are already assembled rather than re-reading the log. The hook must be wired before Supervisor.Start, which builds the sink as it launches.
  • The tracker answers two questions with two clocks: the sampler's tick folds a history point for figures its interval saw (strictly since the previous tick — a point is an observation, never a continuation), and a metrics read reports the current figure only until one interval's worth of quiet ages it out. Engine starts reset tracker, baseline and history together, so one engine's rate never rides the next engine's report.
  • The parser is deliberately narrow: tg\s*= cannot match tg_3s =, so the three-second average and prompt-evaluation rates never read as the current rate.

Spec deltas live in the openspec change engine-generation-speed (to be archived after merge).

@outofcoffee outofcoffee added the enhancement New feature or request label Oct 26, 2026
Parse the instantaneous tg figure from llama.cpp's print_timing lines as
the engine output is captured (a line tee beside the log file), fold it
into the sampler's history, and report it on the metrics API as an
optional current reading and per-tick point. Bar, combined, gauge and
table surfaces draw a Speed row whose sparkline scales to the window's
high watermark in the default colour throughout, and the GPU utilisation
series is relabelled GPU util to GPU.
The real llama.cpp line carries an elapsed-time prefix whose four
dotted groups read as an IPv4 address to the no-cloud-identifiers
check; the parser answers just as well without it.
Review of the rendered serve view: the renamed GPU labels left two dead
columns in the block's label column, and the speed row cut its own
columns rather than sharing the block's.

- statLabelWidth sizes the shared label column to its longest label
  (seven for GPU mem, widening by the GPU index); threaded through the
  sparkline, gauge, active and speed rows and every caller
- the speed figure now starts where the other rows' graphs start and
  the sparkline holds the sparklines' column however many digits the
  figure grows (fixed field spanning the gauge and percentage halves)
- a lapsed figure on an engine that reports one reads 0.00 t/s rather
  than keeping the last burst's rate in front of a watching eye; only
  a never-reporting engine draws no row
- dashBarLineW 25 to 23 so the trailing figure fits the tile
@outofcoffee
outofcoffee force-pushed the feat/engine-generation-speed branch from 3b0b991 to 1b1aa53 Compare October 28, 2026 02:41
The figure drops to 0.00 t/s on a lapse but the sparkline pinned the
last burst to its right edge, so the row contradicted itself: a packed
speed series discarded the quiet samples' positions and the trend's
"now" kept claiming the engine was still generating.

Keep the speed points on the window's time axis instead: one slot per
sample, quiet slots drawn blank, pooled without closing the gaps. Each
speedless tick slides the burst a column toward the past and leaves its
own blank at the row's right edge, agreeing with the figure's zero. A
stopped engine's edge stays legitimately full — its history froze when
the engine did.
Fold the deltas into the main specs — engine-metrics, engine-activity,
daemon-api and remote-metrics-bar-format — and move the change to the
archive.
@outofcoffee
outofcoffee marked this pull request as ready for review October 28, 2026 11:37
@outofcoffee
outofcoffee merged commit 3941834 into main Oct 28, 2026
5 checks passed
@outofcoffee
outofcoffee deleted the feat/engine-generation-speed branch October 28, 2026 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: show engine generation speed

1 participant