Skip to content

feat(ops): restructure the weekly digest around three production windows - #1958

Merged
modelmirror merged 2 commits into
stagingfrom
feat/weekly-digest-produced-windows
Sep 21, 2026
Merged

modelmirror merged 2 commits into
stagingfrom
feat/weekly-digest-produced-windows

Conversation

@modelmirror

Copy link
Copy Markdown
Collaborator

The weekly performance digest reorganised around what was produced, over three windows, with the boards left to the website.

What changed

The digest's blocks are now, in order:

  1. Produced this week — unchanged: window bounds in the heading, cells and events and measured spend over one set of usage.json records, the role × stage table. The spend-backstop paragraph no longer closes it.
  2. Produced this month — the same shape over the spend backstop's own trailing window (spend.window_days in config/tracking.yaml, 30 today), so the census and the verdict beside it are one window over one walk of the ledger; the backstop paragraph closes this block.
  3. Produced this term — the same shape from 1 October of the Term the digest falls in (october_term_year), pinned to that midnight rather than counted back in days so the Term's first morning is never dropped; closes with Forward cells scored (<scope>, ledger to date): …, the frozen-process figure that used to be a health question, with the shakedown state still named when the frozen scope has nothing scored in either stratum.
  4. Backtest results — unchanged, last.

The Health questions and Analytics state blocks are removed, with the helpers only they used. WeeklyAnalytics now carries only the three back-test artifacts and the scorer in force, so the digest no longer loads the leaderboard, claim-scores, big-cases or statpack files. cell_census and spend_over gained an optional since instant. The daily ops report (render_markdown) is byte-unchanged.

Docs: docs/pipeline.md (the block list and the vintage rule, now scoped to the back-test block), docs/cli.md, README.md, the ops-report docstring (which also named a flag that does not exist), and two comment lines in run-ops.yml (the header's block list, and a stale claim that the digest carried a stale-label signal; one comment also named a WEEKLY_TICK env that no step sets).

Example, rendered over the committed ledger

## Produced this week (2026-09-14 to 2026-09-21, the 7d before this digest)
**373** cell(s) over **128** event(s), and **$946.04** of measured model spend …
## Produced this month (2026-08-22 to 2026-09-21, the 30d before this digest)
**476** cell(s) over **145** event(s), and **$1,170.33** of measured model spend …
Spend backstop: **$1,170.33** of **$2,500.00** over the trailing 30d window (**47%** consumed, $1,329.67 left, 476 cell(s)) — clear …
## Produced this term (OT2025, since 2025-10-01 — 355d to 2026-09-21)
**1115** cell(s) over **223** event(s), and **$2,467.92** of measured model spend …
Forward cells scored (frozen, ledger to date): 0 total, no prior snapshot to diff. No frozen-process cells yet — still shakedown.
## Backtest results

Executed checks

  • uv run fedcourts ops-report --digest-out digest.md --generated-at 2026-09-21T08:30:00+00:00 renders the above; the month block and the backstop line read the same 476 cells and $1,170.33 by construction, and a test now pins that equality on a ledger with rows inside and outside the window.
  • The 1 October render is tested directly: OT2026, since 2026-10-01, a 0-day window that keeps the post-midnight cell and drops the 23:00 one from the night before.
  • scripts/gate.sh lint, types, test (4970 passed, 2 pre-existing skips), schemas (no drift; no pydantic model moved), data — all pass. uv run fedcourts process-digest --all byte-identical to origin/staging.

Review

code-reviewer (no blockers; five recommendations, all taken: the ledger-to-date denominator on the rendered line, the docstring synced to the docs, the defusing test re-pointed at the back-test predictor ids since the census labels are enum-valued, the month-equals-backstop assertion, the 1 October parametrize row) and docs-reviewer (one blocker, README's block list, fixed; the ops-report docstring, the two workflow comments and _sourced's rationale also fixed). Not taken: making the window bounds required on ProducedWindow (the fallback is test-only; left for a later tidy).

Touches .github/workflows/run-ops.yml (comments only): waits for the maintainer, per the golden rules. Effect check for the promotion body: the next Monday digest (2026-09-28) opens with ## Produced this week as its first block and carries ## Produced this month and ## Produced this term (OT2025, …) before ## Backtest results, with no ## Health questions or ## Analytics state.

🤖 Generated with Claude Code

modelmirror and others added 2 commits September 21, 2026 19:44
The weekly performance digest drops its Health questions and Analytics
state blocks — the boards are read on the website, and the health bullets
restated the production section — and reports production over three
windows instead: the week, the trailing month, and the October Term to
date. Each block is one shape (cells, events, measured spend, role x stage
table) so the three are read the same way.

The month block's window is the ex-post spend backstop's own, taken from
its config, and the backstop's verdict closes that block: the verdict and
the census printed beside it can then never describe different periods.
The Term block closes with the forward cells scored under the process in
force — a forward cell is minted once at its event and never again, so the
Term is the period that count is worth reading over — stated as a figure,
with the frozen-shakedown state named rather than shown as a bare zero.

`cell_census` and `spend_over` take a `since` cutoff so the Term window can
be pinned to the instant its Term opened: an October Term opens at midnight
on 1 October and the digest renders mid-morning, so a day count would cut
the Term's own first morning out of its own census.

Artifact: the `weekly-digest` issue run-ops opens on the Monday tick.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The weekly performance digest now reads: produced this week, produced this
month (the spend backstop's own trailing window, closing with its verdict),
produced this term (from the Term's 1 October, pinned to that instant rather
than counted back in days, closing with the forward cells scored under the
process in force, stated as a ledger-to-date figure), then the back-test
results unchanged. The health questions and the analytics-state block are
gone: the boards are read on the site, and the health bullets restated the
production block. The ops report, the daily digest and every process-digest
input are untouched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@modelmirror
modelmirror merged commit c360061 into staging Sep 21, 2026
9 checks passed
@modelmirror
modelmirror deleted the feat/weekly-digest-produced-windows branch September 21, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant