Promotion 2026-09-23: staging → main - #1977
Merged
Merged
Conversation
The weekly performance digest drops its Health questions and Analytics state blocks — the boards are read on the website, and the health bullets restated the production section — and reports production over three windows instead: the week, the trailing month, and the October Term to date. Each block is one shape (cells, events, measured spend, role x stage table) so the three are read the same way. The month block's window is the ex-post spend backstop's own, taken from its config, and the backstop's verdict closes that block: the verdict and the census printed beside it can then never describe different periods. The Term block closes with the forward cells scored under the process in force — a forward cell is minted once at its event and never again, so the Term is the period that count is worth reading over — stated as a figure, with the frozen-shakedown state named rather than shown as a bare zero. `cell_census` and `spend_over` take a `since` cutoff so the Term window can be pinned to the instant its Term opened: an October Term opens at midnight on 1 October and the digest renders mid-morning, so a day count would cut the Term's own first morning out of its own census. Artifact: the `weekly-digest` issue run-ops opens on the Monday tick. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The weekly performance digest now reads: produced this week, produced this month (the spend backstop's own trailing window, closing with its verdict), produced this term (from the Term's 1 October, pinned to that instant rather than counted back in days, closing with the forward cells scored under the process in force, stated as a ledger-to-date figure), then the back-test results unchanged. The health questions and the analytics-state block are gone: the boards are read on the site, and the health bullets restated the production block. The ops report, the daily digest and every process-digest input are untouched. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ed-windows feat(ops): restructure the weekly digest around three production windows
…aign A cert back-test campaign ended with no report when one engine's daily quota ran out: the CLI answered 429 on every retry, the runner classified it transient, exhausted its retry budget and raised `EngineFailed`, and `_replay_item_cells` — which absorbed only `EngineUnavailable` — let it escape, discarding every finished cell with the work root. Three changes, so a campaign always lands `metrics/cert-backtest.json`: - `_replay_item_cells` absorbs `EngineFailed` as one lost cell, recorded under `provenance.lost_cells` with the new `engine-failed` reason and printed as it happens, exactly as an unreadable artifact is. - The runner classifies a terminal quota marker (the `gemini` CLI's `TerminalQuotaError` and the prose it prints with it) as non-transient, so the cell fails after one attempt rather than after the backoff budget, and raises `EngineQuotaExhausted` naming the cause. An ordinary bare-429 throttle keeps its retries. - Once an engine reports its quota spent, the campaign stops attempting that engine's cells and records each under the `quota-exhausted` reason, so the report's counts stay honest while no further backoff is spent on a certain failure. Tracked apart from `unavailable`: an engine whose binary was missing and one whose allowance ran out are different facts, and the report says which. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d-is-a-lost-cell fix(backtest): a failed engine call is one lost cell, and a spent quota stops that engine — never the campaign
chore: sync main into staging
… to 2.1.280 Opus 5.5 needs Claude Code 2.1.280 or newer; the pin moves in step across run-analytics, run-backtest and integration-test. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ci(qp-topic-label): offer claude-opus-5-5; claude CLI pin → 2.1.280
chore: sync main into staging
Read each replayed cell's flags.json back beside its prediction and record it on the report as provenance.disclosures (category, severity and a text rule's exposure-candidate reading per note) with a per-predictor tally. The note text goes to the run log, credential-redacted, never into metrics/. Nothing is excluded: the back-test runs no evaluator, so the reading rules state the direction and the digest line and review PR name the candidates. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Bumps the python group with 3 updates: [boto3](https://github.com/boto/boto3), [pypdf](https://github.com/py-pdf/pypdf) and [ruff](https://github.com/astral-sh/ruff). Updates `boto3` from 1.43.93 to 1.43.98 - [Release notes](https://github.com/boto/boto3/releases) - [Commits](boto/boto3@1.43.93...1.43.98) Updates `pypdf` from 6.18.1 to 6.19.0 - [Release notes](https://github.com/py-pdf/pypdf/releases) - [Changelog](https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md) - [Commits](py-pdf/pypdf@6.18.1...6.19.0) Updates `ruff` from 0.16.7 to 0.16.8 - [Release notes](https://github.com/astral-sh/ruff/releases) - [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md) - [Commits](astral-sh/ruff@0.16.7...0.16.8) --- updated-dependencies: - dependency-name: boto3 dependency-version: 1.43.98 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: python - dependency-name: pypdf dependency-version: 6.19.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: python - dependency-name: ruff dependency-version: 0.16.8 dependency-type: direct:development update-type: version-update:semver-patch dependency-group: python ... Signed-off-by: dependabot[bot] <support@github.com>
… candidates Resolve review findings: a lost cell's exposure candidate is named on the review PR in its own clause, the headline carries direction and unreadable notes, scored is settled against the final board, the rule's vocabulary and its comma-split miss are stated and pinned, and the digest says an unmarked note is not a cleared one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A content-keyed summary lane: summarize-plan derives the predicted cases whose newest record differs from the one their newest committed summary was written from; summarize writes one tool-less Messages API call's validated body per case under harness front matter; summarize.yml runs it behind the review hold with the corpus role and a dedicated API key in separate jobs, publishing a reviewed summaries/refresh PR on main. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reject markup in summary bodies, summarize only supremecourt.gov-shaped records, add a time budget and retry-after handling, skip an unreadable response per case, pin the lane in the workflow invariant tests, and bring the security, data-sources and pipeline docs in step. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Provision every petition's case tree first, then run one worker per engine, each walking all petitions' cells for that engine serially, the lanes at once. Same-engine cells never overlap (codex's per-process auth home), quota and missing-binary state is lane-local, results merge deterministically, and an unexpected fault becomes harness-error losses for that lane's remaining cells. Lane output is engine-prefixed; --workers 1 runs the lanes serially. The run-backtest ceiling comment states the lane wall clock; the cap is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…hon-50ed3bdc63 chore(deps): Bump the python group with 3 updates
…he measured basis Resolve review findings: pump threads are daemons joined with a bound and keep draining on a failed write; a harness-error loss prints an ::error:: annotation after the report is written; the concurrency test drives several petitions per lane; bounded-width, mid-petition and absorption-fault paths are tested; docs and the workflow comment give the claude rate, the hung-lane limit, and soften the log-prefix claim. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The lane shares the per-environment Anthropic key the other Claude lanes use (prod's on main, staging's on a staging run) rather than a dedicated secret; the key still reaches only the generate job's two steps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Resolve the stats review: lanes finish petitions at different speeds, so in one shared tree which peer forecasts sat beside a cell depended on timing. Each lane now runs under <work-dir>/<engine>/, provisioned with identical inputs, which also narrows gemini's admitted directory to gemini's cells. State the wall-clock basis with its n and range, say the harness-fault annotation does not fail the step, and note the bounded stderr drain. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
feat(summaries): plain-language case summaries per snapshot record
…closure feat(backtest): per-cell leakage disclosure record
perf(backtest): run each engine's replay cells in its own lane
chore: sync main into staging
chore: sync main into staging
modelmirror
enabled auto-merge
September 23, 2026 19:03
modelmirror
disabled auto-merge
September 23, 2026 19:03
modelmirror
enabled auto-merge
September 23, 2026 19:03
Collaborator
Author
|
Tagged |
modelmirror
had a problem deploying
to
review
September 23, 2026 19:17 — with
GitHub Actions
Failure
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promotion batch — 2026-09-23
Staging → main, ahead of the 2026-09-28 long conference. No process digest moves: no prompt, predictor/evaluator config,
DEFAULT_MODELS,ENGINE_RETRIEVALor MCP manifest entry is in the batch, andtests/test_process_version.py -k blessedis green at the staging head. No forward cell's execution path changes: liverun-predict/run-evaluatecall their engines through the actions and CLIs directly. The sharedpipeline/runner.pychange (#1959) reaches onlylocal-cascade(engine-smoke) and the back-test.run-ops.ymlcommentsclaude-opus-5-5; Claude CLI pin 2.1.259 → 2.1.280 (analytics, integration-test, back-test)summarize.yml, Claude Sonnet 5, environment's Anthropic key)main-baseadmitssummaries/refresh; SECURITY.mdrun-backtest.ymlcommentsHeld out on purpose (drafts): #1964 (Opus 5.5 predict/evaluate, proc-v9 — after the long-conference release), #1971 (actions group incl.
claude-code-action→ v1.0.231, which bundles CLI 2.1.278 < 2.1.280), #1844 (engine-smoke provisioning — conflicts with this batch; rebase after).Engine evidence: a full green
scenario=allrun at98c4607c1(run 35901466753; all six engine-smoke / engine-actions-smoke legs green — this matters because #1962 moves the Claude CLI the engine-smoke leg runs). The head then moved by a data-only re-sync (seed commit81a0153aa,corpus/corpus.db.refonly), which cannot reach a cell, so the head is carried by anall-offlinerun plus thepromote:skip-engine-smokelabel, per When the head moves after a greenallin docs/pipeline.md. The engine evidence for this batch's code is the98c4607c1run above.Effect checks (report each as it lands)
summarize.ymlappears in Actions once this merges:planreports ~10 planned with a cost range; the run waits onreview; after releasegeneratereports ~10 written with cost from response usage (≈$1–2 total), files in thecase-summariesartifact, no PR. The summaries: plain-language case summaries per snapshot #1968 spot check (~10 summaries against their filings, factual errors and slant) is done on that artifact.Until the spot check passes, reject the daily
mainrun'sreviewhold (03:43 UTC) — it will plan the full ~195-case backfill. Once released after the spot check, it opens asummaries/refreshPR with ~195 files:## Produced this week, then## Produced this monthand## Produced this term (OT2025, …)before## Backtest results, with no## Health questionsor## Analytics state.run-backtestcron (spread, once released):::error::cert-backtestannotation and noharness-errorinprovenance.lost_cells; the report carriesprovenance.disclosuresandprovenance.disclosure_tally. On a day gemini's allowance is spent, gemini's cells readquota-exhaustedinlost_cellsand the job log prints the quota drop line, with the campaign completing.qp-topic-labeldispatch withlabel_model=claude-opus-5-5, the labels artifact's batch recordslabeler: claude-code/claude-opus-5-5. (Rehearse on staging first if wanted.)🤖 Generated with Claude Code