Promotion 2026-09-23 (2): staging → main

.github/prompts/summarize.md

@@ -77,6 +77,29 @@ About **250 words in total**:
   Make no prediction about what the Court will do, give no view on who is
   right, and do not characterize the case as important, significant,
   landmark, closely watched, or the like.
+- **Accurate to the record.** Every statement must be something the record
+  shows.
+  - *Allegations are attributed.* A fact that only one side's filing
+    asserts is that side's account: write "the petition says…", "the
+    respondent answers that…". State as fact only what the lower courts found
+    or what both sides agree on.
+  - *Questions keep their direction.* When you restate a question presented,
+    keep who made the rule, whom it binds, and which way it cuts: a question
+    about whether a court's rule requiring something is valid must not become
+    a question about whether a party may do that thing. Check your
+    restatement against the question's own words before moving on.
+  - *Procedure in the docket's own terms.* Leave routine docket entries out.
+    If one matters to where the case stands, name it as the docket does (a
+    motion to extend a deadline, a waiver of the right to respond) and
+    explain it in passing; do not guess at an entry's purpose.
+  - *A request is not its outcome.* A motion, application or request the
+    docket shows only as filed or submitted has not been granted: say what
+    was asked, not that it happened, unless a later entry records the
+    Court's action on it.
+  - *Dates and counts as the entries show them.* A case distributed in
+    August for a September conference was distributed in August for a
+    September conference, not distributed in September; if a case went to two
+    conferences, say two, or "more than one", not "several".
 - **People.** Name parties only as the caption and filings name them. Where
   the filings refer to someone by initials (a minor, for example), use the
   initials. Add no personal detail — addresses, contact details, health,

.github/workflows/summarize.yml

@@ -301,6 +301,20 @@ jobs:
           done < "$RUNNER_TEMP/cases.txt"
           echo "staged ${staged} case record(s), ${failed} failed" | tee -a "$GITHUB_STEP_SUMMARY"
 
+      # The staged tree leaves this job as a public run artifact, so it may
+      # carry only the Court's own docket JSON and filings. The plan screened
+      # each case's newest snapshot, but staging ran after the review hold and
+      # took whatever was newest by then; this re-applies the screen to what
+      # was actually staged, here where the corpus credentials are, and
+      # removes any case that fails it before the upload below. Unconditional:
+      # the upload must never run on an unchecked tree.
+      - name: Check the staged records before they leave this job
+        run: |
+          set -euo pipefail
+          uv run fedcourts summary-stage-check \
+            --plan "$RUNNER_TEMP/plan/summary-plan.json" \
+            --staged "$RUNNER_TEMP/stage" \
+            --summary "$GITHUB_STEP_SUMMARY"
       - name: Upload the staged records
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:

SECURITY.md

@@ -436,9 +436,9 @@ runbook, [docs/security.md](docs/security.md).
   role, its own engine keys for the pre-promotion integration runs, the
   staging read-write role, the staging telemetry App's client id and
   private key — the Issues-only App the repro leg's rehearsal record is
-  minted from — and the staging copy of the case-summary lane's key. A third,
-  `review`, holds no secret, no role, and no branch policy: its entire content
-  is a required-reviewer rule, and it exists only as the audit-logged hold
+  minted from. A third, `review`, holds no secret and no role and admits
+  `main` and `staging`; beyond that its content is a required-reviewer rule,
+  and it exists only as the audit-logged hold
   between a plan that would spend and the spend — run-predict, run-evaluate,
   run-backtest and summarize all bind it; one environment serves every spend
   hold rather than each minting its own. What each hold covers differs by what

docs/case-summaries.md

@@ -69,6 +69,12 @@ The prompt is `.github/prompts/summarize.md`. Its rules:
   passing. The repository holds no site glossary, so the prompt defines each
   term itself, minimally; a site glossary, once it exists, is the definition to
   align the prompt with.
+- **Accurate to the record.** An allegation only one side's filing makes is
+  attributed to that filing; a question presented is restated without changing
+  who made the rule, whom it binds, or which way it cuts; routine docket
+  entries are left out, and one that matters to the posture is named as the
+  docket names it; a request the docket shows only as filed is not reported
+  as granted; dates and counts are as the entries give them.
 - **People.** Named only as the caption and filings name them; initials stay
   initials; no personal detail beyond the dispute.
 
@@ -243,12 +249,18 @@ shortest GitHub offers, one day. It rides the qp-topic extract's footing —
 supremecourt.gov content only, since the plan and `summarize` both refuse a
 CourtListener REST snapshot ([data-sources.md](data-sources.md)) — and is
 wider than the extract in one way: every stored filing of each planned case
-rather than one section of each petition. A race keeps a small residual: a
-REST snapshot stored between the plan and the stage for the same day is staged
-(and then refused by `summarize`), so it crosses the artifact once. Closing
-the channel outright means encrypting the staged tree to a key only the
-generate job holds, or staging in the same job as generation with step-scoped
-corpus credentials.
+rather than one section of each petition. Staging runs after the review hold
+and provisions whatever snapshot is newest by then, so the plan's screen alone
+would let a REST snapshot stored in between reach the artifact. The stage job
+therefore re-applies it to what it actually staged, before the upload and on
+the side of the job boundary that holds the corpus credentials
+(`summary-stage-check`), as an allowlist: a case crosses only if it was
+planned and its tree holds exactly what provisioning writes — the planned day's
+snapshot in the Court's own shape, `context.json`, and the documents manifest
+with one text file per listed document, each fetched from supremecourt.gov. A
+symlink removes the case rather than being followed, and anything else under
+the stage root is removed. A removed case is reported skipped and planned again
+by the next run.
 
 **The written summaries are a public artifact for a week.** The
 `case-summaries` artifact carries the generated files, after the jail and the

docs/cli.md

@@ -146,6 +146,7 @@ committed), plus the spend ledger.
 | `qp-topics` | Measure a topic labeler's JSONL against the hand reference set and accrue `data/qp-topics/qp-topics.json`: the artifact accumulates, so what is written is the union of the committed artifact and this batch's new rows, with prior entries carried forward unchanged. A row inside the reference set publishes the **hand set's adjudicated label**, never the labeler's — the labeler's call there is measurement input, scored into the agreement rate and discarded, so a flip on a reference case moves this run's rate and can change no published row; every other row is published once, since later batches exclude what is already published, and a second labeling of one stops the run. Each row records the batch that first published it and its `source`, and a per-batch ledger keeps every contributing run's labeler and agreement figures. Every label is validated against the `qp-topic-v0` vocabulary, and the labels joined to the extract and to the reference on `case_id` **and** `docket_number` (a half-matching pair is a mis-join and stops the run; so does a labels file that is not exactly the extract's case set, since a partial run measures a prefix rather than a sample and turns the reported `n` into a membership probe on an outcome-encoding reference set). Reports overall agreement beside the rate a **constant labeler** would score, floor-gated per-label agreement, the constitutional-rights / criminal-law / civil-procedure confusion matrix, and the deterministic shadow rules' disagreement rate — whose *level* is uninterpretable off the reference set, only its movement between runs. What it reports is **agreement with the v0 reference raters, not accuracy** — the pooled rate spans the disclosed two-block frame, and the per-stream split is derived at measurement review ([qp-topic.md](qp-topic.md)). The publication gate takes two conditions, the agreement rate and how much of the reference set the run covered; failing either, the artifact is not written, the measurement prints, and the command exits non-zero. There is no override flag. `--frame-rows` is the QP-bearing frame this batch was cut from, which neither input holds — the extract is one batch of the frame and the artifact is the union of every batch — so the extract job passes its own count in as a plain integer, never the `.batch.json` sidecar (whose Term × fee-class shape stays off the labeling job). It lands on this batch's ledger entry, where the docket pack reads it to measure the labeled share of the frame instead of bounding it; omitted, the entry records none and the cut keeps its bound. A count under the rows the batch labeled is refused, since the batch is drawn from the frame. The `run-analytics` mode that dispatches it is in [pipeline.md](pipeline.md). | `--labels`, `--texts`, `--labeler`, `--out`, `--frame-rows` |
 | `summarize-plan` | The case-summary lane's dry run ([case-summaries.md](case-summaries.md)): which predicted cases are owed a plain-language summary, and at what cost. Eligible cases carry a committed prediction; one is owed a summary when the `record_digest` of its newest corpus record — sha256 over the snapshot payload minus its generation stamp, plus each stored document's `(kind, sha256(text))` — differs from the one its newest committed `summaries/<day>.md` was written from, or it has none. Prints the `SummaryPlan` JSON (`summarize` consumes it) and, with `--report`, the markdown the `review` hold is judged on: eligible, up-to-date, planned, deferred and snapshot-less counts, and an estimated cost range at `summaries.model`'s rate. `--limit` plans at most that many, cases without any summary first. A case whose newest snapshot is a CourtListener REST docket rather than the Court's own docket JSON is counted (`not_live_shaped`) and not planned. Reads the corpus (the casestore under the corpus-split mode); writes nothing under `data/`; calls no model. Refuses (exit 2) a configured model `pricing.MODEL_RATES` cannot price. | `--limit`, `--out`, `--report`, `--report-run-url`, `--corpus-backend` |
 | `summarize` | Write the planned case summaries: one Messages API call per case carrying `.github/prompts/summarize.md` and the case's staged record (`provision-snapshot` output under `--staged`; `record/context.json` is not sent) — no tools, thinking off. A response is written to `data/cases/<court>/<docket>/summaries/<snapshot day>.md`, under harness-written front matter (`CaseSummaryFrontMatter`), only if it ended normally, has exactly the three contract sections in order, runs 120–450 words, opens no paragraph with "Whether", and passes the secret scan; otherwise, and for a call that still fails after bounded retries on 429/5xx/529, the case is reported skipped with its reason. The report counts written and skipped cases and the cost of every call that returned, from response usage. The API key is the environment's Anthropic key, read from the variable `summaries.API_KEY_ENV` names; missing, exit 2. Refuses a plan priced for another model than the configured one. `--budget-minutes` stops starting new cases once that much time has passed and reports the rest as deferred, so a slow run returns inside its caller's timeout with what it wrote; a deferred case stays owed. A retried throttle honours the server's `retry-after`, capped at 60 seconds. Refuses a staged snapshot that is not the Court's own docket JSON. Exits 1 when the plan held cases and none was written. | `--plan`, `--staged`, `--report`, `--budget-minutes` |
+| `summary-stage-check` | Remove every staged case-summary record that may not leave the stage job, before its run-artifact upload. An allowlist: a case is kept only if the plan names it (by its exact docket directory name) and its tree holds exactly what `provision-snapshot` writes — the planned day's snapshot in the Court's own docket-JSON shape, `record/context.json`, and `record/documents/documents.json` with one `<kind>.txt` per listed document, every listed `url` on supremecourt.gov over https. An unplanned case, a CourtListener REST snapshot, a snapshot of another day (the corpus moved after the plan), an unreadable file, an unlisted or off-host document, a symlink (never followed), or any stray file under the stage root is removed and printed as a `::warning::`. A removed case is reported skipped by `summarize` and planned again next run. `--summary` appends the kept/removed count line to a file. | `--plan`, `--staged`, `--summary` |
 | `summary-paths` | The case-summary lane's path filter: print the case summary files (`data/cases/<court>/<docket>/summaries/<YYYY-MM-DD>.md`, added or modified) a `--name-status` change set writes. With `--strict`, its publish jail: exit 1 with an `::error::` per change that is anything else (another path, a deletion, a rename). | `--name-status-file`, `--strict` |
 | `tool-usage` | Roll every committed `retrieval_log.json` into an **offered-vs-called** report: which configured MCP tools were never called, which are used by some engines and not others, and call counts per tool / engine / actor. Reads `data/` only — no corpus, no network — so it runs offline and in the gate; the `run-analytics` mode that dispatches it is in [pipeline.md](pipeline.md). Counting contract: call names normalize to `<server>.<tool>` (engines spell one MCP tool `mcp__x__y` or `mcp_x_y`); engine built-ins (shell, file IO, web search) are counted apart from what the manifest offers; a **code-mode** engine invokes everything from inside a freeform builtin call, and both idioms its program reaches through — the MCP manifest and the engine's own builtins, which is where such a program does most of its work — are lifted out of that source into rows of their own, so the freeform call contributes its own builtin row plus one row per lifted call *site the scan reached* (a site inside a loop is still one row); a lifted manifest row carries the same `mcp__<server>__<tool>` spelling a direct item would and normalizes into the offered denominator identically, while a lifted builtin row names the builtin and so is counted apart from that denominator, like any other builtin — gate on the MCP normalization wherever the question is manifest use, and read such an engine's raw call volume as counting the wrapper beside everything it wrapped; the offered denominator is each log's `mcp_tools` snapshot, falling back to the current manifest's advertised set for logs predating that field. A zero means **never called**, not useless, and the report says which. The same walk publishes four further cuts: per-engine **result observability** — a captured `result_digest` is the only evidence the answer side was recorded at all, and a null covers both an empty result and an uncaptured one, so the rate is honestly two-state and the per-tool dead-end rate is withheld for any engine that never captured a result rather than printed as 100%; cells and calls by **mode, role, and actor**; **calls beside cost** per cell, joined from each log's sibling `usage.json`, where a missing record degrades to a null cost and never to free; and **call volume against Brier**, joined to the gradings of each predicted cell and segmented by engine, mode, and forecast moment with the `n` beside every mean. That last block prints a grade, so it is scoped like the boards — blessed processes only unless `--all-versions` — and it publishes a correlation only per (mode, moment) population and only above the floor pre-declared in code as `tool_usage.TOOL_USAGE_CORRELATION_MIN_CELLS`; there is deliberately no pooled coefficient. What it may be read for: [metrics/README.md](../metrics/README.md). | `--out`, `--markdown-out`, `--all-versions` |
 | `ops-report` | Roll pipeline health, **substance**, spend & cost, **agent signals**, data health, and open issues wearing a `run:*` fan-out label into the ops report Markdown (and optional JSON). Each section renders only once its feed exists: `--previous` backs the substance deltas, `--live-frontier` the watchlist readiness, `--corpus-validation` data health, `--trigger-issues` the stale fan-out labels (markers left behind, since nothing keys on a label and each stage derives its own backlog); `--digest-out` renders the weekly performance digest. What each section reports and how `run-ops` publishes it: [pipeline.md](pipeline.md). | `--runs`, `--json`, `--generated-at`, `--corpus-validation`, `--live-frontier`, `--previous`, `--digest-out`, `--data-health-out`, `--trigger-issues`, `--all-versions` |

docs/security.md

@@ -410,8 +410,11 @@ below; `summarize`'s publish job, which runs on `main` only, is the literal
 deliberate exceptions, by environment. The `approval` jobs of run-predict,
 run-evaluate, run-backtest and summarize declare
 **`review`**, an environment that exists *only* for its required reviewers.
-It carries no secrets, no variables, no role, and no deployment-branch
-policy; each job it gates runs one echo under `permissions: {}`, so the
+It carries no secrets, no variables and no role, and its deployment branches
+are `main` and `staging` — `staging` so that summarize's staging rehearsal
+(the one held lane whose jobs resolve their environment from the ref) passes
+the same hold its production run does;
+each job it gates runs one echo under `permissions: {}`, so the
 environment grants nothing and merely withholds the spend behind it — a
 fan-out's matrix, the back-test's fortnightly replay, or a case-summary
 run — until
@@ -461,8 +464,9 @@ the ~8 known shapes) would pass. Relatedly, never put anything sensitive in a
 on the `staging` environment, whose policy is load-bearing twice over: the
 read-only role's trust names it, and so does the one write-capable role
 outside `prod`); `review`
-deliberately carries no branch policy, since it holds nothing a branch could
-take. A job can read the environment's
+admits `main` and `staging` — it holds nothing a branch could take, so its
+policy only decides which refs may request a spend hold, and a staging
+rehearsal of a held lane must be among them. A job can read the environment's
 secrets only when it runs from `main`, so a workflow authored on a PR branch runs
 **without** the App key, agent tokens, or S3 role: a malicious or prompt-injected
 workflow added in a PR cannot exfiltrate secrets on its own PR run; the change
@@ -1096,7 +1100,12 @@ jobs — the newest snapshot and every stored document's text, after the
 contact-detail scrub. The lane plans only cases whose newest snapshot is the
 Court's own docket JSON, so what it carries is supremecourt.gov content, on
 the footing the qp-topic extract is argued on: the plan refuses a
-CourtListener REST snapshot, and so does `summarize` if one is staged. It
+CourtListener REST snapshot, and the stage job re-checks what it staged and
+removes, before the upload, any case whose tree is not exactly what
+provisioning writes — the planned day's Court docket JSON, `context.json`, and
+manifest-listed supremecourt.gov documents (`summary-stage-check`) — so a
+snapshot that changed after the plan never crosses; `summarize` refuses one as
+well. It
 widens that footing in one way the extract does not: the extract carries one
 section of each petition, while this carries every stored filing of each
 planned case. `case-summaries`, seven days, carries the generated summaries