From 6a3625a805be59f2a5aeeea31e86eb07ba00f4ed Mon Sep 17 00:00:00 2001 From: huang47 <157390+huang47@users.noreply.github.com> Date: Wed, 28 Oct 2026 17:02:16 +0800 Subject: [PATCH] fix(pr-risk): discard reports for changed revisions --- .github/workflows/pr-risk.yml | 3 +- docs/callers/pr-risk.md | 2 +- scripts/pr-risk/README.md | 7 ++- scripts/pr-risk/grade-targets.sh | 44 ++++++++++++---- scripts/pr-risk/tests/test_grade_targets.sh | 56 ++++++++++++++++++++- 5 files changed, 98 insertions(+), 14 deletions(-) diff --git a/.github/workflows/pr-risk.yml b/.github/workflows/pr-risk.yml index ceb173a8..d5f046ad 100644 --- a/.github/workflows/pr-risk.yml +++ b/.github/workflows/pr-risk.yml @@ -367,7 +367,8 @@ on: description: >- Retain grade records as an Actions artifact without changing labels, posting comments, or publishing the optional Check Run. The record is advisory and does not establish - that separate reads used one immutable snapshot. + that separate reads used one immutable snapshot. Overrides use the observed base SHA; + a changed or unreadable final snapshot discards the grade. type: boolean required: false default: false diff --git a/docs/callers/pr-risk.md b/docs/callers/pr-risk.md index 23f870ff..1b65661b 100644 --- a/docs/callers/pr-risk.md +++ b/docs/callers/pr-risk.md @@ -127,7 +127,7 @@ fail the caller's next run at startup. | `wait_for_checks_minutes` | `10` | How long to wait for the rest of the check rollup to settle before labeling (clamped to 25 — what a 30-minute job can spend waiting). `0` labels immediately, expect R2 floors from still-pending checks. | | `repo_map_path` | `.github/risk.json` | Consumer risk-map override, read from the PR **base ref**. Repo-relative — a leading `/` or a `..` segment is refused, not resolved. | | `repo_runbooks_path` | `.github/risk-runbooks.json` | Consumer runbook-registry override, read from the PR **base ref**. | -| `report_only` | `false` | Retain per-target records and results as a 30-day Actions artifact. Suppresses label, sticky comment and optional risk Check Run publication even when other inputs request them. GitHub Actions still creates run/check records, and the workflow retains its declared permissions. The record is advisory: separate reads may race with a base or head update. | +| `report_only` | `false` | Retain per-target records and results as a 30-day Actions artifact. Suppresses label, sticky comment and optional risk Check Run publication even when other inputs request them. GitHub Actions still creates run/check records, and the workflow retains its declared permissions. Policy overrides are read at the observed base SHA; a changed or unreadable final snapshot discards the grade. Separate reads can still race, including a revision changing and changing back, so the record is not readiness evidence. | | `sticky_comment` | `false` | Publish ONE sticky PR comment — a single visible line (the tier, the axis that decided it and that axis's reason), with the formula, the per-axis reasons, the map versions and the caveats folded into a collapsed `
` and a "this grade is wrong" checkbox below it. Created once and updated in place, so N pushes leave one comment — with two bounds worth stating. The marker lookup scans only the newest 10 pages of 100 comments (`STICKY_MAX_PAGES`), so on a PR chatty enough to bury the sticky past ~1,000 comments it reads as absent: the next grade posts a DUPLICATE with `disputed=0`, which can also drop an already-applied `risk-grade-disputed` label while the reviewer's box is still ticked. And the upsert is list-then-create / read-then-PATCH with no atomicity — a `pr_numbers` batch and an event run cannot share a per-PR concurrency group, so two first grades can both see no marker and both post, and a tick landing between the read and the PATCH is overwritten. Ordinary single-run pushes behave exactly as described; the guarantee is weaker than it sounds at the edges. **Defaults to false on purpose:** a bot comment on every PR is a visible behaviour change, so it is a per-consumer decision, and a caller that does not opt in behaves byte-identically to before this input existed. Ticking the checkbox applies `risk-grade-disputed` on the next grade — a machine-maintained mirror, distinct from the human-owned `risk-dispute:*` overrides — and needs no permission beyond the `pull-requests: write` already granted. | | `check_run` | `false` | Publish a Check Run on the PR's head commit carrying the tier and the reason: the immutable, timestamped, commit-attached record a mutable label cannot be. Its conclusion is hardcoded `neutral`, so it can never fail a PR even if a repo later marks it required. **Leaving it false does not excuse the `checks: write` grant** — see "Grant the whole union" above; the grant is validated at startup, the input only decides whether the `publish-check` job runs. **A Check Run also requires `enabled: true`.** `publish-check` is gated on `enabled` as well as on this input, while `grade` deliberately admits a `workflow_dispatch` regardless of it — so on exactly the staged-rollout path the `enabled` row advertises, a manual dispatch against a repo still switched off grades, labels and (with `sticky_comment`) comments, but publishes no Check Run. Evaluating the Check Run before switching the repo on therefore means making `enabled` true for that run — `vars.RISK_CONFIG` = `{"enabled": true}` (no PR) or `enabled: true` in the caller's `with:` — because grading alone will not publish one. | | `check_name` | `PR risk (advisory)` | Name of the published Check Run. Only used when `check_run` is true. **Do not reuse the name of a required check.** Nothing validates this value, and the conclusion is hardcoded `neutral` — which branch protection treats as satisfied — so pointing it at a required check's name lets this advisory run stand in for the real merge gate. Keep it distinct and obviously advisory. | diff --git a/scripts/pr-risk/README.md b/scripts/pr-risk/README.md index f05e0996..ae57855d 100644 --- a/scripts/pr-risk/README.md +++ b/scripts/pr-risk/README.md @@ -23,8 +23,11 @@ Set `report_only: true` on the reusable workflow to retain grade records and per results as a 30-day Actions artifact. It skips risk labels, sticky comments and the optional risk Check Run, even if their options are enabled. The workflow itself still creates GitHub Actions run/check records and retains its declared token permissions. -An ungraded result is unknown, never a low-risk verdict. Separate GitHub reads can -race with changes to the base or head; the artifact is not readiness evidence. +An ungraded result is unknown, never a low-risk verdict. The mode reads overrides +at the observed base SHA and discards a grade if the base or head differs on recheck, +the recheck fails, or the grade names another head. The results record includes the +base SHA. Reads can still race, including a revision changing and changing back; +this is not an atomic snapshot or readiness evidence. ## How a grade is computed diff --git a/scripts/pr-risk/grade-targets.sh b/scripts/pr-risk/grade-targets.sh index 72e48929..f42195b2 100755 --- a/scripts/pr-risk/grade-targets.sh +++ b/scripts/pr-risk/grade-targets.sh @@ -248,25 +248,41 @@ settle_grade() { #