diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index cf9c3715..d0346c77 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -435,6 +435,7 @@ All optional except `workflows_ref` (required, no default) — pass them under | `approve_only_max_runs` | `10` | Re-checks per panel round, read off the re-check sentinel, which names the round it counts for; clamped to `card.RECHECK_HEADS_KEPT` (50), past which the count could not grow. | | `carry_forward_unchanged_diff` | `false` | Opt-in: `dismiss-stale-approval` re-approves the live head before dismissing when the PR's own changes (compare API, merge base → head, hunk line numbers ignored, headings and inter-hunk distances kept) equal the ones the approving round reviewed under a threshold no looser than today's — the stacked-PR rebase, re-merge or retarget. What the round reviewed comes only from the approver's own records: the merge base the approval recorded (`approval_merge_base`, when the compare API and the round's sentinel agree — the sentinel can veto a record, never supply one) or what a carried approval copied forward (`carry_baseline`), and the compare API must show it as an ancestor of the reviewed commit; an approval with neither never carries. Only under `pull_request_target`, only for the approver's latest verdict, failing closed (dismissing as before) on anything it cannot compare (`try_carry_forward`); `mark_card_stale` leaves the card alone while such an approval stands. See [the setup guide](../../docs/callers/cursor-review.md#auto-approve). | | `max_rounds` | `5` | Per-PR review-round cap; `0` → none. Counts the posting identity's consolidated reviews since `needs-human-review` was last removed, except a round whose own decision found the PR head had moved (a push superseded it), at most once per reviewed commit and at most `max_rounds` times; at or over the cap the panel is skipped, the PR is labelled `needs-human-review`, one comment lists the latest round's open findings above the threshold, and the `approve_gate` output is `capped`. Fails open on a read error. See [the setup guide](../../docs/callers/cursor-review.md#round-cap-and-the-approve_gate-output). | +| `fail_closed_round_cap` | `false` | Stop the panel if cap evaluation or handoff fails. Requires a positive `max_rounds`; does not change resets, superseded-round exemptions or concurrency. | | `runs_on` | `'"ubuntu-latest"'` | JSON-encoded `runs-on` (same shape as `cursor-review-auto-label.yml`'s) for the four credential-free heavy jobs only: `diff-size`, `preflight`, the `review` panel and `consolidate`. The gate, ledger, Slack notices and every job holding the App token or `pull-requests: write` stay on GitHub-hosted `ubuntu-latest`. Those four run models with unsandboxed shell over PR code, so use a self-hosted pool only if each job gets a fresh pod (fresh filesystem and `$HOME`), holds no cloud identity or token, and can't reach private networks or the metadata server. The image must be linux/x64 with bash, git, curl, jq, python3, gh, tar and GNU coreutils. | | `approver_environment` | `''` | The caller-repo environment holding `APPROVER_TOKEN` (an environment secret, base-branch-only deployment policy), declared on `post-review`, `dismiss-stale-approval`, `auto-retry`, `auto-rereview`, `approve-only` and `axes-closeout` only (and passed to cursor-approve.yml from `approve-start-early`, which declares it there), and only under `pull_request_target`. Pair with a `pull_request_target` caller — see the [trust model](../../docs/callers/cursor-review.md#trust-model). | | `axes_alongside_panel` | `false` | Start cursor-approve's `start` phase (`approve-start-early`, a nested `uses: ./.github/workflows/cursor-approve.yml` with `approve_gate` `pending`, or `capped` at the cap) right after `round-cap`, and the `axis-correctness` / `axis-conformance` nested calls beside the panel; `axes-closeout` (`card.py closeout`, `!cancelled()`) rewrites the still-pending start card THIS run wrote to `no_decision`/`push_then_relabel` when the round's `approve_gate` is not `pass`, or an axis job did not succeed (`axes_ok` false). The start needs `defer_approval`, runs after `dismiss-stale-approval` (and before `post-review`), and only for a round within the diff-size cap with a passing preflight (or the capped hand-off); cursor-approve's start validates `axes` (a non-empty subset of correctness,conformance) and the axis jobs read its normalized `axes` output. A `pending` card carries no `LANE_MARKER`, so `approve-only` never reads an undecided round as a passed panel. None of the four is in `panel-integrity`'s or `blocking-gate`'s `needs:`. Output `axes_started` = `approve-start-early`'s result; `axes_ok` = every axis job `success` or `skipped` (the caller's decide must skip on `false`: run-global artifact names let a job beside the axes claim `axis-` first). Every job is `if: inputs.axes_alongside_panel && …`. See [the guide](../../docs/callers/cursor-approve.md#axes-alongside-the-panel). | diff --git a/.github/cursor-review/auto-approve.py b/.github/cursor-review/auto-approve.py index a3ea3c76..ee22a146 100644 --- a/.github/cursor-review/auto-approve.py +++ b/.github/cursor-review/auto-approve.py @@ -4217,6 +4217,7 @@ def parse_max_rounds(value) -> int: def cmd_round_cap(args) -> int: + strict = getattr(args, "fail_closed_round_cap", False) set_output("capped", "false") set_output("labelled", "false") # The EFFECTIVE cap, for the workflow's `max_rounds` output: an invalid value @@ -4225,11 +4226,13 @@ def cmd_round_cap(args) -> int: try: max_rounds = parse_max_rounds(args.max_rounds) except ValueError as e: - # Fail OPEN like every other way this step can go wrong: a caller typo - # must not take the whole panel down (diff-size hangs off this job). + # Legacy callers fail open; strict callers stop the panel. print(f"::error::{e} — the round cap is not applied this run.") - return 0 + return 1 if strict else 0 if max_rounds <= 0: + if strict: + print("::error::strict round cap requires a positive max_rounds") + return 1 emit("Round cap: off (`max_rounds: 0`).") return 0 set_output("max_rounds", max_rounds) @@ -4247,6 +4250,8 @@ def cmd_round_cap(args) -> int: set_output("labelled", "true") except (RuntimeError, ValueError) as e: print(f"::warning::Could not read this PR's labels: {e}") + if strict: + return 1 try: since = last_unlabeled_at( _paginate(f"repos/{args.repo}/issues/{args.pr_number}/timeline?per_page=100"), HUMAN_REVIEW_LABEL @@ -4254,7 +4259,10 @@ def cmd_round_cap(args) -> int: rounds = round_reviews(list_reviews(args.repo, args.pr_number), args.poster_login, since, marker, max_rounds) except (RuntimeError, ValueError) as e: - # Fail OPEN: the panel runs exactly as it did before the cap existed. + # Legacy callers fail open; strict callers stop the panel. + if strict: + print(f"::error::Could not count review rounds; stopping the panel: {e}") + return 1 print(f"::warning::Could not count this PR's review rounds, so the round cap is not applied this run: {e}") return 0 set_output("rounds", len(rounds)) @@ -4269,11 +4277,14 @@ def cmd_round_cap(args) -> int: # The label is the hand-off AND the only reset, so it goes on BEFORE the PR # is committed to `capped`. Capped-but-unlabelled would be permanent: every # later trigger recounts, retries the same failing write and skips the panel, - # with no label for a human to remove. So a label that cannot be applied - # fails OPEN, like an unreadable count. + # with no label for a human to remove. Legacy callers fail open on this + # error; fail-closed callers stop and require the label or token to be fixed. try: ensure_label(args.repo, args.pr_number) except RuntimeError as e: + if strict: + print(f"::error::Round cap reached but handoff label failed; stopping the panel: {e}") + return 1 print(f"::warning::Round cap reached ({len(rounds)}/{max_rounds}) but the `{HUMAN_REVIEW_LABEL}` label " f"could not be applied ({e}), so the cap is not applied this run and the panel runs. Create the " f"label in this repo (creating one needs `issues: write`), or check the posting token's " @@ -4726,6 +4737,7 @@ def main() -> int: c.add_argument("--max-rounds", required=True) c.add_argument("--threshold", default="") c.add_argument("--poster-login", required=True) + c.add_argument("--fail-closed-round-cap", action="store_true") e = sub.add_parser("approve-external") e.add_argument("--repo", required=True) e.add_argument("--pr-number", required=True) diff --git a/.github/cursor-review/tests/test_auto_approve.py b/.github/cursor-review/tests/test_auto_approve.py index 20278bb2..95e02993 100644 --- a/.github/cursor-review/tests/test_auto_approve.py +++ b/.github/cursor-review/tests/test_auto_approve.py @@ -1822,11 +1822,56 @@ def test_comment_neutralises_model_paths(self): self.assertNotIn("javascript:", body) +class StrictRoundCapPrerequisiteTest(unittest.TestCase): + def run_prerequisite(self, limit, configured, token, slug): + with open(WORKFLOW_PATH, encoding="utf-8") as f: + workflow = f.read() + step = workflow.split("- name: Validate strict cap prerequisites", 1)[1].split("\n - name:", 1)[0] + script = step.split("run: |\n", 1)[1] + script = "\n".join(line[10:] for line in script.splitlines()) + env = {"PATH": os.environ["PATH"], "MAX_ROUNDS": limit, "BOT_CONFIGURED": configured, + "BOT_TOKEN": token, "BOT_SLUG": slug} + return subprocess.run(["bash", "-e", "-c", script], env=env, capture_output=True).returncode + + def test_no_bot_allows_a_positive_limit(self): + self.assertEqual(self.run_prerequisite("3", "false", "", ""), 0) + + def test_configured_bot_needs_both_token_and_identity(self): + for token, slug in (("", ""), ("token", ""), ("", "bot")): + with self.subTest(token=bool(token), slug=bool(slug)): + self.assertNotEqual(self.run_prerequisite("3", "true", token, slug), 0) + self.assertEqual(self.run_prerequisite("3", "true", "token", "bot"), 0) + + def test_zero_and_negative_limits_are_rejected(self): + for limit in ("0", "-1"): + with self.subTest(limit=limit): + self.assertNotEqual(self.run_prerequisite(limit, "false", "", ""), 0) + def test_count_step_passes_the_opt_in_to_the_real_entrypoint(self): + with open(WORKFLOW_PATH, encoding="utf-8") as f: + workflow = f.read() + step = workflow.split("- name: Count review rounds", 1)[1].split("\n diff-size:", 1)[0] + script = "\n".join(line[10:] for line in step.split("run: |\n", 1)[1].splitlines()) + with tempfile.TemporaryDirectory() as d: + with open(os.path.join(d, "auto-approve.py"), "w", encoding="utf-8") as f: + f.write("import json, os, sys\nopen(os.environ['ARGS'], 'w').write(json.dumps(sys.argv[1:]))\n") + path = os.path.join(d, "args") + env = {"PATH": os.environ["PATH"], "CURSOR_REVIEW_ASSETS": d, "ARGS": path, + "APP_SLUG": "review", "FAIL_CLOSED_ROUND_CAP": "true", "REPO": "o/r", + "PR_NUMBER": "1", "MAX_ROUNDS": "3", "THRESHOLD": ""} + subprocess.run(["bash", "-e", "-c", script], env=env, check=True, capture_output=True) + with open(path, encoding="utf-8") as f: + args = json.load(f) + self.assertIn("--fail-closed-round-cap", args) + self.assertEqual(args[args.index("--max-rounds") + 1], "3") + self.assertEqual(args[args.index("--poster-login") + 1], "review[bot]") + + + class RoundCapCommandTest(unittest.TestCase): """The cap path end to end, with gh stubbed by endpoint.""" def run_cap(self, reviews, max_rounds="5", timeline=(), comments=(), label_exists=True, fail=(), labels=(), - probe_error="HTTP 404"): + probe_error="HTTP 404", strict=False): writes = [] def fake_gh(args, payload=None): @@ -1863,7 +1908,7 @@ def fake_gh(args, payload=None): out = os.path.join(d, "out") open(out, "w").close() args = argparse.Namespace(repo="o/r", pr_number="1", max_rounds=max_rounds, threshold="medium", - poster_login=BOT) + poster_login=BOT, fail_closed_round_cap=strict) with mock.patch.dict(os.environ, {"GITHUB_OUTPUT": out}), \ mock.patch.object(AA, "gh", fake_gh), \ mock.patch.object(AA, "open_thread_ids", lambda *a: {"11", "12"}), \ @@ -1921,6 +1966,30 @@ def test_unreadable_reviews_fail_open(self): rc, out, writes = self.run_cap(self.FIVE, fail=("/reviews",)) self.assertEqual((rc, out["capped"], writes), (0, "false", [])) + def test_strict_cap_allows_two_and_hands_off_at_three(self): + rc, out, _ = self.run_cap(self.FIVE[:2], max_rounds="3", strict=True) + self.assertEqual((rc, out["capped"], out["rounds"]), (0, "false", "2")) + rc, out, _ = self.run_cap(self.FIVE[:3], max_rounds="3", strict=True) + self.assertEqual((rc, out["capped"], out["rounds"]), (0, "true", "3")) + + def test_strict_cap_stops_if_labels_cannot_be_read(self): + rc, _, _ = self.run_cap(self.FIVE[:2], max_rounds="3", fail=("repos/o/r/pulls/1$",), strict=True) + self.assertNotEqual(rc, 0) + + def test_strict_cap_rejects_invalid_limit(self): + for limit in ("2.5", "0", "-1"): + with self.subTest(limit=limit): + rc, _, _ = self.run_cap([], max_rounds=limit, strict=True) + self.assertNotEqual(rc, 0) + + def test_strict_cap_stops_if_review_history_is_unreadable(self): + rc, _, _ = self.run_cap(self.FIVE, max_rounds="3", fail=("/reviews",), strict=True) + self.assertNotEqual(rc, 0) + + def test_strict_cap_stops_if_handoff_label_fails(self): + rc, _, _ = self.run_cap(self.FIVE[:3], max_rounds="3", fail=("issues/1/labels",), strict=True) + self.assertNotEqual(rc, 0) + def test_a_failed_label_write_fails_open(self): # Capped-but-unlabelled would be permanent (no label to remove = no # reset), so a label that cannot be applied runs the panel instead. diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index a5661710..f471751a 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -765,6 +765,14 @@ on: type: number required: false default: 5 + fail_closed_round_cap: + description: >- + Stop the panel if review history or the handoff label cannot be read or + written. Requires a positive max_rounds and the configured posting bot + token when one is configured. Does not change reset or concurrency rules. + type: boolean + required: false + default: false runs_on: description: >- JSON-encoded `runs-on` for the four jobs that hold no write-scoped @@ -1793,7 +1801,8 @@ jobs: # is minted, so this is a write-token job: it checks out NO PR code, only # this repo's assets at the pinned `workflows_ref`, and stays on # GitHub-hosted ubuntu-latest. The logic lives in auto-approve.py's - # `round-cap`, which fails OPEN on a read error (the panel runs). + # `round-cap`, which fails open on a read error by default; callers can opt in + # to fail-closed handling. name: Round cap needs: gate if: needs.gate.outputs.should_run == 'true' && needs.gate.outputs.already_reviewed != 'true' @@ -1891,6 +1900,22 @@ jobs: app-id: ${{ inputs.bot_app_id }} private-key: ${{ secrets.BOT_APP_PRIVATE_KEY }} + - name: Validate strict cap prerequisites + if: ${{ inputs.fail_closed_round_cap }} + env: + MAX_ROUNDS: ${{ inputs.max_rounds }} + BOT_TOKEN: ${{ steps.bot_token.outputs.token }} + BOT_SLUG: ${{ steps.bot_token.outputs.app-slug }} + run: | + if ! python3 -c 'import sys; sys.exit(0 if float(sys.argv[1]) > 0 else 1)' "$MAX_ROUNDS"; then + echo "::error::fail_closed_round_cap requires a positive max_rounds" + exit 1 + fi + if [ "$BOT_CONFIGURED" = true ] && { [ -z "$BOT_TOKEN" ] || [ -z "$BOT_SLUG" ]; }; then + echo "::error::The configured posting bot token could not be minted; cannot count its reviews" + exit 1 + fi + - name: Count review rounds id: cap if: ${{ inputs.max_rounds > 0 }} @@ -1902,16 +1927,19 @@ jobs: APP_SLUG: ${{ steps.bot_token.outputs.app-slug }} MAX_ROUNDS: ${{ inputs.max_rounds }} THRESHOLD: ${{ inputs.approve_max_severity }} + FAIL_CLOSED_ROUND_CAP: ${{ inputs.fail_closed_round_cap }} PR_NUMBER: ${{ github.event.pull_request.number }} REPO: ${{ github.repository }} run: | if [ -n "$APP_SLUG" ]; then LOGIN="${APP_SLUG}[bot]"; else LOGIN="github-actions[bot]"; fi + STRICT_FLAG=() + if [ "$FAIL_CLOSED_ROUND_CAP" = true ]; then STRICT_FLAG=(--fail-closed-round-cap); fi python3 "$CURSOR_REVIEW_ASSETS/auto-approve.py" round-cap \ --repo "$REPO" \ --pr-number "$PR_NUMBER" \ --max-rounds "$MAX_ROUNDS" \ --threshold "$THRESHOLD" \ - --poster-login "$LOGIN" + --poster-login "$LOGIN" "${STRICT_FLAG[@]}" diff-size: name: Diff size check diff --git a/docs/callers/cursor-review.md b/docs/callers/cursor-review.md index 317d30a3..5db89881 100644 --- a/docs/callers/cursor-review.md +++ b/docs/callers/cursor-review.md @@ -148,6 +148,7 @@ pull-requests: write # posting the consolidated review (and, at the round cap, | `approve_only_max_runs` | `10` | Cap on `approve_only_on_push` re-checks per panel round, counted on the status card; a new panel round resets it. At the cap a push is left to a relabel (or `auto_rereview_after_dismissal`). Clamped to `50`, the heads the card's record keeps; unparsable re-checks nothing. | | `carry_forward_unchanged_diff` | `false` | Opt-in, for stacked PRs: when a push or a retarget would dismiss the approver's approval but the PR's **own** changes are identical to the ones a round reviewed (a rebase or re-merge onto a moved base, e.g. a lower PR of the stack changed or landed), re-approve the live head, then dismiss the old approval; the card keeps its approval and no round is scheduled. Identical is the compare API's diff from the merge base the approval recorded to the reviewed commit against the diff from the live base to the live head, file by file, context lines and hunk headings included, hunk line numbers ignored. Anything it cannot vouch for dismisses as before, and an approval that records no merge base (every one an earlier version posted, too) never carries. Only under `pull_request_target`, and only with `approve_max_severity` set and `bot_app_id` posting the rounds (with `defer_approval`, pass cursor-approve that App's `poster_login` too). See [Auto-approve](#auto-approve) for every condition and the [trust model](#trust-model) for what it trusts. | | `max_rounds` | `5` | Cap on review rounds per PR (`0` → no cap). At or over it, no panel runs: the PR is labelled `needs-human-review`, one comment lists the latest round's open findings above the threshold, and the `approve_gate` output is `capped`. A round whose own decision found the PR head had moved (a newer push superseded it) does not count, at most once per reviewed commit and at most `max_rounds` times. Removing the label resets the count. See [round cap](#round-cap-and-the-approve_gate-output). | +| `fail_closed_round_cap` | `false` | Stop the panel if the cap cannot be evaluated or handoff fails. Requires a positive `max_rounds` and, when configured, the posting bot token and identity. This preserves the existing reset, superseded-round exemptions and concurrency behavior; it does not guarantee a lifetime limit on all reviewer executions. | | `runs_on` | `'"ubuntu-latest"'` | JSON-encoded `runs-on` for `diff-size`, `preflight`, the `review` panel and `consolidate` only — the jobs that hold no write credential. Every other job stays on GitHub-hosted `ubuntu-latest`. A self-hosted pool is fine only if it is one-job-per-fresh-pod, identity-free and private-network-isolated (those jobs run models with shell over PR code), on linux/x64 with bash, git, curl, jq, python3, gh, tar and GNU coreutils. Empty falls back to the default, so `${{ vars.CURSOR_REVIEW_RUNS_ON }}` is safe while the variable is unset; e.g. `'["self-hosted", "linux", "x64"]'`. | | `approver_environment` | `''` | The caller-repo environment that holds `APPROVER_TOKEN` (as an environment secret). Declared as `environment:` on the jobs that read the token, only under `pull_request_target`, and on none that checks out PR code. With a deployment-branch policy of the base branch only, the token is unreachable from `pull_request` runs. Use with a `pull_request_target` caller — see the [trust model](#trust-model). Empty = no environment. | | `axes_alongside_panel` | `false` | Only for a caller that runs [cursor-approve](cursor-approve.md): start its `start` phase and the **correctness** and **conformance** axes as soon as the round cap has run, beside the panel, instead of after this workflow — so an approved round no longer waits ~5 minutes for the axes after the panel. The caller keeps its decide job on `approve_gate == 'pass'` (plus the `axes_ok` term the guide gives) and gates its own start and axes on the `axes_started` output. Needs a `pull_request_target` caller, `defer_approval: true`, `approve_max_severity` and `APPROVER_TOKEN`; a round over the diff-size cap or with a failed preflight runs no axis. Every job it adds is skipped while it is `false`. See [Axes alongside the panel](cursor-approve.md#axes-alongside-the-panel). | @@ -1228,8 +1229,9 @@ round with no decision, and one whose head moved during the decision's own post Creating a repo label needs `issues: write`, which the caller's `pull-requests: write` does not grant, so **create `needs-human-review` once by hand** unless the `bot_app_id` App has Issues write. If the label cannot be - applied, the cap fails open for that run (the panel runs) with a warning, - rather than capping a PR it gave no way to reset; + applied, the cap fails open for that run (the panel runs) with a warning by + default. With `fail_closed_round_cap: true`, the cap job fails and the panel + does not run; fix the label or token and retry; - posts **one** comment listing the latest round's open findings above `approve_max_severity` (every open finding when that is empty). A hidden `` marker keeps a re-trigger from posting it @@ -1239,8 +1241,9 @@ round with no decision, and one whose head moved during the decision's own post While the label is on, auto-approve never approves the PR. **Removing the label resets the cap**: only rounds after its most recent removal (read from the issue timeline) count, so a human who has looked at the PR can hand it back to the bot -for another `max_rounds` rounds. A count that cannot be read fails open — the -panel runs, as it did before the cap existed. Under `blocking: true` a capped +for another `max_rounds` rounds. By default, a count that cannot be read fails +open and the panel runs. With `fail_closed_round_cap: true`, the cap job fails +and the panel does not run. Under `blocking: true` a capped head holds **Blocking gate** red, the same as an over-cap diff: no panel looked at it. With `max_rounds` above 0 the gate reads the live label on every event, so resolving the old threads cannot turn it green while `needs-human-review` is