diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 5ce7e255..4ea8f0b9 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -435,6 +435,7 @@ All optional except `workflows_ref` (required, no default) — pass them under | `approve_only_max_runs` | `10` | Re-checks per panel round, read off the re-check sentinel, which names the round it counts for; clamped to `card.RECHECK_HEADS_KEPT` (50), past which the count could not grow. | | `carry_forward_unchanged_diff` | `false` | Opt-in: `dismiss-stale-approval` re-approves the live head before dismissing when the PR's own changes (compare API, merge base → head, hunk line numbers ignored, headings and inter-hunk distances kept) equal the ones the approving round reviewed under a threshold no looser than today's — the stacked-PR rebase, re-merge or retarget. What the round reviewed comes only from the approver's own records: the merge base the approval recorded (`approval_merge_base`, when the compare API and the round's sentinel agree — the sentinel can veto a record, never supply one) or what a carried approval copied forward (`carry_baseline`), and the compare API must show it as an ancestor of the reviewed commit; an approval with neither never carries. Only under `pull_request_target`, only for the approver's latest verdict, failing closed (dismissing as before) on anything it cannot compare (`try_carry_forward`); `mark_card_stale` leaves the card alone while such an approval stands. See [the setup guide](../../docs/callers/cursor-review.md#auto-approve). | | `max_rounds` | `5` | Per-PR review-round cap; `0` → none. Counts the posting identity's consolidated reviews since `needs-human-review` was last removed, except a round whose own decision found the PR head had moved (a push superseded it), at most once per reviewed commit and at most `max_rounds` times; at or over the cap the panel is skipped, the PR is labelled `needs-human-review`, one comment lists the latest round's open findings above the threshold, and the `approve_gate` output is `capped`. Fails open on a read error. See [the setup guide](../../docs/callers/cursor-review.md#round-cap-and-the-approve_gate-output). | +| `preserve_round_count` | `false` | Keep counting prior delivered reviews after the human handoff label is removed. Does not count undelivered panels or serialize runs. | | `count_superseded_rounds` | `false` | Count delivered reviews even when a later decision found the head moved. Does not count undelivered panels or change resets. | | `fail_closed_round_cap` | `false` | Stop the panel if cap evaluation or handoff fails. Requires a positive `max_rounds`; does not change resets, superseded-round exemptions or concurrency. | | `runs_on` | `'"ubuntu-latest"'` | JSON-encoded `runs-on` (same shape as `cursor-review-auto-label.yml`'s) for the four credential-free heavy jobs only: `diff-size`, `preflight`, the `review` panel and `consolidate`. The gate, ledger, Slack notices and every job holding the App token or `pull-requests: write` stay on GitHub-hosted `ubuntu-latest`. Those four run models with unsandboxed shell over PR code, so use a self-hosted pool only if each job gets a fresh pod (fresh filesystem and `$HOME`), holds no cloud identity or token, and can't reach private networks or the metadata server. The image must be linux/x64 with bash, git, curl, jq, python3, gh, tar and GNU coreutils. | diff --git a/.github/cursor-review/auto-approve.py b/.github/cursor-review/auto-approve.py index f4d30755..c30a69cb 100644 --- a/.github/cursor-review/auto-approve.py +++ b/.github/cursor-review/auto-approve.py @@ -58,7 +58,7 @@ ``round-cap`` (in `round-cap`, before the panel starts) Count the consolidated reviews the posting identity has already left on this PR — only those after the most recent removal of ``needs-human-review``, - so removing the label resets the cap. At or over `max_rounds`, label the PR + so removing the label resets the cap by default. At or over `max_rounds`, label the PR ``needs-human-review`` (creating the label if missing), post ONE comment listing the latest round's open findings above the threshold (at most once per cap, keyed on ``ROUND_CAP_MARKER``), and tell the workflow to skip the @@ -270,7 +270,7 @@ TOLERABLE_CELL_STATUS = "error" # The label the round cap applies and decide() refuses to approve over. Its -# removal (an `unlabeled` timeline event) is what resets the round count. +# removal (an `unlabeled` timeline event) resets the round count by default. HUMAN_REVIEW_LABEL = "needs-human-review" # The per-PR veto. cursor-review.yml's `gate` job and its `trigger` concurrency # slot hard-code the same string, so a rename has to land in all three. @@ -280,6 +280,7 @@ # vetoed PR, so nothing else would withdraw an approval already standing. VETO_LABELS = (SKIP_REVIEW_LABEL,) ROUND_CAP_MARKER = "" +PRESERVED_CAP_MARKER = "" # post-review.py opens two bodies with CONSOLIDATED_MARKER that report a round # which reviewed NOTHING — the "Review failed" error review and the # all-panel-cells-failed review (both `delivers=False`). Neither spent a panel's @@ -4076,12 +4077,13 @@ def superseded_by_push(panel: dict, reviews: list, login: str, marker: str) -> s return "" -def cap_comment_posted(comments: list, poster_login: str, since) -> bool: +def cap_comment_posted(comments: list, poster_login: str, since, preserve: bool = False) -> bool: """True when this cap's comment is already on the PR (posted after `since`).""" return any( isinstance(c, dict) and ((c.get("user") or {}).get("login") or "").lower() == poster_login.lower() - and ROUND_CAP_MARKER in (c.get("body") or "") + and (PRESERVED_CAP_MARKER if preserve else ROUND_CAP_MARKER) in (c.get("body") or "") + and (preserve or PRESERVED_CAP_MARKER not in (c.get("body") or "")) and _after(c.get("created_at"), since) for c in comments ) @@ -4108,7 +4110,8 @@ def cap_findings(review_comments: list, open_ids, threshold: str) -> list: return out -def render_cap_comment(rounds: int, max_rounds: int, findings: list, threshold: str, review_url: str) -> str: +def render_cap_comment(rounds: int, max_rounds: int, findings: list, threshold: str, review_url: str, + preserve: bool = False) -> str: render_code_ref = _load_post_review().render_code_ref scope = f"above `{threshold}`" if threshold else "still open" lines = [ @@ -4116,8 +4119,11 @@ def render_cap_comment(rounds: int, max_rounds: int, findings: list, threshold: "### 🛑 Cursor Review — round cap reached", f"This PR has had {rounds} review round(s) (`max_rounds: {max_rounds}`), so no new " f"panel ran and it is labelled `{HUMAN_REVIEW_LABEL}`. Auto-approve will not " - "approve it while the label is on; removing the label resets the round count.", + "approve it while the label is on; " + ("removing the label does not reset the round count." + if preserve else "removing the label resets the round count."), ] + if preserve: + lines.insert(1, PRESERVED_CAP_MARKER) if findings: lines.append(f"\nFindings from the latest round {scope}:") for sev, path, line in findings[:30]: @@ -4256,7 +4262,8 @@ def cmd_round_cap(args) -> int: since = last_unlabeled_at( _paginate(f"repos/{args.repo}/issues/{args.pr_number}/timeline?per_page=100"), HUMAN_REVIEW_LABEL ) - rounds = round_reviews(list_reviews(args.repo, args.pr_number), args.poster_login, since, marker, + preserve = getattr(args, "preserve_round_count", False) + rounds = round_reviews(list_reviews(args.repo, args.pr_number), args.poster_login, None if preserve else since, marker, 0 if getattr(args, "count_superseded_rounds", False) else max_rounds) except (RuntimeError, ValueError) as e: # Legacy callers fail open; strict callers stop the panel. @@ -4274,11 +4281,8 @@ def cmd_round_cap(args) -> int: emit(f"Round cap: {len(rounds)}/{max_rounds} round(s) so far — running the panel.") return 0 - # The label is the hand-off AND the only reset, so it goes on BEFORE the PR - # is committed to `capped`. Capped-but-unlabelled would be permanent: every - # later trigger recounts, retries the same failing write and skips the panel, - # with no label for a human to remove. Legacy callers fail open on this - # error; fail-closed callers stop and require the label or token to be fixed. + # Apply the handoff before recording `capped`. Legacy callers fail open if + # that write fails; fail-closed callers require the label or token to be fixed. try: ensure_label(args.repo, args.pr_number) except RuntimeError as e: @@ -4296,7 +4300,7 @@ def cmd_round_cap(args) -> int: failed = [] try: comments = _paginate(f"repos/{args.repo}/issues/{args.pr_number}/comments?per_page=100") - if cap_comment_posted(comments, args.poster_login, since): + if cap_comment_posted(comments, args.poster_login, since, preserve): emit("Round cap: the cap comment is already on this PR — not posting it again.") else: latest = rounds[-1] @@ -4304,7 +4308,7 @@ def cmd_round_cap(args) -> int: f"repos/{args.repo}/pulls/{args.pr_number}/reviews/{latest['id']}/comments?per_page=100" ) findings = cap_findings(review_comments, open_thread_ids(args.repo, int(args.pr_number)), threshold) - body = render_cap_comment(len(rounds), max_rounds, findings, threshold, latest.get("html_url") or "") + body = render_cap_comment(len(rounds), max_rounds, findings, threshold, latest.get("html_url") or "", preserve) gh(["api", "-X", "POST", f"repos/{args.repo}/issues/{args.pr_number}/comments", "--input", "-"], {"body": body}) except (RuntimeError, ValueError, KeyError) as e: failed.append(f"comment: {e}") @@ -4739,6 +4743,7 @@ def main() -> int: c.add_argument("--poster-login", required=True) c.add_argument("--fail-closed-round-cap", action="store_true") c.add_argument("--count-superseded-rounds", action="store_true") + c.add_argument("--preserve-round-count", action="store_true") e = sub.add_parser("approve-external") e.add_argument("--repo", required=True) e.add_argument("--pr-number", required=True) diff --git a/.github/cursor-review/tests/test_auto_approve.py b/.github/cursor-review/tests/test_auto_approve.py index 36ba8ae9..a930b51b 100644 --- a/.github/cursor-review/tests/test_auto_approve.py +++ b/.github/cursor-review/tests/test_auto_approve.py @@ -1856,13 +1856,14 @@ def test_count_step_passes_the_opt_in_to_the_real_entrypoint(self): f.write("import json, os, sys\nopen(os.environ['ARGS'], 'w').write(json.dumps(sys.argv[1:]))\n") path = os.path.join(d, "args") env = {"PATH": os.environ["PATH"], "CURSOR_REVIEW_ASSETS": d, "ARGS": path, - "APP_SLUG": "review", "FAIL_CLOSED_ROUND_CAP": "true", "COUNT_SUPERSEDED_ROUNDS": "true", "REPO": "o/r", + "APP_SLUG": "review", "FAIL_CLOSED_ROUND_CAP": "true", "COUNT_SUPERSEDED_ROUNDS": "true", "PRESERVE_ROUND_COUNT": "true", "REPO": "o/r", "PR_NUMBER": "1", "MAX_ROUNDS": "3", "THRESHOLD": ""} subprocess.run(["bash", "-e", "-c", script], env=env, check=True, capture_output=True) with open(path, encoding="utf-8") as f: args = json.load(f) self.assertIn("--fail-closed-round-cap", args) self.assertIn("--count-superseded-rounds", args) + self.assertIn("--preserve-round-count", args) self.assertEqual(args[args.index("--max-rounds") + 1], "3") self.assertEqual(args[args.index("--poster-login") + 1], "review[bot]") @@ -1872,7 +1873,7 @@ class RoundCapCommandTest(unittest.TestCase): """The cap path end to end, with gh stubbed by endpoint.""" def run_cap(self, reviews, max_rounds="5", timeline=(), comments=(), label_exists=True, fail=(), labels=(), - probe_error="HTTP 404", strict=False, count_superseded=False): + probe_error="HTTP 404", strict=False, count_superseded=False, preserve=False): writes = [] def fake_gh(args, payload=None): @@ -1909,7 +1910,8 @@ def fake_gh(args, payload=None): out = os.path.join(d, "out") open(out, "w").close() args = argparse.Namespace(repo="o/r", pr_number="1", max_rounds=max_rounds, threshold="medium", - poster_login=BOT, fail_closed_round_cap=strict, count_superseded_rounds=count_superseded) + poster_login=BOT, fail_closed_round_cap=strict, count_superseded_rounds=count_superseded, + preserve_round_count=preserve) with mock.patch.dict(os.environ, {"GITHUB_OUTPUT": out}), \ mock.patch.object(AA, "gh", fake_gh), \ mock.patch.object(AA, "open_thread_ids", lambda *a: {"11", "12"}), \ @@ -1950,6 +1952,27 @@ def test_a_second_trigger_does_not_repost_the_comment(self): self.assertEqual((rc, out["capped"]), (0, "true")) self.assertEqual([p for _, p, _ in writes], ["repos/o/r/issues/1/labels"]) + def test_opt_in_keeps_review_count_when_handoff_label_is_removed(self): + rc, out, _ = self.run_cap(self.FIVE[:3], max_rounds="3", preserve=True, + timeline=[unlabeled("2026-01-03T12:00:00Z")]) + self.assertEqual((rc, out["capped"], out["rounds"]), (0, "true", "3")) + + def test_preserved_cap_replaces_old_reset_guidance_with_accurate_comment(self): + prior = {"user": {"login": BOT}, "body": AA.ROUND_CAP_MARKER, + "created_at": "2026-01-06T00:00:00Z"} + _, _, writes = self.run_cap(self.FIVE[:3], max_rounds="3", preserve=True, comments=[prior]) + posted = [payload["body"] for _, path, payload in writes if path.endswith("/comments")] + self.assertEqual(len(posted), 1) + self.assertIn("removing the label does not reset", posted[0]) + + def test_resettable_cap_ignores_prior_preserved_guidance(self): + prior = {"user": {"login": BOT}, "body": AA.ROUND_CAP_MARKER + "\n" + AA.PRESERVED_CAP_MARKER, + "created_at": "2026-01-06T00:00:00Z"} + _, _, writes = self.run_cap(self.FIVE[:3], max_rounds="3", comments=[prior]) + posted = [payload["body"] for _, path, payload in writes if path.endswith("/comments")] + self.assertEqual(len(posted), 1) + self.assertIn("removing the label resets", posted[0]) + def test_unlabel_resets_the_cap(self): rc, out, writes = self.run_cap(self.FIVE, timeline=[unlabeled("2026-01-03T12:00:00Z")]) self.assertEqual((rc, out["capped"], out["rounds"], writes), (0, "false", "2", [])) diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index 3ff3d4b0..1f3d90d5 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -753,8 +753,9 @@ on: comment listing the latest round's open findings above `approve_max_severity` (every open finding when that is empty), and sets the `approve_gate` output to `capped`. Auto-approve never - approves a PR carrying that label. Removing the label resets the - count: only rounds after its most recent removal count. A round whose + approves a PR carrying that label. By default removing the label resets + the count: only rounds after its most recent removal count. Set + preserve_round_count to retain earlier reviews. A round whose own decision found the PR head had moved (a newer push superseded it) does not count either, at most once per reviewed commit and at most `max_rounds` times, so a PR runs at most twice the cap's @@ -765,6 +766,13 @@ on: type: number required: false default: 5 + preserve_round_count: + description: >- + Keep counting prior delivered reviews when the human handoff label is + removed. This does not count undelivered panels or serialize runs. + type: boolean + required: false + default: false count_superseded_rounds: description: >- Include delivered reviews whose decisions found the head moved in the @@ -1936,6 +1944,7 @@ jobs: THRESHOLD: ${{ inputs.approve_max_severity }} FAIL_CLOSED_ROUND_CAP: ${{ inputs.fail_closed_round_cap }} COUNT_SUPERSEDED_ROUNDS: ${{ inputs.count_superseded_rounds }} + PRESERVE_ROUND_COUNT: ${{ inputs.preserve_round_count }} PR_NUMBER: ${{ github.event.pull_request.number }} REPO: ${{ github.repository }} run: | @@ -1943,6 +1952,7 @@ jobs: STRICT_FLAG=() if [ "$FAIL_CLOSED_ROUND_CAP" = true ]; then STRICT_FLAG+=(--fail-closed-round-cap); fi if [ "$COUNT_SUPERSEDED_ROUNDS" = true ]; then STRICT_FLAG+=(--count-superseded-rounds); fi + if [ "$PRESERVE_ROUND_COUNT" = true ]; then STRICT_FLAG+=(--preserve-round-count); fi python3 "$CURSOR_REVIEW_ASSETS/auto-approve.py" round-cap \ --repo "$REPO" \ --pr-number "$PR_NUMBER" \ @@ -7273,9 +7283,9 @@ jobs: run: | # The round cap is not exempt either, for the same reason as the size # cap below: no panel looked at this head. The PR is handed to a human - # (needs-human-review); removing that label resets the cap. + # (needs-human-review); reset behavior depends on the caller policy. if [ "$CAPPED" = "true" ]; then - echo "::error::This PR hit cursor-review's max_rounds cap, so no review ran on this head and it is labelled needs-human-review — an unreviewed head cannot pass a blocking gate. Have a human review it, then remove the label to resume automated rounds." + echo "::error::This PR hit cursor-review's max_rounds cap, so no review ran on this head and it is labelled needs-human-review — an unreviewed head cannot pass a blocking gate. Have a human review it; any further automated rounds depend on the caller policy." exit 1 fi if [ "$DIFF_SIZE_RESULT" = "success" ] && [ "$WITHIN_CAP" != "true" ]; then @@ -7349,7 +7359,7 @@ jobs: exit 1 fi if printf '%s\n' "$labels" | grep -qixF 'needs-human-review'; then - echo "::error::This PR carries needs-human-review (cursor-review's max_rounds cap, or a human hand-off), so automated rounds are paused and no panel has reviewed this head — an unreviewed head cannot pass a blocking gate. Have a human review it, then remove the label to resume automated rounds." + echo "::error::This PR carries needs-human-review (cursor-review's max_rounds cap, or a human hand-off), so automated rounds are paused and no panel has reviewed this head — an unreviewed head cannot pass a blocking gate. Have a human review it; any further automated rounds depend on the caller policy." exit 1 fi diff --git a/docs/callers/cursor-review.md b/docs/callers/cursor-review.md index 9c6bdbe1..80d82c30 100644 --- a/docs/callers/cursor-review.md +++ b/docs/callers/cursor-review.md @@ -147,7 +147,8 @@ pull-requests: write # posting the consolidated review (and, at the round cap, | `approve_only_on_push` | `false` | Re-run only the **axes** on a push after a round whose panel passed. On `synchronize`/`reopened`, when the approver's [status card](cursor-approve.md#a-push-re-checked-by-the-axes-alone) shows the last round got past its panel (its axes ran, whatever they said), the reusable runs **no panel** and sets `approve_gate` to `pass`, so the caller's existing cursor-approve jobs judge the pushed head. After `auto_rereview_delay_minutes` (a burst of pushes yields one re-check, on the head still live), and only while the PR is open, not a draft, carries the review label and neither `skip-cursor-review` nor `needs-human-review`, has no approver approval already on that head, no open review thread above `approve_max_severity` (or unbadged), no review label applied since the approver's last decision (a full round in flight), and is under `approve_only_max_runs`. No consolidated review is posted, so it never counts toward `max_rounds`. Needs `max_rounds` > 0 (an uncapped round's card names no round to count re-checks under) and never follows the **last** round of `max_rounds`, which is a human's. A panel round that does not pass leaves the lane: the next push needs a full round, as before — and so, for good, does a status card anyone but the approver identity ever edited (read from the comment's edit history: anyone with write access can edit it). When it re-checks a push, `auto_rereview_after_dismissal` does not relabel for it. Relabelling still runs a full round. Needs `APPROVER_TOKEN`, `approve_max_severity` and a `pull_request_target` caller; no effect under `run_without_label`. With `auto_rereview_after_dismissal` on too, a push on a round whose **panel** blocked (cursor-review's own decide requested changes — not an untrusted or capped round) starts a **full** round by itself through the same debounced relabel, once per head, under the same conditions and `max_rounds`. | | `approve_only_max_runs` | `10` | Cap on `approve_only_on_push` re-checks per panel round, counted on the status card; a new panel round resets it. At the cap a push is left to a relabel (or `auto_rereview_after_dismissal`). Clamped to `50`, the heads the card's record keeps; unparsable re-checks nothing. | | `carry_forward_unchanged_diff` | `false` | Opt-in, for stacked PRs: when a push or a retarget would dismiss the approver's approval but the PR's **own** changes are identical to the ones a round reviewed (a rebase or re-merge onto a moved base, e.g. a lower PR of the stack changed or landed), re-approve the live head, then dismiss the old approval; the card keeps its approval and no round is scheduled. Identical is the compare API's diff from the merge base the approval recorded to the reviewed commit against the diff from the live base to the live head, file by file, context lines and hunk headings included, hunk line numbers ignored. Anything it cannot vouch for dismisses as before, and an approval that records no merge base (every one an earlier version posted, too) never carries. Only under `pull_request_target`, and only with `approve_max_severity` set and `bot_app_id` posting the rounds (with `defer_approval`, pass cursor-approve that App's `poster_login` too). See [Auto-approve](#auto-approve) for every condition and the [trust model](#trust-model) for what it trusts. | -| `max_rounds` | `5` | Cap on review rounds per PR (`0` → no cap). At or over it, no panel runs: the PR is labelled `needs-human-review`, one comment lists the latest round's open findings above the threshold, and the `approve_gate` output is `capped`. A round whose own decision found the PR head had moved (a newer push superseded it) does not count, at most once per reviewed commit and at most `max_rounds` times. Removing the label resets the count. See [round cap](#round-cap-and-the-approve_gate-output). | +| `max_rounds` | `5` | Cap on review rounds per PR (`0` → no cap). At or over it, no panel runs: the PR is labelled `needs-human-review`, one comment lists the latest round's open findings above the threshold, and the `approve_gate` output is `capped`. A round whose own decision found the PR head had moved (a newer push superseded it) does not count, at most once per reviewed commit and at most `max_rounds` times. Removing the label resets the count unless `preserve_round_count` is enabled. See [round cap](#round-cap-and-the-approve_gate-output). | +| `preserve_round_count` | `false` | Keep counting prior delivered reviews after the human handoff label is removed. Does not count undelivered panels or serialize runs. | | `count_superseded_rounds` | `false` | Count delivered reviews even when a later decision found the head moved. Does not count undelivered panels or change resets. | | `fail_closed_round_cap` | `false` | Stop the panel if the cap cannot be evaluated or handoff fails. Requires a positive `max_rounds` and, when configured, the posting bot token and identity. This preserves the existing reset, superseded-round exemptions and concurrency behavior; it does not guarantee a lifetime limit on all reviewer executions. | | `runs_on` | `'"ubuntu-latest"'` | JSON-encoded `runs-on` for `diff-size`, `preflight`, the `review` panel and `consolidate` only — the jobs that hold no write credential. Every other job stays on GitHub-hosted `ubuntu-latest`. A self-hosted pool is fine only if it is one-job-per-fresh-pod, identity-free and private-network-isolated (those jobs run models with shell over PR code), on linux/x64 with bash, git, curl, jq, python3, gh, tar and GNU coreutils. Empty falls back to the default, so `${{ vars.CURSOR_REVIEW_RUNS_ON }}` is safe while the variable is unset; e.g. `'["self-hosted", "linux", "x64"]'`. | @@ -1239,10 +1240,12 @@ round with no decision, and one whose head moved during the decision's own post again for the same cap; - sets `approve_gate` to `capped`. -While the label is on, auto-approve never approves the PR. **Removing the label +While the label is on, auto-approve never approves the PR. By default, **removing the label resets the cap**: only rounds after its most recent removal (read from the issue timeline) count, so a human who has looked at the PR can hand it back to the bot -for another `max_rounds` rounds. By default, a count that cannot be read fails +for another `max_rounds` rounds. With `preserve_round_count: true`, removing the +label does not reset this count; a later trigger at the cap restores the handoff. +By default, a count that cannot be read fails open and the panel runs. With `fail_closed_round_cap: true`, the cap job fails and the panel does not run. Under `blocking: true` a capped head holds **Blocking gate** red, the same as an over-cap diff: no panel looked