Skip to content

feat(cursor-review): auto-retry a round the panel broke, with an announced delay - #409

Merged
mattmillerai merged 2 commits into
mainfrom
matt/cursor-review-auto-retry-errored-rounds
Oct 11, 2026
Merged

mattmillerai merged 2 commits into
mainfrom
matt/cursor-review-auto-retry-errored-rounds

Conversation

@mattmillerai

@mattmillerai mattmillerai commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Follow-up to #407. That PR made it possible to re-run a round on the same commit; this makes it automatic for the one cause that needs it most.

Why

A round can end with no decision for a reason that has nothing to do with the code — overwhelmingly reviewer cells erroring on provider capacity or a cell timeout. Re-running it is the entire fix, and until #407 there was no way to:

  • the gate dedupes on the consolidated review for the current HEAD SHA
  • that review is COMMENTED, which GitHub will not dismiss
  • the diff is fine, so there is nothing to push

So a whole panel was spent and a human had to notice the card and relabel by hand.

What

Two new inputs, both off by default:

input default what it does
auto_retry_errored_rounds false extends the BE-19526 moved-head retry to the errored-panel cause
auto_retry_delay_minutes 0 waits before relabelling — capacity failures need time to clear

The card and the standing block name the wait before it starts (in about 5 minutes). That is the point of the delay being visible: an author who reads "re-run automatically" and then sees nothing happen for five minutes re-runs it by hand, which is the work this removes.

What bounds it

All pre-existing except the last:

  • one retry per PR, counted off decide's own marked block, shared with the moved-head cause rather than a second budget
  • max_rounds — a round already at the cap is not retried into it
  • a token that can actually relabel (a GITHUB_TOKEN-applied label fires no run)
  • no standing block, no retry — nothing would count it
  • never a structural cause or findings above the threshold: the next round would hit the same wall. On an unchanged head that includes findings from the cells that did complete, and open threads above the threshold (blocking_without_trust). Mixed causes follow the strictest one.
  • after the wait, the job re-reads the PR and skips a closed PR or one labelled needs-human-review

The part worth reviewing closely

The gate now has two force signals and they are deliberately separate. The human one (rereview_command) scans PR comments and authorizes by association. The retry posts no comment. Its signal is the marked block decide writes, and the gate accepts it only when it:

  • was authored by a Bot (the App identity) or by the identity that applied the label (the APPROVER_TOKEN PAT, which the auto-retry job relabels as)
  • has decide's own shape: the auto-approve marker first, and the retry marker on a line of its own
  • is not dismissed
  • is pinned to the commit
  • is newer, by review id, than the newest consolidated review The comment query stays guarded on rereview_command alone, so enabling auto-retry cannot widen who may force a round by commenting; test_the_retry_cannot_use_the_human_comment_path pins that by reading the actual guard line.

The gate hardcodes the marker literals because it runs before any asset is loaded. A test pins them equal to auto_approve.AUTO_RETRY_MARKER / APPROVE_MARKER, and another pins card.py's clamp equal to the workflow's.

A real bug found while wiring this

The relabel job budgeted timeout-minutes: 5. Any wait past that would have killed the round the card had already promised — the one failure mode where announcing a retry is worse than not retrying. Now 35 (5 + the 30 the delay clamps to), pinned by a test that fails if the budget drops below the cap. The job is also no longer named Re-run the round (head moved), since it now serves two causes.

Tests

New cases in test_auto_retry_errored.py:

  • the eligibility truth table, including every False a retry cannot fix
  • the gate's real jq filter, extracted from the workflow and run against the real render_standing_body output and against these payload shapes: older id, same-second block, other commit, PAT identity that relabelled, other human, dismissed, quoted marker, two marked, null body/id, empty list
  • the delay clamp, run through bash with sleep stubbed: 0 → none, 5 → 300s, 31/1000/23-digit → 1800s, abc/-5/1.5 → none
  • the predicate matched against a reason generated by panel_gate, not a hand-written string, so producer and matcher cannot drift
  • constant and cap parity, and the job-timeout floor

Plus behavioural cases in AutoRetryTest that drive the real cmd_decide and cover the card marker, already-blocking findings and threads, and the moved-head exemption. There are also cmd_auto_retry cases for a PR that is closed, labelled for a human, or unreadable.

Enabling it

Nothing changes for any existing caller — both inputs default off, and the zero-delay card sentence is byte-identical to today's. Turning it on for cloud is a separate PR there.

Provenance

Verified:

  • cursor-review suite: 1525 tests OK (local, head b9ade28)
  • cursor-approve suite: 236 OK; workflow-pins suite: 551 OK (43 skipped)
  • check_workflow_pins.py, check-org-repo-literals.sh and check_agents_md.py pass; cursor-review.yml parses as YAML
  • CI on b9ade28 was still pending at the time of this update

Deviations:

  • Review round 1 (10 panel findings) addressed in b9ade28. The gate identity check now accepts a Bot author or the label sender, instead of keying on the approver login, because the gate holds no token to resolve that login.
  • Under APPROVER_TOKEN, a hand relabel on the same commit is still deduped. This is documented, and rereview_command is the hand path.

🤖 Generated with Claude Code

…unced delay

A round can end with no decision for a reason that has nothing to do with
the code — most often reviewer cells erroring on provider capacity or a cell
timeout. Until `rereview_command` (#407) there was no way to re-run one: the
gate dedupes on the consolidated review for the current HEAD SHA, that review
is `COMMENTED` so GitHub will not dismiss it, and the diff is fine so there is
nothing to push. #407 made the hand path possible; this makes it automatic.

`auto_retry_errored_rounds: true` (default false) extends the BE-19526
moved-head retry to the errored-panel cause, and `auto_retry_delay_minutes`
waits first — the failures this retries are capacity, and a few minutes is
what makes the second attempt different from the first. The card and the
standing block name the wait before it starts, because an author who reads
"re-run automatically" and sees nothing happen for five minutes re-runs it by
hand, which is the work the input exists to remove.

Bounds, all of them pre-existing except the last:

- one retry per PR, counted off decide's own marked block, SHARED with the
  moved-head cause rather than a second budget
- `max_rounds`, and a round at the cap is not retried into it
- a token that can actually relabel (a `GITHUB_TOKEN` label fires no run)
- no standing block, no retry — nothing would count it
- never a structural cause or findings above the threshold: the next round
  would hit the same wall. Mixed causes follow the strictest one.

The gate's force signal for this path is that marked block — `user.type ==
"Bot"`, pinned to the commit, newer than the newest consolidated review, and
capped at one — NOT the `rereview_command` comment query, which stays guarded
on that input alone so enabling the retry cannot widen who may force a round
by commenting. A test pins the gate's hardcoded marker equal to
auto-approve.py's `AUTO_RETRY_MARKER` (the gate runs before any asset loads,
so the literal is duplicated on purpose).

Also fixes a real bug found while wiring this: the relabel job budgeted
`timeout-minutes: 5`, so any wait past it would have killed the round the card
had already promised. Now 35 (5 + the 30 the delay is clamped to), pinned by a
test, and the job is no longer named "(head moved)" now that it serves two
causes.

Tests: 44 new in test_auto_retry_errored.py (eligibility truth table, the
gate's jq marker filter executed against 7 payload shapes, the clamp run
through bash, constant/cap parity, the human-path separation) plus 6
behavioural cases in AutoRetryTest. Every new guard mutation-checked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

  • Run on-demand review

This review includes 8 billable files and costs up to $2.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 55 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 125 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: bfa386ac-7bb3-4bfc-b6fe-7dc34171d940

📥 Commits

Reviewing files that changed from the base of the PR and between ceedf39 and b9ade28.


📒 Files selected for processing (8)
  • .github/cursor-approve/card.py
  • .github/cursor-review/README.md
  • .github/cursor-review/auto-approve.py
  • .github/cursor-review/tests/test_auto_approve.py
  • .github/cursor-review/tests/test_auto_retry_errored.py
  • .github/workflows/cursor-review.yml
  • docs/callers/cursor-approve.md
  • docs/callers/cursor-review.md

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mattmillerai mattmillerai added the cursor-review Multi-model cursor review label Oct 11, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 10 finding(s).

Severity Count
🟠 High 1
🟡 Medium 5
🟢 Low 2
⚪ Nit 2

Panel: 6/6 reviewers contributed findings.

Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread .github/cursor-review/auto-approve.py Resolved
Comment thread .github/cursor-review/auto-approve.py Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread docs/callers/cursor-review.md Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
Comment thread .github/workflows/cursor-review.yml Resolved
…retry

- Gate: accept the marked block from a Bot OR the identity that applied the
  label, so an APPROVER_TOKEN (PAT, User-typed) retry is no longer deduped
  away; order by review id rather than one-second submitted_at; require
  decide's own block shape (auto-approve marker first, retry marker on its
  own line) and exclude DISMISSED; drop the `-le 1` that turned a second
  marked block into a dedupe.
- decide: do not spend the retry on an unchanged head whose completed cells
  or open threads already block (decide_gate never reaches that check on an
  untrusted round); set the card's next marker to `relabel` when retrying.
- auto-retry job: re-read the PR after the wait and skip a closed PR or one
  labelled needs-human-review; reject over-long delay values before the
  numeric clamp; summary text no longer claims the head moved.
- Docs: caller example sets approve_max_severity and bot_app_id, inputs say
  "No effect without approve_max_severity", warn that any event in a
  PR-keyed cancel-in-progress group can cancel the waiting retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmillerai
mattmillerai merged commit f104746 into main Oct 11, 2026
8 checks passed
@mattmillerai
mattmillerai deleted the matt/cursor-review-auto-retry-errored-rounds branch October 11, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cursor-review Multi-model cursor review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants