Skip to content

fix(cursor-approve): allow the checkout axes the shell they were meant to have - #375

Merged
mattmillerai merged 3 commits into
mainfrom
matt/axis-checkout-precomputed-diff
Oct 8, 2026
Merged

mattmillerai merged 3 commits into
mainfrom
matt/axis-checkout-precomputed-diff

Conversation

@mattmillerai

@mattmillerai mattmillerai commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

ELI-5

The correctness, conformance and completeness reviewers are told to read the PR's diff with git. Every shell command they try gets silently rejected, so they never see the diff and approve by guessing from files on disk. This PR gives them the shell they were always meant to have. The reviewer gets the merge-base range and decides how to review it. If the shell ever breaks again, the job now fails visibly instead of returning a blind verdict.

What was wrong

cursor-axis-base.yml writes .cursor/cli.json with allow: []. Unless no_shell is set, it also writes deny: []. cursor-agent --print has no one to approve a command, so it rejects any shell call that no allow rule matches. On checkout axes, every git diff and git log came back {"rejected": {..., "reason": ""}}. The agents fell back to Read and Grep and said so in their verdicts ("Shell/git was unavailable to me"). The caller docs promised the completeness axis "keeps a shell for git", which was never true.

Change

  • Explicit allow rule. When no_shell is false (the checkout axes), the config adds Shell(*) to allow. Business and design are unchanged: they get no shell, the existing deny list, and the existing check that fails the job if a shell call appears anyway.
  • Guards against a quiet regression:
    • Every axis now fails if its transcript has no recognizable tool call. That check used to run only on no_shell axes.
    • A checkout axis fails if every shell call it made was rejected, which is exactly this bug.
  • Docs and the input header describe the shell and the guards.
  • PR-authored ignore files are removed before the agent runs: .cursorignore and .cursorindexingignore are deleted alongside the PR's .cursor/ config, so a PR cannot hide its own changes from the read and search tools of the reviewer judging it.

Revised after review

The first version precomputed the diff into .git/ and pointed the agent at it. @huntcsg pointed out that a large diff can blow up the agent's context, and that with a shell the agent can choose its own approach: --stat first, read only the relevant hunks, or script a check over a mechanical change. The prompt already names the range (git diff <merge_base>...<head>), so this version drops the precompute.

Risk

This is the same unsandboxed shell over PR code that the cursor-review panel already accepts (see the cursor-review.yml header). The axis job has contents: read, a checkout with persist-credentials: false, and CURSOR_API_KEY in the environment. For completeness, the existing note about the Linear proxy token on a passwordless-sudo runner still applies, and the docs keep it.

Testing

  • python3 -m unittest discover -s .github/cursor-approve/tests -p 'test_*.py': OK. Two tests are new: the allow rule plus the all-rejected guard, and the every-axis tool-call guard.
  • The cursor-review and workflow-pins suites are OK.
  • actionlint: nothing new. Its only warnings are the existing job.workflow_sha ones.
  • Not yet verified end to end. I couldn't run cursor-agent locally to confirm that Shell(*) turns rejected into executed calls. The new guard turns the first real round into the test: either the transcript shows shell calls succeeding, or the axis job fails with "all N shell call(s) were rejected".

Provenance

  • Authored by: agent-work loop
  • Verified:
    • cursor-approve unittest suite: 164 tests OK at the current head, including the .cursorignore removal assertion.
    • check_workflow_pins.py, check_agents_md.py --root . and the org-repo-literal lint pass.
    • PR CI checks are green at the current head. The end-to-end cursor-agent run is still unverified (see Testing).
  • Deviations:
    • The precomputed-diff approach from the first revision was replaced with an explicit Shell(*) allow rule after review.
    • Also removes PR-authored .cursorignore and .cursorindexingignore files. This was not in the original scope.

🤖 Generated with Claude Code

…hell they never had

The correctness and conformance axes were told to run `git diff` from the
merge base, but their Cursor config had an empty allow list and an empty
deny list. In --print mode that rejects every shell command, so each one
came back `rejected` and the agent reviewed the head tree blind: no diff,
no deletions, no commit log. Axis summaries said so ("Shell/git was
unavailable to me").

The workflow now runs that git itself for a checkout axis and leaves the
output beside the PR context file under pr/.git: the merge-base diff (cut
at 1 MB), its --name-status list, and the PR's commit log. A new
prompt-checkout.md addendum points the agent at them. Every axis is now
denied Shell(*), Write(**) and WebFetch(*) explicitly, so the boundary is
stated instead of falling out of an empty allow list. Allowing Shell(git)
instead would have been a full shell (`git -c alias.x='!sh' x`) for an
agent holding CURSOR_API_KEY over PR code. The precompute passes
--no-ext-diff --no-textconv so a PR's .gitattributes cannot run a driver.
@coderabbitai

coderabbitai Bot commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review paused — included plan limit reached

Keep your review moving with free on-demand reviews.

  • Run this review for free

On-demand reviews are free for the next 2 days.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Promotion and pricing details

On-demand reviews are free for the next 2 days. After that, they cost $0.25 per reviewed file.

Review limit details

Or wait 9 minutes for your next included review.

Check out review usage here.

Limit details: You’ve used the included review currently available. Your 130 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 32275738-8fd0-4091-a576-8676aa141886
📥 Commits

Reviewing files that changed from the base of the PR and between 9f977b7 and ccf1100.

📒 Files selected for processing (3)
  • .github/cursor-approve/tests/test_context_axes.py
  • .github/workflows/cursor-axis-base.yml
  • docs/callers/cursor-approve.md

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: ASSERTIVE
  • Plan: Team
  • Run ID: 44584e3a-35ff-4b02-8ab2-ad696c74a90e
📥 Commits

Reviewing files that changed from the base of the PR and between 100472a and 9f977b7.

📒 Files selected for processing (5)
  • .github/cursor-approve/prompt-checkout.md
  • .github/cursor-approve/tests/test_aggregate.py
  • .github/cursor-approve/tests/test_context_axes.py
  • .github/workflows/cursor-axis-base.yml
  • docs/callers/cursor-approve.md

Included review availability: This review used your included allowance. 0 included reviews remain after this review. Your included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your free on-demand review promotion remains active until October 9, 2026 at 6:00 PM UTC.


📝 Walkthrough

Walkthrough

The workflow now prepares Git context files for checkout axes and denies shell, write, and web-fetch tools for all axes. The checkout prompt and caller documentation describe the saved context and no-shell execution.

Changes

Checkout context and tool restrictions

Layer / File(s) Summary
Prepare checkout context and enforce tool restrictions
.github/workflows/cursor-axis-base.yml, .github/cursor-approve/tests/test_context_axes.py
The workflow generates a merge-base diff, changed-file list, and commit log for checkout axes. It caps the diff at 1 MiB and adds a truncation notice when needed. Checkout axes use the checkout prompt. All axes deny shell, write, and web-fetch tools. Tests check the context commands and tool settings.
Document checkout prompt and caller behavior
.github/cursor-approve/prompt-checkout.md, .github/cursor-approve/tests/test_aggregate.py, docs/callers/cursor-approve.md
The prompt describes how to read the saved Git context and handle a truncated diff. Caller documentation describes no-shell execution and precomputed checkout context. The prompt-file test excludes the checkout prompt as an addendum.

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 9f977

Checkout axes now read Git output prepared by the workflow, and shell, write, and web-fetch tools are denied for every axis. No concrete defect was found. An end-to-end run is still needed to confirm the agent can read the prepared diff.

🚥 Pre-merge checks | ✅ 2
✅ Passed checks (2 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
✨ Simplify code
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@mattmillerai mattmillerai added the cursor-review Multi-model cursor review label Oct 8, 2026
@huntcsg

huntcsg commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

@mattmillerai one reason not to hand the diff to the reviewer is that it can blow up context if the diff is large. If I recall cursor review just doesn't do large diff sizes, but sometimes the diff is mechancial or there are other ways to evaluate (e.g. agent may write some python or shell script to detect true diff).

Why not just tell the reviewer the range it's reviewing and give it the choice for how to approach?

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Cursor Review — Consolidated panel

Triggered by @mattmillerai.

Found 9 finding(s).

Severity Count
🟡 Medium 4
🟢 Low 2
⚪ Nit 3

Panel: 6/6 reviewers contributed findings.

Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved
Comment thread .github/workflows/cursor-axis-base.yml Resolved

@huntcsg huntcsg left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please see my comment. I'm not sure that you should hand the diff to the agent. Obviously your call. I think it will reduce the number of eligible PRs.

…ng them a diff

Review feedback: a precomputed diff pushes the agent toward reading one
large file and can blow up its context, where with a shell it can choose
(--stat first, read only the hunks that matter, script a check over a
mechanical change). The prompt already names the merge-base range.

Reverts the precompute and prompt-checkout.md. Instead, a checkout axis
(no_shell false) gets an explicit Shell(*) allow rule, which is what was
missing: --print rejects any command no allow rule matches. This is the
same unsandboxed shell over PR code the cursor-review panel accepts.

Two transcript guards make the blind-verdict failure loud:
- every axis, not just no_shell ones, fails with no recognizable tool call;
- a checkout axis fails if every shell call it made was rejected.
@mattmillerai mattmillerai changed the title fix(cursor-approve): give checkout axes the git output instead of a shell they never had fix(cursor-approve): allow the checkout axes the shell they were meant to have Oct 8, 2026
@mattmillerai

Copy link
Copy Markdown
Member Author

@huntcsg Agreed, and reworked: the precomputed diff is gone. The real bug was that --print rejects any shell command no allow rule matches, and the checkout axes had allow: [], so they never had a shell at all. They now get an explicit Shell(*) rule (the same exposure the cursor-review panel already accepts), and the prompt keeps just naming the merge-base range so the agent picks its own approach. I also added a guard that fails the axis if every shell call is rejected, so this can't silently regress.

… axis runs

A PR could ship a .cursorignore or .cursorindexingignore that hides its own
changed files from the read and search tools of the reviewer judging it.
Delete them alongside the PR-authored .cursor/ config.
@mattmillerai
mattmillerai merged commit c14822e into main Oct 8, 2026
8 checks passed
@mattmillerai
mattmillerai deleted the matt/axis-checkout-precomputed-diff branch October 8, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cursor-review Multi-model cursor review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants