Repository navigation
fix(cursor-review): retry a panel cell on Cursor's transient resource_exhausted error - #388
Conversation
…_exhausted error A cell whose cursor-agent exits non-zero with Cursor's retriable marker (`RetriableError: [resource_exhausted]`) is re-run by stop-on-submit.py after 30s, then 90s, before falling through to the pre-seeded status=error path. Only a cell with no submission and no recorded findings is re-run (the MCP tool appends findings), no retry starts past 300s so the last attempt still fits the 15-minute step cap, and each retry emits one ::warning:: naming the cell, attempt and marker. Any other exit, a signal death or a forwarded signal behaves exactly as before.
|
Warning Review paused — included plan limit reachedKeep your review moving with free on-demand reviews.
On-demand reviews are free for one more day.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Promotion and pricing detailsOn-demand reviews are free for one more day. After that, they cost $0.25 per reviewed file. Review limit detailsOr wait 5 minutes for your next included review. Limit details: You’ve used the included review currently available. Your 129 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Review configuration: ⚙️ Run configuration
📒 Files selected for processing (4)
Comment |
There was a problem hiding this comment.
🔍 Cursor Review — Consolidated panel
Triggered by @mattmillerai.
Found 6 finding(s).
| Severity | Count |
|---|---|
| 🟠 High | 1 |
| 🟢 Low | 4 |
| ⚪ Nit | 1 |
Panel: 5/6 reviewers contributed findings.
Reviewers that did not contribute: kimi-k3-high:adversarial (error)
| child = subprocess.Popen(command, start_new_session=True, | ||
| stdout=subprocess.PIPE, stderr=subprocess.PIPE) | ||
| current[0] = child | ||
| tees = (Tee(child.stdout, sys.stdout.fileno()), |
There was a problem hiding this comment.
🟢 Low — Both Tee(...) objects are built after the agent is spawned but before the try/finally that SIGKILLs its process group. If either one raises (for example Thread.start() hitting a thread/pid limit, or sys.stdout.fileno() failing), the wrapper exits and leaves the credential-bearing agent group running unsupervised. That breaks the documented rule that every exit path ends with SIGKILL, so an orphan could rewrite findings.json after the upload. Move the tee construction inside the try. Raised by 3 of 6 reviewers (claude-opus-5-5-xhigh adversarial, gpt-5.6-sol-xhigh adversarial, kimi-k3-high edge-case).
| signal_group(child.pid, signal.SIGKILL) | ||
| current[0] = None | ||
| tails = [tee.drain() for tee in tees] | ||
| for stream in (child.stdout, child.stderr): |
There was a problem hiding this comment.
🟢 Low — drain() can time out when a process that escaped the group (for example a setsid descendant of the --trust shell) still holds a pipe's write end. When it does, the pump thread is still blocked in os.read(fd), and stream.close() frees that fd number anyway. A later loop iteration of that stale thread can then read from a reused fd (the next attempt's pipe, or read_findings' os.open). It would steal the retry's output, so the retry marker or the passthrough bytes go missing, or it would corrupt the findings read. Don't close a stream, or don't retry, while its pump thread is still alive. Or give the thread a select-based loop with a stop flag. Raised by 5 of 6 reviewers (claude-opus-5-5-xhigh adversarial, gpt-5.6-sol-xhigh adversarial, claude-opus-5-5-xhigh edge-case, gpt-5.6-sol-xhigh edge-case, kimi-k3-high edge-case).
| def retriable_marker(*tails): | ||
| """The fixed marker name Cursor's retriable error carries, or None.""" | ||
| for name, pattern in RETRIABLE: | ||
| if any(pattern.search(tail) for tail in tails): |
There was a problem hiding this comment.
🟢 Low — The marker can match anywhere in the last 64 KiB of either stream. Under --print, stdout is model-written text that can quote the diff under review (this PR contains RetriableError: [resource_exhausted]) or be steered by prompt injection. So any unrelated non-zero exit before findings are recorded gets treated as capacity exhaustion, which costs up to two extra billed runs and 120s of backoff. Anchor the match to Cursor's own error line, for example the final non-empty line of stderr (or of stdout, if that is where the CLI reports it), rather than a free-text search. Raised by 5 of 6 reviewers (claude-opus-5-5-xhigh adversarial, gpt-5.6-sol-xhigh adversarial, claude-opus-5-5-xhigh edge-case, gpt-5.6-sol-xhigh edge-case, kimi-k3-high edge-case).
Every attempt inherits the wrapper's stdin, the `< /tmp/prompt.txt`
redirect, together with its shared file offset. The first attempt reads
it to EOF, so a retried cursor-agent read an empty prompt and failed at
once ("No prompt provided for print mode"): a retry could never recover
a cell. The wrapper now records stdin's offset at startup and seeks back
to it before every retry. A stdin that cannot seek (a pipe, a tty)
cannot be replayed, so it rules the retry out and the wrapper logs why.
Two smaller fixes from the same review:
- The retry `::warning::` always starts a line of its own. It was glued
onto the agent's last stderr write when that write had no trailing
newline, and the runner then did not parse it as a command.
- The retriable marker is matched in the agent's stderr only. stdout is
the model's own text, which the PR under review can steer, so it is
inherited as is again instead of being teed and searched.
New tests use a stub agent that reads its whole stdin: every attempt
must get the full prompt (from the offset stdin started at), a pipe on
stdin must rule the retry out, a marker on stdout must not trigger one,
and the warning must start a line after an unterminated stderr. The
retry tests now feed the wrapper a prompt file, as the workflow does,
instead of inheriting the test runner's stdin.
…rr drains A forwarded signal (the step cap) that arrived after the agent exited by itself, while its stderr was still draining, was recorded but ignored: the wrapper returned the agent's own code, possibly 0, instead of 128+signal, and the drain could still run its full 5s past the signal. The drain now stops as soon as a signal is pending, and run() reports any received signal as the wrapper's status before considering a retry. New test: an agent that exits 0 leaving a process outside its group that holds stderr open; SIGTERM during the drain must exit 143 at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#393) * fix(cursor-review): move the default Moonshot cell back to kimi-k3-max `kimi-k3-high` is capacity-bound, so the built-in panel is currently one lane short on every consumer that has not overridden it — which is most of them. The cells do not run slow, they die: ~25 seconds in, `RetriableError: [resource_exhausted]` after three internal reconnects, zero findings, leg red, `Panel integrity` red with it. Measured on one consumer over 56 panels, 24 of 76 `kimi-k3-high` cells red (32%), 19 of those this error, and 12 of 22 on the last day. Not a property of one repo: the same model threw the same error the same day on two unrelated consumers. And #388's in-step retry does not rescue it, because the condition is sustained rather than transient — both cells on this repo's own panel for #392 exhausted all three attempts. `-max` rather than dropping the lane, because the lane earns its place. The 2026-08-27 attribution that cut Gemini was re-run over 1,158 judge-kept findings (82 PRs), normalised per finding raised so panel size flatters nobody: model raised sole distinct sole/PR c/h sole kimi-k3-max 229 39 17% 1.05 3/58 kimi-k3-high 81 22 27% 1.05 0/11 gemini-3.1-pro 99 12 12% 0.33 0/58 Kimi contributes 1.05 unique findings per PR at BOTH tiers and uniquely caught 3 of 58 critical/high at `-max`. Gemini — cut for exactly this reason — managed 0.33 and 0 of 58. Dropping Kimi would cost about three times what dropping Gemini did, so the fix is the tier, not removal. This deliberately gives back the spend the `-high` notch in #243 was for; it is a partial reversal of that change, not an oversight. And the reliability it buys is NOT established at this tier under present conditions: `-max` was 0 of 36 cells red, but that was 2026-10-07, before the capacity trouble, and the two tiers may share a pool. Recorded in the workflow so the next person does not have to rediscover it, along with what the evidence says to do if `-max` goes the same way: drop to a two-lab Cursor panel rather than substitute Gemini or Grok. Grok `4.7-high` was tried on 2026-10-09 and raised 0 unique findings in 17 while timing out on 11 of 20 cells. Default change, so it reaches every consumer on the next SHA bump; a repo that wants the old tier back sets CURSOR_PANEL_MODELS. README panel table and history updated alongside. Tests: cursor-review 1257, workflow-pins 551 + pin lint, agents-md — all green. The historical-era fixtures and `test_catalog_drift.py`'s REAL_PANEL pin their own eras by design and are deliberately untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(cursor-review): rejoin the panel_models sentence the Kimi note split Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(cursor-review): move the remaining kimi-k3-high examples to the -max pin The CURSOR_PANEL_MODELS copy-paste examples would otherwise change two axes and silently put a fleet back on the capacity-bound -high tier. Also reword the README's 32% figure to match the source data. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ELI-5
Sometimes Cursor says "I'm too busy right now, try again" (
RetriableError: [resource_exhausted]) about 25 seconds into a review cell. Today the cell gives up straight away and the panel posts with reviewers missing. With this change the cell waits a bit and tries again, at most twice, before it gives up.What changed
stop-on-submit.py(the wrapper around each panel cell'scursor-agent) gets a retry loop:[resource_exhausted]orRetriableError. stderr passes through the wrapper and is copied on to its own stderr byte for byte, so/tmp/review-stderr.txtgets the same content as before. stdout is inherited untouched.< /tmp/prompt.txt) is rewound to its starting offset before every retry; a stdin that cannot seek rules the retry out.--retry-delays 30,90, so at most 2 retries.findings.jsonto parse as an object whose status is notokand whosefindingslist is empty.review-output-mcp.pyappends each recorded finding, so re-running a cell that already recorded some would duplicate them. Those cells, and an unreadable file, fail exactly as they do today.--retry-window 300seconds after the wrapper started (delay included). The last attempt therefore still has about 10 of the 15 minutes for a full review. If it runs past that, the step cap kills it as before.::warning::Reviewer cell <review_type>/<model>: attempt N/3 failed with Cursor's retriable error [resource_exhausted] (exit 1); retrying in 30s. The warning names only the fixed marker, never text the agent printed, and the cell label is sanitized.cursor-review.ymlRun cursor reviewpasses--retry-delays 30,90 --retry-window 300 --cell "$REVIEW_TYPE/$MODEL"(it addsREVIEW_TYPEto the step env) and has a comment explaining why. A cell that uses up its retries still endsstatus=error, andFail the leg when the cell did not submitstill turns it red.stop-on-submit.pyupdated.Judgment calls
okstatus and no recorded findings, so partial reviews are never duplicated.Provenance
.github/cursor-review/testssuite (1188 tests, Python 3.12) passes on the current head, includingtest_stop_on_submit.py(41 tests)RetryTest(real wrapper over a stub agent that reads its whole stdin): retry then success, retries used up, every attempt gets the full prompt (also when stdin starts partway in), a pipe on stdin rules the retry out, a marker on stdout isn't retried, the warning starts its own line, other errors, exit 0 and signal death not retried, submitted or partially recorded findings not re-run, retry window, signal during the backoffcheck_workflow_pins.pyandcheck_agents_md.pypass on the current head;check-org-repo-literals.sh, shellcheck and actionlint were run on earlier roundsResidual
ci-cursor-review.ymlruns on the consumer repo were re-read here. The matcher is based on the quoted log tailRetriableError: [resource_exhausted] Error. After the next cursor-review bump, check a consumer run with a Kimi (kimi-k3-high) failure for the new::warning::Reviewer cell …line and a second attempt in theRun cursor reviewlog.status=error, as the ticket allows. The model swap and counting the direct-OpenAI cell toward the gate are the separate follow-ups the ticket names.review-openai-direct) and the judge/consolidate step get no retry. Both are out of scope per the ticket.Review fixes
< /tmp/prompt.txt, and every attempt shares that one open file and its read position. The first attempt read it to the end, so a retriedcursor-agentgot an empty prompt and failed at once (No prompt provided for print mode). The retry could never save a cell. The wrapper now notes where stdin starts and moves it back there before every retry. If stdin can't be rewound (a pipe or a terminal), it doesn't retry and logs one line saying why. A workflow comment and the wiring test now pin the file redirect.::warning::always starts on its own line. When the agent's last stderr write had no trailing newline, the warning was glued onto it and the runner didn't pick it up. The wrapper now starts a new line first when needed./tmp/review-raw.txtagain, as before this PR. This replaces the "both stdout and stderr are scanned" judgment call above.