From 7630b11da4cd3fc349ecc21b301f05bbf3efbbfc Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 8 Oct 2026 06:25:23 +0000 Subject: [PATCH 1/2] feat(cursor-review): end a panel cell's agent once findings.json reports status=ok MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Some reviewer agents keep running after `cursor_review_finish` has written status=ok, idling until the 15-minute `Run cursor review` cap kills them. stop-on-submit.py wraps cursor-agent, polls findings.json every 5s, and once it parses as an object with status "ok" gives the agent 15s to exit, then SIGTERMs its process group and SIGKILLs it 10s later — out within 30s of the submission, exiting 0 so the leg stays green. It only reads findings.json, so the artifact is byte-for-byte what the MCP tool wrote. A missing, partial, unparseable or non-ok file never triggers a stop, and the cap's/cancel's signals are forwarded to the agent's group, so a cell that never submits still runs to the cap and fails its leg as before. Co-Authored-By: Claude Opus 5.5 --- .github/cursor-review/README.md | 1 + .github/cursor-review/stop-on-submit.py | 140 +++++++++ .../tests/test_stop_on_submit.py | 286 ++++++++++++++++++ .github/workflows/cursor-review.yml | 24 +- 4 files changed, 445 insertions(+), 6 deletions(-) create mode 100644 .github/cursor-review/stop-on-submit.py create mode 100644 .github/cursor-review/tests/test_stop_on_submit.py diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 24a2ec7f..5e1081b9 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -217,6 +217,7 @@ the consolidated review's panel table. See [Panel integrity](#panel-integrity). | [`prompt-edge-case.md`](prompt-edge-case.md) | Prompt for the correctness/logic review pass. | | [`prompt-judge.md`](prompt-judge.md) | Prompt the judge model uses to consolidate panel findings into one review. | | [`review-output-mcp.py`](review-output-mcp.py) | Dependency-free stdio MCP server. Reviewers record individual findings and finish; the judge submits the final findings array. It validates and atomically writes the normalized records consumed by later jobs. | +| [`stop-on-submit.py`](stop-on-submit.py) | Wraps each panel cell's `cursor-agent` and polls its `findings.json`. Once the file parses as a JSON object with `"status": "ok"`, the agent gets 15s to exit by itself, then SIGTERM to its process group, then SIGKILL 10s later, so a cell that has submitted exits **within 30 seconds** of submitting instead of idling to the 15-minute step cap. It only reads the file, so the artifact is byte-for-byte what the MCP tool wrote, and the stop exits 0, so the leg stays green. A missing, partial, unparseable or non-`ok` file never triggers a stop, so a cell that never submits still runs to the cap and fails its leg as before. The step cap's and a run cancel's signals are forwarded to the agent's group. | | [`post-review.py`](post-review.py) | Reads the judge's consolidated findings and posts **one** PR review with line-anchored inline comments and severity badges. | | [`auto-approve.py`](auto-approve.py) | Opt-in auto-approve (`approve_max_severity`): `decide` (in `post-review`) approves / requests changes / reaches no decision (withdrawing its own earlier approvals and leaving one standing REQUEST_CHANGES with the reasons and the next step, replaced by the next no-decision round and dismissed by the next approving one; none on a `capped` round, which withdraws the earlier one) per its fail-closed rules, then writes cursor-approve's status card for the outcome (via [`../cursor-approve/card.py`](../cursor-approve/card.py); no card for an author `approve_authors` does not list); `dismiss-stale` (the **Dismiss stale auto-approval** job) withdraws the bot's own approvals that are stale against the PR's live head and base (a push, a reopen, a retarget) — and every one of them while the PR carries `skip-cursor-review` or `needs-human-review` — plus its own standing REQUEST_CHANGES while the PR carries either label (re-read after the reviews are listed) or `approve_max_severity` is empty (then only once none of its approvals still stands), on every event of an open PR, whether or not `approve_max_severity` is still set; `round-cap` (the **Round cap** job, ahead of the panel) enforces `max_rounds`. On an APPROVE that survived its post-write re-check, `decide` also resolves (re-reading each first) and then replies on the posting identity's own at-or-below-threshold threads nobody else commented in (nothing, if any live thread is above the threshold); [`build-ledger.py`](build-ledger.py) never counts that reply as an answer. `decide` also writes the `approve_gate` output, and `round-cap` feeds the `round` and `max_rounds` outputs. Reads open-thread severity through `gate-unresolved.py`'s query. See [`docs/callers/cursor-review.md`](../../docs/callers/cursor-review.md#auto-approve). | | [`gate-unresolved.py`](gate-unresolved.py) | The opt-in blocking gate (`blocking: true` → the **Blocking gate** job): queries the PR's review threads and exits non-zero while any cursor-review finding thread is unresolved and non-outdated. Dropped from `cursor-review.yml` by accident in #31 and restored by BE-4691 — see [the blocking section](#optional-make-the-review-blocking). Double-billed: [`build-ledger.py`](build-ledger.py) also imports it for `CONSOLIDATED_MARKER`, the paging `reviewThreads` GraphQL query and the `iter_threads` / `is_cursor_thread` helpers, so the gate and the ledger can never disagree about which threads are ours. | diff --git a/.github/cursor-review/stop-on-submit.py b/.github/cursor-review/stop-on-submit.py new file mode 100644 index 00000000..2ec8946e --- /dev/null +++ b/.github/cursor-review/stop-on-submit.py @@ -0,0 +1,140 @@ +#!/usr/bin/env python3 +"""Run a panel cell's agent, and end it once its findings.json says status=ok. + +A reviewer cell's work is done the moment `cursor_review_finish` writes +`status: ok` to its findings.json (review-output-mcp.py), but cursor-agent does +not always exit then: some cells sat on a finished review until the 15-minute +`Run cursor review` step cap killed them. This wrapper runs the agent command +and polls the findings file while it runs. Once the file exists, parses as a +JSON object and carries the exact string `"status": "ok"`, the agent gets +`--linger` seconds to exit on its own, then SIGTERM to its whole process group, +then SIGKILL after `--grace` seconds. With the defaults (5s poll, 15s linger, +10s grace) a submitted cell is gone within 30 seconds of its submission. + +What it deliberately does NOT do: + * touch the findings file — it only reads it, so the bytes the agent's MCP + tool wrote are exactly the bytes the upload, the leg check and the judge + see; + * stop anything that has not submitted — a missing, partial, unparseable, + non-object or non-`ok` file is "keep running", so a cell that never + submits still runs to the step cap and is reported exactly as before. + +Exit status: the agent's own exit code when it exits by itself (before or +after submitting); 0 when this wrapper stopped it after a verified submission. +Status lines go to stderr with a `stop-on-submit:` prefix. + +Usage: + stop-on-submit.py --findings /tmp/findings-out/findings.json -- cursor-agent ... +""" + +import argparse +import json +import os +import signal +import subprocess +import sys +import time + +FORWARDED_SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) + + +def log(message): + print(f"stop-on-submit: {message}", file=sys.stderr, flush=True) + + +def submitted(path): + """True only for a complete, parseable JSON object with status == "ok". + + The file is agent-writable (the agent runs `--trust` with a shell), so + every failure mode reads as "not submitted" rather than raising: + `ValueError` covers both JSONDecodeError and the UnicodeDecodeError that + invalid UTF-8 raises; RecursionError covers pathologically nested input. + """ + try: + with open(path, encoding="utf-8") as source: + record = json.load(source) + except (OSError, ValueError, RecursionError): + return False + return isinstance(record, dict) and record.get("status") == "ok" + + +def signal_group(pgid, signum): + try: + os.killpg(pgid, signum) + except (ProcessLookupError, PermissionError): + pass + + +def wait(child, timeout): + """The child's exit code, or None if it is still running after `timeout`.""" + try: + return child.wait(timeout=timeout) + except subprocess.TimeoutExpired: + return None + + +def run(command, findings, poll, linger, grace): + # Own process group, so a stop reaches cursor-agent's children too (the + # MCP server, any shell it spawned) — not just the top-level process. + child = subprocess.Popen(command, start_new_session=True) + pgid = child.pid + + # The step cap and a run cancel signal THIS process; pass them on, so a cell + # that never submits still dies at the cap exactly as it did unwrapped. + def forward(signum, _frame): + signal_group(pgid, signum) + + for signum in FORWARDED_SIGNALS: + signal.signal(signum, forward) + + while True: + code = wait(child, poll) + if code is not None: + return code + if not submitted(findings): + continue + log(f"findings report status=ok; giving the agent {linger:g}s to exit on its own") + code = wait(child, linger) + if code is not None: + return code + # Re-read before signalling: a status that has stopped being ok since + # (the file is agent-writable) is not a submission to stop on. + if submitted(findings): + break + log("findings no longer report status=ok; continuing to wait") + + started = time.monotonic() + log("agent still running after submitting; sending SIGTERM") + signal_group(pgid, signal.SIGTERM) + if wait(child, grace) is None: + log(f"agent did not exit within {grace:g}s of SIGTERM; sending SIGKILL") + # SIGKILL the group even when the top-level process exited: a grandchild + # that ignored SIGTERM must not outlive the step and rewrite the findings. + signal_group(pgid, signal.SIGKILL) + child.wait() + log(f"agent stopped {time.monotonic() - started:.1f}s after SIGTERM; findings left as written") + return 0 + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + parser.add_argument("--findings", required=True) + parser.add_argument("--poll", type=float, default=5.0) + parser.add_argument("--linger", type=float, default=15.0) + parser.add_argument("--grace", type=float, default=10.0) + parser.add_argument("command", nargs=argparse.REMAINDER) + args = parser.parse_args(argv) + command = args.command[1:] if args.command[:1] == ["--"] else args.command + if not command: + parser.error("no command given after --") + for name in ("poll", "linger", "grace"): + if getattr(args, name) <= 0: + parser.error(f"--{name} must be positive") + code = run(command, args.findings, args.poll, args.linger, args.grace) + # A signal death reads as the shell would report it (143 for SIGTERM), not + # as Python's negative returncode wrapped mod 256. + return 128 - code if code < 0 else code + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/.github/cursor-review/tests/test_stop_on_submit.py b/.github/cursor-review/tests/test_stop_on_submit.py new file mode 100644 index 00000000..e8bad8e2 --- /dev/null +++ b/.github/cursor-review/tests/test_stop_on_submit.py @@ -0,0 +1,286 @@ +#!/usr/bin/env python3 +"""Tests for stop-on-submit.py, the wrapper that ends a submitted panel cell. + +Two layers. `submitted()` is the trigger, so it is pinned case by case: only a +complete JSON object whose status is the exact string "ok" may stop a cell. +The end-to-end tests run the real wrapper against a stub agent (a Python child +that writes findings.json and then hangs, the way a real stalled cell does) +with sub-second timings, standing in for the 15-minute step cap with a short +subprocess timeout. + +Run: python3 -m unittest discover -s .github/cursor-review/tests -p 'test_*.py' -v +""" + +import importlib.util +import json +import os +import signal +import subprocess +import sys +import tempfile +import textwrap +import time +import unittest + +SCRIPT = os.path.normpath( + os.path.join(os.path.dirname(__file__), "..", "stop-on-submit.py") +) +SPEC = importlib.util.spec_from_file_location("stop_on_submit", SCRIPT) +stop_on_submit = importlib.util.module_from_spec(SPEC) +SPEC.loader.exec_module(stop_on_submit) + +OK = {"status": "ok", "model": "m", "review_type": "adversarial", "findings": []} +TIMINGS = ["--poll", "0.1", "--linger", "0.3", "--grace", "0.5"] + + +class SubmittedTest(unittest.TestCase): + def setUp(self): + self.dir = tempfile.TemporaryDirectory() + self.path = os.path.join(self.dir.name, "findings.json") + + def tearDown(self): + self.dir.cleanup() + + def write(self, data): + with open(self.path, "wb") as f: + f.write(data if isinstance(data, bytes) else data.encode("utf-8")) + + def test_status_ok_is_submitted(self): + self.write(json.dumps(OK)) + self.assertTrue(stop_on_submit.submitted(self.path)) + + def test_never_written_is_not_submitted(self): + self.assertFalse(stop_on_submit.submitted(self.path)) + + def test_error_status_is_not_submitted(self): + # The pre-seeded artifact, and the shape record_finding leaves mid-review. + self.write(json.dumps({"status": "error", "error": "x", "findings": []})) + self.assertFalse(stop_on_submit.submitted(self.path)) + + def test_partial_write_is_not_submitted(self): + whole = json.dumps(OK) + for cut in range(len(whole)): + self.write(whole[:cut]) + self.assertFalse(stop_on_submit.submitted(self.path), whole[:cut]) + + def test_non_object_and_non_string_status_are_not_submitted(self): + for body in ('["ok"]', '"ok"', '{"status": true}', '{"status": "OK"}', + '{"status": "ok "}', '{"findings": []}', "null"): + self.write(body) + self.assertFalse(stop_on_submit.submitted(self.path), body) + + def test_invalid_utf8_and_deep_nesting_are_not_submitted(self): + self.write(b'{"status": "ok", "x": "\xff"}') + self.assertFalse(stop_on_submit.submitted(self.path)) + self.write("[" * 100000 + "]" * 100000) + self.assertFalse(stop_on_submit.submitted(self.path)) + + def test_a_directory_is_not_submitted(self): + os.mkdir(self.path) + self.assertFalse(stop_on_submit.submitted(self.path)) + + +class WrapperTest(unittest.TestCase): + def setUp(self): + self.dir = tempfile.TemporaryDirectory() + self.path = os.path.join(self.dir.name, "findings.json") + self.pidfile = os.path.join(self.dir.name, "pids") + + def tearDown(self): + # Never leave a stub behind, whatever a failing assertion skipped. + self.kill_stubs() + self.dir.cleanup() + + def kill_stubs(self): + if os.path.exists(self.pidfile): + with open(self.pidfile) as f: + for line in f: + try: + os.kill(int(line), signal.SIGKILL) + except (ProcessLookupError, ValueError): + pass + + def agent(self, body): + """A stub agent: records its pid, then runs `body` (Python).""" + prelude = textwrap.dedent(f"""\ + import json, os, signal, subprocess, sys, time + PATH = {self.path!r} + def record_pid(pid): + with open({self.pidfile!r}, "a") as f: + f.write(f"{{pid}}\\n") + record_pid(os.getpid()) + def write(text): + tmp = PATH + ".tmp" + with open(tmp, "w") as f: + f.write(text) + os.replace(tmp, PATH) + """) + return [sys.executable, "-c", prelude + textwrap.dedent(body)] + + def wrap(self, body, *, cap=None, extra=TIMINGS): + return subprocess.Popen( + [sys.executable, SCRIPT, "--findings", self.path, *extra, "--", + *self.agent(body)], + stderr=subprocess.PIPE, + text=True, + ) + + def finish(self, proc, cap): + """(exit code, stderr, seconds), or (None, …) if still up at `cap`.""" + started = time.monotonic() + try: + _, err = proc.communicate(timeout=cap) + return proc.returncode, err, time.monotonic() - started + except subprocess.TimeoutExpired: + # Still up at the stand-in cap. Kill the stubs first: they inherit + # the stderr pipe, so communicate() would otherwise wait them out. + proc.kill() + self.kill_stubs() + _, err = proc.communicate() + return None, err, time.monotonic() - started + + def pids(self): + with open(self.pidfile) as f: + return [int(line) for line in f] + + def assert_gone(self, pid): + deadline = time.monotonic() + 2 + while time.monotonic() < deadline: + try: + os.kill(pid, 0) + except ProcessLookupError: + return + time.sleep(0.05) + self.fail(f"pid {pid} is still running") + + def test_ok_cell_is_stopped_early_and_findings_are_untouched(self): + proc = self.wrap(f"""\ + write({json.dumps(json.dumps(OK))}) + time.sleep(600) + """) + code, err, took = self.finish(proc, cap=10) + self.assertEqual(code, 0, err) + self.assertLess(took, 5) + self.assertIn("sending SIGTERM", err) + with open(self.path, "rb") as f: + self.assertEqual(f.read(), json.dumps(OK).encode("utf-8")) + self.assert_gone(self.pids()[0]) + + def test_findings_bytes_are_exactly_what_the_agent_wrote(self): + # Odd formatting and key order survive: the wrapper never rewrites. + raw = '{ "findings":[{"file":"a.py","line":1}] ,\n "status" : "ok"}\n' + proc = self.wrap(f"""\ + write({raw!r}) + time.sleep(600) + """) + code, err, _ = self.finish(proc, cap=10) + self.assertEqual(code, 0, err) + with open(self.path) as f: + self.assertEqual(f.read(), raw) + + def test_not_ok_cell_runs_to_the_cap(self): + proc = self.wrap("""\ + write(json.dumps({"status": "error", "findings": []})) + time.sleep(600) + """) + code, err, _ = self.finish(proc, cap=3) + self.assertIsNone(code, f"wrapper exited ({code}) on a non-ok cell: {err}") + self.assertNotIn("SIGTERM", err) + + def test_partial_write_does_not_stop_the_cell(self): + # Written in place (no rename), and never completed. + proc = self.wrap(f"""\ + with open(PATH, "w") as f: + f.write({json.dumps(OK)[:-5]!r}) + time.sleep(600) + """) + code, err, _ = self.finish(proc, cap=3) + self.assertIsNone(code, err) + + def test_never_written_runs_to_the_cap(self): + proc = self.wrap("time.sleep(600)\n") + code, err, _ = self.finish(proc, cap=3) + self.assertIsNone(code, err) + + def test_agent_exit_code_passes_through(self): + for status in ("error", "ok"): + with self.subTest(status=status): + proc = self.wrap(f"""\ + write(json.dumps({{"status": {status!r}}})) + sys.exit(3) + """) + code, err, _ = self.finish(proc, cap=10) + self.assertEqual(code, 3, err) + self.assertNotIn("SIGTERM", err) + + def test_agent_that_exits_during_the_linger_is_not_signalled(self): + proc = self.wrap(f"""\ + write({json.dumps(json.dumps(OK))}) + time.sleep(0.2) + sys.exit(0) + """, extra=["--poll", "0.05", "--linger", "5", "--grace", "1"]) + code, err, _ = self.finish(proc, cap=10) + self.assertEqual(code, 0, err) + self.assertNotIn("SIGTERM", err) + + def test_status_that_reverts_during_the_linger_is_not_stopped(self): + proc = self.wrap(f"""\ + write({json.dumps(json.dumps(OK))}) + time.sleep(0.1) + write(json.dumps({{"status": "error"}})) + time.sleep(600) + """, extra=["--poll", "0.05", "--linger", "0.5", "--grace", "0.5"]) + code, err, _ = self.finish(proc, cap=3) + self.assertIsNone(code, err) + + def test_sigterm_ignoring_agent_and_its_children_are_killed(self): + proc = self.wrap(f"""\ + signal.signal(signal.SIGTERM, signal.SIG_IGN) + kid = subprocess.Popen([sys.executable, "-c", + "import signal, time; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(600)"]) + record_pid(kid.pid) + write({json.dumps(json.dumps(OK))}) + time.sleep(600) + """) + code, err, took = self.finish(proc, cap=10) + self.assertEqual(code, 0, err) + self.assertIn("sending SIGKILL", err) + self.assertLess(took, 5) + for pid in self.pids(): + self.assert_gone(pid) + + def test_signals_to_the_wrapper_reach_the_agent(self): + # The step cap and a run cancel signal the wrapper, not the agent's + # separate process group; the agent must still die with it. + proc = self.wrap("time.sleep(600)\n") + deadline = time.monotonic() + 5 + while not os.path.exists(self.pidfile) and time.monotonic() < deadline: + time.sleep(0.05) + proc.send_signal(signal.SIGTERM) + code, err, _ = self.finish(proc, cap=5) + self.assertEqual(code, 128 + signal.SIGTERM, err) + self.assert_gone(self.pids()[0]) + + def test_missing_command_is_a_usage_error(self): + result = subprocess.run( + [sys.executable, SCRIPT, "--findings", self.path, "--"], + capture_output=True, text=True, + ) + self.assertEqual(result.returncode, 2) + + +class WorkflowWiringTest(unittest.TestCase): + def test_run_step_wraps_the_reviewer_agent(self): + workflow = os.path.normpath(os.path.join( + os.path.dirname(__file__), "..", "..", "workflows", "cursor-review.yml")) + with open(workflow, encoding="utf-8") as f: + text = f.read() + step = text.split(" - name: Run cursor review\n", 1)[1].split("\n - name:", 1)[0] + self.assertIn('python3 "$CURSOR_REVIEW_ASSETS/stop-on-submit.py"', step) + self.assertIn("--findings /tmp/findings-out/findings.json --", step) + self.assertLess(step.index("stop-on-submit.py"), step.index("cursor-agent \\")) + self.assertIn("timeout-minutes: 15", step) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index 197a3abc..e9521a88 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -2472,6 +2472,15 @@ jobs: # the PR's rollup instead of a silent hole in the panel count. Keeping # the absorption HERE rather than dropping `continue-on-error` is what # lets the artifact upload still run, so the panel keeps consolidating. + # + # The cap is for cells that never submit. A cell that HAS submitted — + # its findings.json parses and reads `status: ok` — is ended by + # stop-on-submit.py within 30 seconds of the submission instead of + # idling to the cap (some agents keep running after + # `cursor_review_finish`). The wrapper only READS findings.json, so the + # artifact is byte-for-byte what the MCP tool wrote; it exits 0 on that + # path, and a cell that never reaches `status: ok` is not stopped, so + # it still runs to this cap and fails its leg exactly as before. continue-on-error: true timeout-minutes: 15 env: @@ -2487,12 +2496,15 @@ jobs: # # Capture the exit code without tripping the shell's `set -e`. If the # agent fails before finishing its MCP submission, the pre-seeded - # structured artifact remains status=error. - cursor-agent \ - --print \ - --trust \ - --approve-mcps \ - --model "$MODEL" \ + # structured artifact remains status=error. stop-on-submit.py writes + # its own `stop-on-submit:` status lines into the stderr capture. + python3 "$CURSOR_REVIEW_ASSETS/stop-on-submit.py" \ + --findings /tmp/findings-out/findings.json -- \ + cursor-agent \ + --print \ + --trust \ + --approve-mcps \ + --model "$MODEL" \ < /tmp/prompt.txt \ > /tmp/review-raw.txt 2>/tmp/review-stderr.txt \ && CURSOR_EXIT=0 || CURSOR_EXIT=$? From 593cef287f5d8113807809277cf02071d64c04a0 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Thu, 8 Oct 2026 07:28:01 +0000 Subject: [PATCH 2/2] fix(cursor-review): forward the step's signals to stop-on-submit and reap the agent group on every exit Review round on #379: - the `Run cursor review` shell now runs the wrapper in the background and traps INT/TERM/HUP to forward them, so the step cap / a cancel no longer kills bash alone and leaves the wrapper and agent running past the step; a signalled step still exits 128+signal - the wrapper installs its handlers before spawning, and after a forwarded signal gives the group at most 5s, SIGKILLs it and exits 128+signal even when the agent traps the signal and exits 0 - SIGKILL to the agent's group on every exit path (try/finally), including the agent exiting by itself - submitted() reads only a regular file (O_NONBLOCK|O_NOFOLLOW + fstat), capped at 8 MiB, and catches MemoryError - --poll/--linger/--grace reject nan/inf Co-Authored-By: Claude Opus 5.5 --- .github/cursor-review/README.md | 2 +- .github/cursor-review/stop-on-submit.py | 122 ++++++++++++++---- .../tests/test_stop_on_submit.py | 76 +++++++++++ .github/workflows/cursor-review.yml | 29 ++++- 4 files changed, 203 insertions(+), 26 deletions(-) diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 5e1081b9..16d55264 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -217,7 +217,7 @@ the consolidated review's panel table. See [Panel integrity](#panel-integrity). | [`prompt-edge-case.md`](prompt-edge-case.md) | Prompt for the correctness/logic review pass. | | [`prompt-judge.md`](prompt-judge.md) | Prompt the judge model uses to consolidate panel findings into one review. | | [`review-output-mcp.py`](review-output-mcp.py) | Dependency-free stdio MCP server. Reviewers record individual findings and finish; the judge submits the final findings array. It validates and atomically writes the normalized records consumed by later jobs. | -| [`stop-on-submit.py`](stop-on-submit.py) | Wraps each panel cell's `cursor-agent` and polls its `findings.json`. Once the file parses as a JSON object with `"status": "ok"`, the agent gets 15s to exit by itself, then SIGTERM to its process group, then SIGKILL 10s later, so a cell that has submitted exits **within 30 seconds** of submitting instead of idling to the 15-minute step cap. It only reads the file, so the artifact is byte-for-byte what the MCP tool wrote, and the stop exits 0, so the leg stays green. A missing, partial, unparseable or non-`ok` file never triggers a stop, so a cell that never submits still runs to the cap and fails its leg as before. The step cap's and a run cancel's signals are forwarded to the agent's group. | +| [`stop-on-submit.py`](stop-on-submit.py) | Wraps each panel cell's `cursor-agent` and polls its `findings.json`. Once the file parses as a JSON object with `"status": "ok"`, the agent gets 15s to exit by itself, then SIGTERM to its process group, then SIGKILL 10s later, so a cell that has submitted exits **within 30 seconds** of submitting instead of idling to the 15-minute step cap. It only reads the file, so the artifact is byte-for-byte what the MCP tool wrote, and the stop exits 0, so the leg stays green. A missing, partial, unparseable or non-`ok` file never triggers a stop, so a cell that never submits still runs to the cap and fails its leg as before. The step cap's and a run cancel's signals reach it through the step shell's traps and are forwarded to the agent's group, which is SIGKILLed at most 5s later (exit 128+signal). Every exit path, including an agent that exits by itself, ends with SIGKILL to the group, so nothing it left running can rewrite the findings after the upload. Only a regular file of at most 8 MiB is read (no FIFO/symlink/device). | | [`post-review.py`](post-review.py) | Reads the judge's consolidated findings and posts **one** PR review with line-anchored inline comments and severity badges. | | [`auto-approve.py`](auto-approve.py) | Opt-in auto-approve (`approve_max_severity`): `decide` (in `post-review`) approves / requests changes / reaches no decision (withdrawing its own earlier approvals and leaving one standing REQUEST_CHANGES with the reasons and the next step, replaced by the next no-decision round and dismissed by the next approving one; none on a `capped` round, which withdraws the earlier one) per its fail-closed rules, then writes cursor-approve's status card for the outcome (via [`../cursor-approve/card.py`](../cursor-approve/card.py); no card for an author `approve_authors` does not list); `dismiss-stale` (the **Dismiss stale auto-approval** job) withdraws the bot's own approvals that are stale against the PR's live head and base (a push, a reopen, a retarget) — and every one of them while the PR carries `skip-cursor-review` or `needs-human-review` — plus its own standing REQUEST_CHANGES while the PR carries either label (re-read after the reviews are listed) or `approve_max_severity` is empty (then only once none of its approvals still stands), on every event of an open PR, whether or not `approve_max_severity` is still set; `round-cap` (the **Round cap** job, ahead of the panel) enforces `max_rounds`. On an APPROVE that survived its post-write re-check, `decide` also resolves (re-reading each first) and then replies on the posting identity's own at-or-below-threshold threads nobody else commented in (nothing, if any live thread is above the threshold); [`build-ledger.py`](build-ledger.py) never counts that reply as an answer. `decide` also writes the `approve_gate` output, and `round-cap` feeds the `round` and `max_rounds` outputs. Reads open-thread severity through `gate-unresolved.py`'s query. See [`docs/callers/cursor-review.md`](../../docs/callers/cursor-review.md#auto-approve). | | [`gate-unresolved.py`](gate-unresolved.py) | The opt-in blocking gate (`blocking: true` → the **Blocking gate** job): queries the PR's review threads and exits non-zero while any cursor-review finding thread is unresolved and non-outdated. Dropped from `cursor-review.yml` by accident in #31 and restored by BE-4691 — see [the blocking section](#optional-make-the-review-blocking). Double-billed: [`build-ledger.py`](build-ledger.py) also imports it for `CONSOLIDATED_MARKER`, the paging `reviewThreads` GraphQL query and the `iter_threads` / `is_cursor_thread` helpers, so the gate and the ledger can never disagree about which threads are ours. | diff --git a/.github/cursor-review/stop-on-submit.py b/.github/cursor-review/stop-on-submit.py index 2ec8946e..3c0767d5 100644 --- a/.github/cursor-review/stop-on-submit.py +++ b/.github/cursor-review/stop-on-submit.py @@ -17,10 +17,19 @@ see; * stop anything that has not submitted — a missing, partial, unparseable, non-object or non-`ok` file is "keep running", so a cell that never - submits still runs to the step cap and is reported exactly as before. + submits still runs to the step cap and is reported exactly as before; + * outlive its agent's process group — every exit path, including the agent + exiting by itself and an exception here, ends with SIGKILL to the group, + so nothing the agent left running can rewrite the findings afterwards. + +A SIGINT/SIGTERM/SIGHUP to this wrapper (the step cap, a run cancel) is passed +to the agent's group; the agent then gets at most 5 seconds before SIGKILL — +inside the runner's own SIGINT -> SIGTERM -> kill escalation — and the wrapper +exits 128+signal whatever the agent's own status was. Exit status: the agent's own exit code when it exits by itself (before or -after submitting); 0 when this wrapper stopped it after a verified submission. +after submitting); 0 when this wrapper stopped it after a verified submission; +128+signal when a forwarded signal ended it. Status lines go to stderr with a `stop-on-submit:` prefix. Usage: @@ -29,13 +38,24 @@ import argparse import json +import math import os import signal +import stat import subprocess import sys import time FORWARDED_SIGNALS = (signal.SIGINT, signal.SIGTERM, signal.SIGHUP) +# A real findings.json is at most a few hundred KB (review-output-mcp.py caps +# each finding); anything larger is not a submission, and never read whole. +MAX_FINDINGS_BYTES = 8 * 1024 * 1024 +# Upper bound on the agent's grace once the wrapper itself is signalled: the +# runner escalates SIGINT -> SIGTERM 7.5s later -> kill, and the group must be +# SIGKILLed before the wrapper is, or the agent outlives the step. +SIGNAL_GRACE = 5.0 +# How often a wait re-checks for a forwarded signal. +WAIT_SLICE = 0.1 def log(message): @@ -46,14 +66,26 @@ def submitted(path): """True only for a complete, parseable JSON object with status == "ok". The file is agent-writable (the agent runs `--trust` with a shell), so - every failure mode reads as "not submitted" rather than raising: - `ValueError` covers both JSONDecodeError and the UnicodeDecodeError that - invalid UTF-8 raises; RecursionError covers pathologically nested input. + every failure mode reads as "not submitted" rather than raising or + blocking: only a regular file is read (O_NONBLOCK|O_NOFOLLOW, so a FIFO or + a symlink to a device cannot stall or flood the poll), at most + MAX_FINDINGS_BYTES of it; `ValueError` covers both JSONDecodeError and the + UnicodeDecodeError that invalid UTF-8 raises; RecursionError covers + pathologically nested input. """ try: - with open(path, encoding="utf-8") as source: - record = json.load(source) - except (OSError, ValueError, RecursionError): + fd = os.open(path, os.O_RDONLY | os.O_NONBLOCK | os.O_NOFOLLOW) + except OSError: + return False + try: + with open(fd, "rb") as source: + if not stat.S_ISREG(os.fstat(source.fileno()).st_mode): + return False + data = source.read(MAX_FINDINGS_BYTES + 1) + if len(data) > MAX_FINDINGS_BYTES: + return False + record = json.loads(data.decode("utf-8")) + except (OSError, ValueError, RecursionError, MemoryError): return False return isinstance(record, dict) and record.get("status") == "ok" @@ -65,36 +97,79 @@ def signal_group(pgid, signum): pass -def wait(child, timeout): - """The child's exit code, or None if it is still running after `timeout`.""" - try: - return child.wait(timeout=timeout) - except subprocess.TimeoutExpired: - return None +def wait(child, timeout, interrupted=lambda: False): + """The child's exit code, or None if it is still running after `timeout` + (or as soon as `interrupted()` turns true).""" + deadline = time.monotonic() + timeout + while True: + try: + return child.wait(timeout=max(0.0, min(WAIT_SLICE, deadline - time.monotonic()))) + except subprocess.TimeoutExpired: + if interrupted() or time.monotonic() >= deadline: + return None def run(command, findings, poll, linger, grace): - # Own process group, so a stop reaches cursor-agent's children too (the - # MCP server, any shell it spawned) — not just the top-level process. - child = subprocess.Popen(command, start_new_session=True) - pgid = child.pid + received = [] + child = None # The step cap and a run cancel signal THIS process; pass them on, so a cell # that never submits still dies at the cap exactly as it did unwrapped. + # Installed BEFORE the spawn, so no signal can kill the wrapper in between + # and orphan the agent; one landing before `child` is set is forwarded by + # stop_on_signal instead. def forward(signum, _frame): - signal_group(pgid, signum) + received.append(signum) + if child is not None: + signal_group(child.pid, signum) for signum in FORWARDED_SIGNALS: signal.signal(signum, forward) + # Own process group, so a stop reaches cursor-agent's children too (the + # MCP server, any shell it spawned) — not just the top-level process. + child = subprocess.Popen(command, start_new_session=True) + try: + return supervise(child, findings, poll, linger, grace, received) + finally: + # Every path out — the agent exiting by itself, a stop, an exception + # here: nothing left in the agent's group may outlive the wrapper and + # rewrite the findings after the upload and leg check read them. + signal_group(child.pid, signal.SIGKILL) + + +def stop_on_signal(child, signum, grace): + """End the agent after a signal to the wrapper; report 128+signum.""" + pgid = child.pid + log(f"received signal {signum}; stopping the agent") + signal_group(pgid, signum) + if wait(child, min(grace, SIGNAL_GRACE)) is None: + log("agent did not exit after the forwarded signal; sending SIGKILL") + signal_group(pgid, signal.SIGKILL) + child.wait() + return -signum + + +def supervise(child, findings, poll, linger, grace, received): + pgid = child.pid + + def interrupted(): + return bool(received) + while True: - code = wait(child, poll) + if received: + return stop_on_signal(child, received[0], grace) + code = wait(child, poll, interrupted) + if received: + continue if code is not None: return code if not submitted(findings): continue log(f"findings report status=ok; giving the agent {linger:g}s to exit on its own") - code = wait(child, linger) + code = wait(child, linger, interrupted) + if received: + continue if code is not None: return code # Re-read before signalling: a status that has stopped being ok since @@ -128,8 +203,9 @@ def main(argv=None): if not command: parser.error("no command given after --") for name in ("poll", "linger", "grace"): - if getattr(args, name) <= 0: - parser.error(f"--{name} must be positive") + value = getattr(args, name) + if not (math.isfinite(value) and value > 0): + parser.error(f"--{name} must be a positive finite number") code = run(command, args.findings, args.poll, args.linger, args.grace) # A signal death reads as the shell would report it (143 for SIGTERM), not # as Python's negative returncode wrapped mod 256. diff --git a/.github/cursor-review/tests/test_stop_on_submit.py b/.github/cursor-review/tests/test_stop_on_submit.py index e8bad8e2..bd2a01f2 100644 --- a/.github/cursor-review/tests/test_stop_on_submit.py +++ b/.github/cursor-review/tests/test_stop_on_submit.py @@ -79,6 +79,23 @@ def test_a_directory_is_not_submitted(self): os.mkdir(self.path) self.assertFalse(stop_on_submit.submitted(self.path)) + def test_a_fifo_is_not_submitted_and_does_not_block(self): + # A blocking open() on a FIFO with no writer would hang the poll forever. + os.mkfifo(self.path) + self.assertFalse(stop_on_submit.submitted(self.path)) + + def test_a_symlink_is_not_submitted(self): + target = os.path.join(self.dir.name, "real.json") + with open(target, "w") as f: + json.dump(OK, f) + os.symlink(target, self.path) + self.assertFalse(stop_on_submit.submitted(self.path)) + + def test_an_oversized_file_is_not_submitted(self): + padding = "x" * (stop_on_submit.MAX_FINDINGS_BYTES) + self.write(json.dumps({"status": "ok", "pad": padding})) + self.assertFalse(stop_on_submit.submitted(self.path)) + class WrapperTest(unittest.TestCase): def setUp(self): @@ -261,6 +278,59 @@ def test_signals_to_the_wrapper_reach_the_agent(self): self.assertEqual(code, 128 + signal.SIGTERM, err) self.assert_gone(self.pids()[0]) + def wait_for_pids(self, count): + deadline = time.monotonic() + 5 + while time.monotonic() < deadline: + if os.path.exists(self.pidfile) and len(self.pids()) >= count: + return + time.sleep(0.05) + self.fail(f"stub never recorded {count} pid(s)") + + def test_signal_trapping_agent_is_killed_and_reported_as_signalled(self): + # An agent that traps the forwarded signal and stays up must not hold + # the wrapper (and the step) until the runner's unforwardable kill; one + # that traps it and exits 0 must not read as success either. + proc = self.wrap("""\ + signal.signal(signal.SIGTERM, signal.SIG_IGN) + kid = subprocess.Popen([sys.executable, "-c", + "import signal, time; signal.signal(signal.SIGTERM, signal.SIG_IGN); time.sleep(600)"]) + record_pid(kid.pid) + time.sleep(600) + """, extra=["--poll", "5", "--linger", "5", "--grace", "0.5"]) + self.wait_for_pids(2) + started = time.monotonic() + proc.send_signal(signal.SIGTERM) + code, err, _ = self.finish(proc, cap=10) + self.assertEqual(code, 128 + signal.SIGTERM, err) + self.assertLess(time.monotonic() - started, 3, "signal not noticed mid-poll") + self.assertIn("sending SIGKILL", err) + for pid in self.pids(): + self.assert_gone(pid) + + def test_agent_that_exits_on_its_own_leaves_nothing_behind(self): + # A grandchild still in the agent's group could rewrite findings.json + # after the upload; the wrapper reaps the group on this path too. + proc = self.wrap("""\ + kid = subprocess.Popen([sys.executable, "-c", "import time; time.sleep(600)"]) + record_pid(kid.pid) + sys.exit(4) + """) + code, err, _ = self.finish(proc, cap=10) + self.assertEqual(code, 4, err) + for pid in self.pids(): + self.assert_gone(pid) + + def test_non_finite_timings_are_a_usage_error(self): + for flag in ("--poll", "--linger", "--grace"): + for value in ("nan", "inf", "0", "-1"): + with self.subTest(flag=flag, value=value): + result = subprocess.run( + [sys.executable, SCRIPT, "--findings", self.path, flag, value, + "--", sys.executable, "-c", "pass"], + capture_output=True, text=True, + ) + self.assertEqual(result.returncode, 2, result.stderr) + def test_missing_command_is_a_usage_error(self): result = subprocess.run( [sys.executable, SCRIPT, "--findings", self.path, "--"], @@ -279,6 +349,12 @@ def test_run_step_wraps_the_reviewer_agent(self): self.assertIn('python3 "$CURSOR_REVIEW_ASSETS/stop-on-submit.py"', step) self.assertIn("--findings /tmp/findings-out/findings.json --", step) self.assertLess(step.index("stop-on-submit.py"), step.index("cursor-agent \\")) + # The runner signals the step's shell, not the wrapper: the shell must + # run the wrapper in the background and forward the signals to it. + self.assertIn("> /tmp/review-raw.txt 2>/tmp/review-stderr.txt &\n", step) + for sig in ("INT", "TERM", "HUP"): + self.assertIn(f"trap 'forward_signal {sig}' {sig}", step) + self.assertLess(step.index("trap 'forward_signal TERM'"), step.index("stop-on-submit.py\"")) self.assertIn("timeout-minutes: 15", step) diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index e9521a88..3a4da647 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -2498,6 +2498,24 @@ jobs: # agent fails before finishing its MCP submission, the pre-seeded # structured artifact remains status=error. stop-on-submit.py writes # its own `stop-on-submit:` status lines into the stderr capture. + # + # The step cap and a run cancel signal THIS shell, which would die + # without passing the signal on, leaving the wrapper and the agent's + # own process group running past the step and still able to write + # findings.json while the always() steps below read and upload it. + # So the wrapper runs in the background with the signals forwarded, + # and the shell waits it out (a trapped signal interrupts `wait`, so + # keep waiting until the wrapper is actually gone, then collect its + # real status). + CURSOR_PID= + CURSOR_SIGNALLED= + forward_signal() { + CURSOR_SIGNALLED=1 + if [ -n "$CURSOR_PID" ]; then kill "-$1" "$CURSOR_PID" 2>/dev/null || true; fi + } + trap 'forward_signal INT' INT + trap 'forward_signal TERM' TERM + trap 'forward_signal HUP' HUP python3 "$CURSOR_REVIEW_ASSETS/stop-on-submit.py" \ --findings /tmp/findings-out/findings.json -- \ cursor-agent \ @@ -2506,8 +2524,13 @@ jobs: --approve-mcps \ --model "$MODEL" \ < /tmp/prompt.txt \ - > /tmp/review-raw.txt 2>/tmp/review-stderr.txt \ - && CURSOR_EXIT=0 || CURSOR_EXIT=$? + > /tmp/review-raw.txt 2>/tmp/review-stderr.txt & + CURSOR_PID=$! + while kill -0 "$CURSOR_PID" 2>/dev/null; do + wait "$CURSOR_PID" || true + done + wait "$CURSOR_PID" && CURSOR_EXIT=0 || CURSOR_EXIT=$? + trap - INT TERM HUP echo "$CURSOR_EXIT" > /tmp/review-exit-code echo "cursor-agent exit code: $CURSOR_EXIT" @@ -2516,6 +2539,8 @@ jobs: echo "" echo "=== Stderr ===" cat /tmp/review-stderr.txt + # A signalled step still fails as one, as it did before the wrapper. + if [ -n "$CURSOR_SIGNALLED" ]; then exit "$CURSOR_EXIT"; fi - name: Show structured findings if: always()