From 3476cd7195ad52630461460a75d12fb55a56cf77 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 11:09:16 -0700 Subject: [PATCH 1/3] feat(cursor-review): make the per-cell agent cap an input (cell_timeout_minutes) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The per-cell agent cap was a hardcoded `timeout-minutes: 15` with no way for a caller to raise it, and it is now the dominant cause of red panels on the two busiest consumers. Measured over 56 panels / 372 cells on one repo: 43 cells red, and 22 of those were this cap — `exit 130`, zero findings, leg red, `Panel integrity` red with it. It is a CLIPPED TAIL, not a hang catcher: of the 329 cells that did submit, 18% finished within three minutes of the cap. The other busy consumer shows the same shape on a different lane — every `grok-4.7-high` cell that reached ~17 minutes died, 11 of 20, while the lanes with a 4-6 minute median never failed once. A cell that honestly needs 20 minutes has no way to ask for it. `cell_timeout_minutes` (number, default 15) is that knob, and the default reproduces today's four numbers exactly, so an unset input changes nothing. THE COUPLING IS NOW IN THE WORKFLOW. Three further caps have to move with the cell cap, and until now a comment asked the next editor to do it by hand ("COUPLED to `consolidate` below — bump this and you must bump those too"). That is the shape of every drift this repo has had. They are derived instead: cell agent step cell_timeout_minutes 15 cell job + 15 (checkout, CLI, artifacts) 30 judge step x 2 (reads EVERY cell's findings) 30 judge job + 10 (its own overhead) 40 Why each ordering matters, not just tidiness: a job cap at or under its own step cap turns a dead step into a CANCELLED job, which for a cell means `Fail the leg when the cell did not submit` never reads the artifact back (a red leg becomes a cancelled run) and for the judge means every panel cell is discarded instead of falling back to the degraded union-of-findings review. A cell cap raised past the judge's leaves the judge adjudicating a panel it cannot finish reading — the exact silent skew the 40/30 split was introduced to fix. Bounded 5-45 by a new preflight step that runs BEFORE the panel fans out. `type: number` buys only numeric-ness: 0, -5, 1.5 and 600 all parse, and the judge's job cap doubles whatever gets through, so a typo'd 600 would mint a 20-hour job. Bounded rather than clamped, deliberately — silently correcting a caller's 600 to 45 would review a long diff on a budget nobody chose and report success. The step also prints every derived cap, because they are no longer visible in a diff of this file. `test_workflow_cell_timeout.py` pins it from three directions: the exact expression at each of the six sites (a re-hardcoded literal is the regression), the ordering invariants evaluated across the whole admitted range rather than at the default, and the bound check by RUNNING its shell against real values. Verified it fails on both a re-hardcoded judge cap and a loosened ceiling. Documented in docs/callers/cursor-review.md and .github/cursor-review/README.md (both required — test_inputs_docs_drift.py checks that table two-way), with the "is it a tail or a hang" test spelled out so the knob is raised on evidence. Co-Authored-By: Claude Opus 5 --- .github/cursor-review/README.md | 1 + .../tests/test_stop_on_submit.py | 7 +- .../tests/test_workflow_cell_timeout.py | 230 ++++++++++++++++++ .github/workflows/cursor-review.yml | 114 +++++++-- docs/callers/cursor-review.md | 1 + 5 files changed, 328 insertions(+), 25 deletions(-) create mode 100644 .github/cursor-review/tests/test_workflow_cell_timeout.py diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 6350cd32..07166f68 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -376,6 +376,7 @@ All optional except `workflows_ref` (required, no default) — pass them under | `panel_models` | `''` (built-in list) | JSON array of Cursor model ids that **replaces** the panel list; each still runs both review types and is validated against the live catalog by preflight. For per-repo experiments (e.g. `-xhigh` vs `-max` tiers). | | `openai_direct_model` | `''` (off) | OPT-IN. An OpenAI model id reviewed through the OpenAI API directly (pinned Codex CLI, `package.json` here) as two cells (**adversarial** + **edge-case**, the Cursor cells' prompts) on GitHub-hosted runners, by default **replacing** the Cursor `gpt-*` cells (next row). Needs the `OPENAI_API_KEY` secret and a plain id (`[A-Za-z0-9._-]`); without either, preflight warns and leaves the cells off and the panel unchanged. Not part of `panel_models` (not in the Cursor catalog). Whether the cells gate depends on the next row. | | `openai_direct_replaces_cursor_openai` | `true` | When the direct cells are in effect, drop the Cursor `gpt-*` cell(s) so the OpenAI lane is reviewed once — the direct cells then **gate** like the Cursor cells they replace: counted in `Panel integrity` and the auto-approve gate, a failed one red in the rollup and named `(direct, )` in the review's "did not contribute" line. `false` runs both for a side-by-side backend comparison, with the direct cells **advisory** (findings reach the judge; never counted, never red). Preflight fails rather than leave the Cursor panel empty. | +| `cell_timeout_minutes` | `15` | Wall-clock cap on one reviewer cell's agent step (Cursor or direct). Over it the cell exits 130, uploads `status: error` with no findings, reddens its leg and fails `Panel integrity`. Shared by the cell's in-step retries, so it bounds attempts + backoff. Derives three more caps that cannot be set alone: cell job = this + 15, `Run judge` step = this × 2, judge job = that + 10. At the default: 15 / 30 / 30 / 40, today's values exactly. Preflight bounds it 5–45. | | `skip_bot_branch_prefixes` | `ci/bump- chore/refresh- auto/refresh-` | Skip the panel when the PR author is a Bot **and** the head branch starts with one of these prefixes (machine pin bumps / catalog refreshes). `''` reviews every bot PR. | | `diff_size_cap` | `5000` | Max counted changed lines (after generated-file exclusion and comment discounting); larger PRs are skipped. | | `ignore_comments` | `true` | Discount blank/comment-only lines from the size count (count-only; the panel still sees them). | diff --git a/.github/cursor-review/tests/test_stop_on_submit.py b/.github/cursor-review/tests/test_stop_on_submit.py index 271b650f..db34996c 100644 --- a/.github/cursor-review/tests/test_stop_on_submit.py +++ b/.github/cursor-review/tests/test_stop_on_submit.py @@ -653,7 +653,12 @@ def test_run_step_wraps_the_reviewer_agent(self): for sig in ("INT", "TERM", "HUP"): self.assertIn(f"trap 'forward_signal {sig}' {sig}", step) self.assertLess(step.index("trap 'forward_signal TERM'"), step.index("stop-on-submit.py\"")) - self.assertIn("timeout-minutes: 15", step) + # The cap is `cell_timeout_minutes`, not a literal: a hardcoded 15 here + # would have to be edited by anyone raising the input, and a literal + # left in the workflow is exactly the drift the derived caps removed. + self.assertIn( + "timeout-minutes: ${{ inputs.cell_timeout_minutes }}", step) + self.assertNotIn("timeout-minutes: 15", step) # Transient Cursor capacity errors are retried, named per cell. self.assertIn("--retry-delays 30,90", step) self.assertIn('--cell "$REVIEW_TYPE/$MODEL"', step) diff --git a/.github/cursor-review/tests/test_workflow_cell_timeout.py b/.github/cursor-review/tests/test_workflow_cell_timeout.py new file mode 100644 index 00000000..dfe49597 --- /dev/null +++ b/.github/cursor-review/tests/test_workflow_cell_timeout.py @@ -0,0 +1,230 @@ +#!/usr/bin/env python3 +"""Pins `cell_timeout_minutes` and the three caps DERIVED from it. + +The per-cell agent cap used to be a literal `timeout-minutes: 15`, and the +three numbers that have to move with it — the cell's job cap, the judge's step +cap, the judge's job cap — were kept in line by a COMMENT asking the next +editor to bump them by hand ("COUPLED to `consolidate` below"). That is the +shape of every drift this repo has had: correct today, silently wrong after the +one edit nobody re-read. + +Why the coupling matters rather than being tidiness: the judge reads EVERY +cell's findings at the same reasoning tier, so a cell cap raised past the +judge's leaves the judge adjudicating a panel it cannot finish reading, and a +judge JOB cap at or under its STEP cap turns a hung judge into a cancelled job, +which discards every panel cell instead of falling back to the degraded +union-of-findings review. A cell JOB cap at or under its step cap does the same +to the cell: the job dies before `Fail the leg when the cell did not submit` +can read the artifact back, so a dead cell reads as a cancelled run rather than +a red leg. + +So the arithmetic now lives in the workflow as expressions, and this file pins +it from three directions: + + * the exact expression at each of the six sites (a re-hardcoded literal + fails, which is the regression), + * the ORDERING invariants, evaluated over the whole admitted range rather + than at the default — a derivation that holds at 15 and inverts at 45 is + the bug this is for, + * the preflight bound check, by RUNNING its shell against real values, since + `type: number` admits 0, -5 and 600 and the derived judge job cap doubles + whatever gets through. + +Deliberately parsed WITHOUT PyYAML, like its sibling suites: this repo is +stdlib-only and CI installs no requirements for these tests. + +Run: python3 .github/cursor-review/tests/test_workflow_cell_timeout.py +""" + +import os +import re +import subprocess +import unittest + +WORKFLOW = os.path.normpath( + os.path.join(os.path.dirname(__file__), "..", "..", "workflows", "cursor-review.yml") +) + +FLOOR, CEILING, DEFAULT = 5, 45, 15 + +# site -> the expression that must be there. `None` for the step sites means +# "the bare input", spelled out per-site so a copy-paste between jobs fails. +CELL = "${{ inputs.cell_timeout_minutes }}" +CELL_JOB = "${{ inputs.cell_timeout_minutes + 15 }}" +JUDGE_STEP = "${{ inputs.cell_timeout_minutes * 2 }}" +JUDGE_JOB = "${{ inputs.cell_timeout_minutes * 2 + 10 }}" + +# The values today's workflow had before the input existed. The default must +# reproduce them exactly, or this stopped being a no-op refactor. +HISTORICAL = {"cell_step": 15, "cell_job": 30, "judge_step": 30, "judge_job": 40} + + +def text(): + with open(WORKFLOW, encoding="utf-8") as fh: + return fh.read() + + +def job_block(src, name): + start = src.index("\n %s:\n" % name) + 1 + rest = re.compile(r"^ [a-z0-9-]+:$", re.M).search(src, start + 1) + return src[start : rest.start()] if rest else src[start:] + + +def step_block(src, name): + start = src.index(" - name: %s\n" % name) + nxt = src.find("\n - name:", start + 10) + return src[start:nxt] if nxt != -1 else src[start:] + + +def job_level_timeout(block): + m = re.search(r"^ timeout-minutes: (.+)$", block, re.M) + return m.group(1).strip() if m else None + + +def step_level_timeout(block): + m = re.search(r"^ timeout-minutes: (.+)$", block, re.M) + return m.group(1).strip() if m else None + + +def derive(cell): + return { + "cell_step": cell, + "cell_job": cell + 15, + "judge_step": cell * 2, + "judge_job": cell * 2 + 10, + } + + +class InputDeclaration(unittest.TestCase): + def test_declared_as_a_number_defaulting_to_the_historical_cap(self): + src = text() + blk = src[src.index(" cell_timeout_minutes:") :][:2600] + self.assertRegex(blk, r"\n type: number\n") + self.assertRegex(blk, r"\n required: false\n") + self.assertRegex(blk, r"\n default: %d\n" % DEFAULT) + + def test_description_states_the_bounds_and_the_derivations(self): + # The knob is unusable without knowing what else moves with it. + src = text() + blk = src[src.index(" cell_timeout_minutes:") :][:2600] + self.assertIn("%d-%d" % (FLOOR, CEILING), blk) + for frag in ("+ 15", "x 2", "+ 10"): + self.assertIn(frag, blk, "description must spell out the derivations") + + +class DerivedSites(unittest.TestCase): + def setUp(self): + self.src = text() + + def test_cursor_cell_step_and_job(self): + blk = job_block(self.src, "review") + self.assertEqual(job_level_timeout(blk), CELL_JOB) + self.assertEqual(step_level_timeout(step_block(blk, "Run cursor review")), CELL) + + def test_direct_cell_step_and_job(self): + blk = job_block(self.src, "review-openai-direct") + self.assertEqual(job_level_timeout(blk), CELL_JOB) + self.assertEqual( + step_level_timeout(step_block(blk, "Run direct-API review")), CELL + ) + + def test_judge_step_and_job(self): + blk = job_block(self.src, "consolidate") + self.assertEqual(job_level_timeout(blk), JUDGE_JOB) + self.assertEqual(step_level_timeout(step_block(blk, "Run judge")), JUDGE_STEP) + + def test_no_literal_cap_survives_in_the_three_coupled_jobs(self): + # The regression is one site re-hardcoded while the others stay + # expressions — green CI, skewed panel. + for name in ("review", "review-openai-direct", "consolidate"): + blk = job_block(self.src, name) + for lit in (15, 30, 40): + self.assertNotIn( + "timeout-minutes: %d\n" % lit, + blk, + "%s re-hardcoded a cap; derive it from cell_timeout_minutes" % name, + ) + + +class DerivationInvariants(unittest.TestCase): + def test_default_reproduces_the_historical_caps_exactly(self): + self.assertEqual(derive(DEFAULT), HISTORICAL) + + def test_every_admitted_value_keeps_the_ordering(self): + # Checked across the range, not at the default: a derivation that holds + # at 15 and inverts at 45 is exactly what this guards. + for cell in range(FLOOR, CEILING + 1): + d = derive(cell) + with self.subTest(cell=cell): + # A job must outlive its own step, or a dead step becomes a + # cancelled job and the artifact is never read back. + self.assertGreater(d["cell_job"], d["cell_step"]) + self.assertGreater(d["judge_job"], d["judge_step"]) + # The judge must never have less model budget than one cell. + self.assertGreaterEqual(d["judge_step"], d["cell_step"]) + + +class PreflightBoundCheck(unittest.TestCase): + """Runs the guard's own shell, rather than asserting its text.""" + + @classmethod + def setUpClass(cls): + blk = step_block(text(), "Validate the cell timeout") + cls.script = blk.split(" run: |\n", 1)[1] + cls.script = "".join( + line[10:] if line.startswith(" " * 10) else line + for line in cls.script.splitlines(keepends=True) + ) + + def run_guard(self, value): + return subprocess.run( + ["bash", "-c", self.script], + env={**os.environ, "CELL_TIMEOUT": str(value)}, + capture_output=True, + text=True, + ) + + def test_it_runs_before_any_cell_spends(self): + src = text() + blk = job_block(src, "preflight") + self.assertLess( + blk.index("- name: Validate the cell timeout"), + blk.index("- name: Define panel models"), + "the bound check must precede the panel fan-out it protects", + ) + + def test_accepts_the_whole_admitted_range(self): + for v in (FLOOR, DEFAULT, CEILING): + with self.subTest(v=v): + r = self.run_guard(v) + self.assertEqual(r.returncode, 0, r.stderr) + + def test_reports_every_derived_cap_it_accepted(self): + # The derived numbers are invisible in a diff now, so the run has to + # say what they came out as. + r = self.run_guard(DEFAULT) + for v in HISTORICAL.values(): + self.assertIn(str(v), r.stdout, r.stdout) + + def test_rejects_out_of_range(self): + for v in (0, FLOOR - 1, CEILING + 1, 600): + with self.subTest(v=v): + r = self.run_guard(v) + self.assertEqual(r.returncode, 1, r.stdout) + self.assertIn("::error::", r.stdout) + + def test_rejects_what_type_number_still_admits(self): + # `type: number` does not mean "positive whole number". + for v in ("-5", "1.5", "", "15m", "1e3"): + with self.subTest(v=v): + r = self.run_guard(v) + self.assertEqual(r.returncode, 1, (v, r.stdout, r.stderr)) + self.assertIn("::error::", r.stdout) + + def test_the_guard_names_the_value_it_rejected(self): + r = self.run_guard(600) + self.assertIn("600", r.stdout) + + +if __name__ == "__main__": + unittest.main() diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index f6467082..813245e5 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -455,6 +455,30 @@ on: type: number required: false default: 0 + cell_timeout_minutes: + description: >- + Wall-clock cap, in minutes, on ONE reviewer cell's agent step — + `Run cursor review` for a Cursor cell, `Run direct-API review` for a + direct-OpenAI one. 15 (default) is the historical value, so an unset + input changes nothing. A cell that runs out of clock exits 130, + uploads `status: error` with no findings, fails its leg and so fails + `Panel integrity` — the cap is a guillotine, not a safety net, and + raising it is the only way to review a diff that honestly needs + longer. It is also the budget a cell's RETRIES share + (`resource_exhausted` is retried in-step), so it bounds attempts plus + backoff, not one attempt. Three further caps are DERIVED from this and + cannot be set separately, which is what keeps the panel and its judge + in step: each cell's JOB cap is this + 15 (checkout, CLI install, + artifact up/download — overhead outside the agent step), the judge's + `Run judge` step cap is this x 2 (it reads EVERY cell's findings at the + same reasoning tier, so it needs more than any one of them), and the + judge's job cap is its step + 10. At the default that reproduces + today's 30 / 15 / 30 / 40 exactly. Bounded 5-45 by preflight, which + fails before any cell spends rather than letting a typo mint an + hours-long job. + type: number + required: false + default: 15 max_rounds: description: >- Cap on review rounds per PR. Before the panel starts, the run counts @@ -2006,6 +2030,36 @@ jobs: # ('false') they stay advisory. openai_direct_counts: ${{ steps.models.outputs.openai_direct_counts }} steps: + - name: Validate the cell timeout + # FIRST, and before a single cell spends. `type: number` buys only + # numeric-ness: 0, -5 and 600 all parse. The three derived caps make a + # bad value worse than it looks — the judge's job cap is 2x + 10, so a + # typo'd 600 mints a 20-hour job and a 0 makes every cell fail + # instantly with no clue why. Neither is visible in a diff of this + # file, because the numbers are expressions now. + # + # Bounded rather than clamped, deliberately: silently correcting a + # caller's 600 to 45 would review a long diff on a budget nobody chose + # and report success. The floor is 5 because a cell has never submitted + # faster than ~1 minute and the retry backoff alone spends 2; the + # ceiling is 45 because that is already a 100-minute judge job, and + # anything needing more is a diff-size problem, not a timeout problem. + env: + CELL_TIMEOUT: ${{ inputs.cell_timeout_minutes }} + run: | + set -euo pipefail + case "$CELL_TIMEOUT" in + ''|*[!0-9]*) + echo "::error::cell_timeout_minutes must be a whole number of minutes between 5 and 45; got '${CELL_TIMEOUT}'." + exit 1 + ;; + esac + if [ "$CELL_TIMEOUT" -lt 5 ] || [ "$CELL_TIMEOUT" -gt 45 ]; then + echo "::error::cell_timeout_minutes must be between 5 and 45; got ${CELL_TIMEOUT}. The judge's job cap is twice this plus 10, so the ceiling is what keeps a typo from minting an hours-long job." + exit 1 + fi + echo "::notice::Cell agent cap ${CELL_TIMEOUT}m; cell job $((CELL_TIMEOUT + 15))m; judge step $((CELL_TIMEOUT * 2))m; judge job $((CELL_TIMEOUT * 2 + 10))m." + - name: Define panel models id: models # Single source of truth for the per-lab model pins. The `review` @@ -2322,14 +2376,17 @@ jobs: needs: [gate, diff-size, preflight, ledger] if: needs.gate.outputs.should_run == 'true' && needs.gate.outputs.already_reviewed != 'true' && needs.diff-size.outputs.within_cap == 'true' runs-on: ${{ fromJSON(inputs.runs_on || '"ubuntu-latest"') }} - # COUPLED to `consolidate` below — bump this and you must bump those too. - # The judge reads all of this panel's artifacts at the same reasoning - # tier, so it needs at least a cell's budget: its `Run judge` step cap - # tracks THIS number, and its job cap sits above the step so a hung judge - # trips the step (degraded fallback still posts) rather than the job (all - # panel cells discarded). Letting this cap outgrow them recreates exactly - # the silent skew that fix was for. - timeout-minutes: 30 + # DERIVED from `cell_timeout_minutes`, and deliberately not settable alone. + # This coupling used to be a comment telling the next editor to bump three + # other numbers by hand: the judge reads all of this panel's artifacts at + # the same reasoning tier, so it needs at least a cell's budget, and its job + # cap must sit above its step cap so a hung judge trips the step (degraded + # fallback still posts) rather than the job (all panel cells discarded). A + # cell cap that outgrew them recreated exactly the silent skew that fix was + # for — so the arithmetic lives in the workflow now, where it cannot be + # half-applied. The +15 is this job's non-agent overhead: checkout, CLI + # install, artifact download and upload, none of which the step cap covers. + timeout-minutes: ${{ inputs.cell_timeout_minutes + 15 }} permissions: contents: read strategy: @@ -2587,15 +2644,16 @@ jobs: echo "Prompt size: $(wc -c < /tmp/prompt.txt) bytes" - name: Run cursor review - # Step cap, absorbed by continue-on-error (same idiom as the judge - # step): healthy cells finish in 3-7 min and every cell's p90 is under - # 12 min, but a stalled agent — almost always one that never reaches - # its MCP submission — used to run to the 30-min JOB timeout, which - # GitHub reports as a cancelled job and a cancelled run, and stretched - # the whole review to 35+ min. Killing the step instead leaves the - # pre-seeded status=error artifact for the judge to count as a - # non-contributing cell, exactly what the job timeout produced, but - # 15 minutes sooner and without CANCELLING the run. + # Step cap (`cell_timeout_minutes`, default 15), absorbed by + # continue-on-error (same idiom as the judge step): healthy cells finish + # in 3-7 min and every cell's p90 is under 12 min, but a stalled agent — + # almost always one that never reaches its MCP submission — used to run + # to the JOB timeout, which GitHub reports as a cancelled job and a + # cancelled run, and stretched the whole review past it. Killing the + # step instead leaves the pre-seeded status=error artifact for the judge + # to count as a non-contributing cell, exactly what the job timeout + # produced, but a cap earlier and without CANCELLING the run. The job + # cap is derived as this + 15, so that ordering holds at every value. # # The cap is what this `continue-on-error` absorbs, and that is all it # absorbs: the LEG is not green on this path. `Fail the leg when the @@ -2618,14 +2676,14 @@ jobs: # agent that exits non-zero with `RetriableError: [resource_exhausted]` # (~25s in, before recording anything) is re-run after 30s, then 90s. # Only a cell with no recorded findings is re-run, and no retry starts - # later than 300s in, so the last attempt still has ~10 min of this cap - # for a full review. Each retry is a `::warning::` naming the cell; a + # later than 300s in, so the last attempt still has all but ~5 min of + # this cap for a full review — which is why the cap's floor is 5. Each retry is a `::warning::` naming the cell; a # cell that exhausts them fails its leg exactly as before. A retry # re-reads the whole prompt because the wrapper rewinds its stdin, so # keep feeding it as `< /tmp/prompt.txt`: a pipe cannot be rewound, # and turns the retries off. continue-on-error: true - timeout-minutes: 15 + timeout-minutes: ${{ inputs.cell_timeout_minutes }} env: CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }} MODEL: ${{ matrix.model }} @@ -2837,7 +2895,9 @@ jobs: continue-on-error: ${{ needs.preflight.outputs.openai_direct_counts != 'true' }} # Matches the `review` job's cap so the judge's budget reasoning (see # `review`) holds for this cell too; the agent step caps lower, at 15. - timeout-minutes: 30 + # DERIVED from `cell_timeout_minutes`, the same +15 as the Cursor cells + # above and for the same overhead; see that job's note. + timeout-minutes: ${{ inputs.cell_timeout_minutes + 15 }} permissions: contents: read strategy: @@ -3054,7 +3114,7 @@ jobs: # Capped well under the job so a hung agent trips the STEP and the # seeded artifact still uploads below, rather than tripping the job and # discarding the cell. Same 15 minutes the Cursor cells get. - timeout-minutes: 15 + timeout-minutes: ${{ inputs.cell_timeout_minutes }} env: MODEL: ${{ inputs.openai_direct_model }} REVIEW_TYPE: ${{ matrix.review_type }} @@ -3238,7 +3298,10 @@ jobs: # checkout, the CLI install, three artifact downloads and the upload — # which a panel cell does not all pay, so the judge's effective MODEL # budget is the step cap, not this one. - timeout-minutes: 40 + # + # DERIVED from `cell_timeout_minutes` (step x 2, + 10 slack), so raising a + # cell's cap carries the judge with it instead of leaving it behind. + timeout-minutes: ${{ inputs.cell_timeout_minutes * 2 + 10 }} # contents: read ONLY. This job checks out PR-authored code and runs the # judge agent over it, so it must carry nothing write-scoped — `post-review` # below holds pull-requests: write and the bot credentials instead. @@ -3509,7 +3572,10 @@ jobs: # in-script, and a judge that did not submit is reported to the PR as # `judge_status` by `Post review`. continue-on-error: true - timeout-minutes: 30 + # DERIVED: twice `cell_timeout_minutes`. The judge adjudicates EVERY + # cell's findings at the same reasoning tier, so a budget equal to one + # cell's is the floor, not the target. + timeout-minutes: ${{ inputs.cell_timeout_minutes * 2 }} env: CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }} run: | diff --git a/docs/callers/cursor-review.md b/docs/callers/cursor-review.md index 60d58c1a..5498b1b8 100644 --- a/docs/callers/cursor-review.md +++ b/docs/callers/cursor-review.md @@ -116,6 +116,7 @@ pull-requests: write # posting the consolidated review (and, at the round cap, | `openai_direct_model` | `''` | Opt-in. An OpenAI model id reviewed directly through the OpenAI API (pinned Codex CLI) as two cells — **adversarial** and **edge-case**, with the same prompts the Cursor cells use — always on GitHub-hosted `ubuntu-latest`. By default they **replace** the Cursor-hosted OpenAI (`gpt-*`) panel cells — see the next row, which also decides whether they gate. Requires `secrets.OPENAI_API_KEY` and a plain id (`[A-Za-z0-9._-]`); without either, preflight warns, the cells stay off and the panel is unchanged. | | `openai_direct_replaces_cursor_openai` | `true` | Only with `openai_direct_model` (and its key) in effect: drop the Cursor `gpt-*` cell(s) from the panel so the OpenAI lane is reviewed once. The direct cells then **gate** exactly like the Cursor cells they replace: they count toward `Panel integrity` and the auto-approve gate (`approve_max_failed_reviewers`), a failed one turns its leg red, and the review's "did not contribute" line names it, e.g. `gpt-5.6-sol:edge-case (direct, error)`. `false` runs both, for a side-by-side comparison of the two backends; the direct cells are then **advisory** — their findings reach the judge, but they are never counted and their legs never fail the run. Preflight fails rather than leave the Cursor panel empty. | | `skip_bot_branch_prefixes` | `ci/bump- chore/refresh- auto/refresh-` | Skip the panel for Bot-authored PRs on these branch prefixes (machine pin bumps / refreshes). `''` to review every bot PR. | +| `cell_timeout_minutes` | `15` | Wall-clock cap on ONE reviewer cell's agent step (`Run cursor review`, or `Run direct-API review` for a direct cell). A cell that runs out of clock exits 130 and uploads `status: error` with no findings, which fails its leg and so fails `Panel integrity` — so this is a guillotine, not a safety net. Raise it when cells are dying at the cap rather than hanging: compare how long the cells that DID submit took, and if a meaningful share finish within a few minutes of the cap you are clipping a tail, not catching hangs. It is also the budget a cell's retries share (`resource_exhausted` is retried in-step), so it bounds attempts plus backoff. Three further caps derive from this one and cannot be set separately — each cell's job cap is this + 15 (checkout, CLI install, artifacts), the judge's `Run judge` step cap is this × 2 (it reads every cell's findings at the same tier), and the judge's job cap is its step + 10. Bounded 5–45; preflight fails loud outside that, before any cell spends. | | `diff_size_cap` | `5000` | Skip review above this diff size. An over-cap PR is not silent — see the gotcha below. Under `blocking: true` it is also not green: an unreviewed PR cannot pass the gate. | | `ignore_comments` | `true` | Discount blank/comment-only lines from the size count (count-only — the panel still sees them). | | `review_label` | `cursor-review` | The label that triggers a run. | From c8474ac709752d617341a56cc729532a2d4c50a2 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 18:21:09 +0000 Subject: [PATCH 2/3] fix(cursor-review): compute derived timeout caps in preflight, not in expressions Actions expressions have no arithmetic operators, so `${{ inputs.cell_timeout_minutes + 15 }}` (and the `* 2` judge caps) failed the reusable at load for every caller. The "Validate the cell timeout" step now writes cell_job / judge_step / judge_job to $GITHUB_OUTPUT, preflight exposes them as job outputs, and the three coupled jobs read them via fromJSON(needs.preflight.outputs.*). Tests pin the new expressions, run the guard against a real $GITHUB_OUTPUT, and fail on any `+ - * /` inside a `${{ }}` in cursor-review.yml. Co-Authored-By: Claude Opus 5.5 --- .../tests/test_workflow_cell_timeout.py | 116 ++++++++++++++++-- .github/workflows/cursor-review.yml | 25 +++- 2 files changed, 123 insertions(+), 18 deletions(-) diff --git a/.github/cursor-review/tests/test_workflow_cell_timeout.py b/.github/cursor-review/tests/test_workflow_cell_timeout.py index dfe49597..97e83cda 100644 --- a/.github/cursor-review/tests/test_workflow_cell_timeout.py +++ b/.github/cursor-review/tests/test_workflow_cell_timeout.py @@ -18,17 +18,23 @@ can read the artifact back, so a dead cell reads as a cancelled run rather than a red leg. -So the arithmetic now lives in the workflow as expressions, and this file pins -it from three directions: +So the arithmetic now lives in the workflow — in `preflight`'s shell, exposed +as job outputs, because Actions expressions have NO arithmetic operators: +`${{ inputs.x + 15 }}` is a parse error that fails the reusable at load for +every caller, default included. This file pins it from four directions: * the exact expression at each of the six sites (a re-hardcoded literal fails, which is the regression), + * no `${{ }}` anywhere in the workflow uses `+`, `-`, `*` or `/` outside a + string literal (the load-time failure above, which a text-only pin of the + expression would happily keep green), * the ORDERING invariants, evaluated over the whole admitted range rather than at the default — a derivation that holds at 15 and inverts at 45 is the bug this is for, * the preflight bound check, by RUNNING its shell against real values, since `type: number` admits 0, -5 and 600 and the derived judge job cap doubles - whatever gets through. + whatever gets through — and reading back the caps it writes to + $GITHUB_OUTPUT, which the six sites consume. Deliberately parsed WITHOUT PyYAML, like its sibling suites: this repo is stdlib-only and CI installs no requirements for these tests. @@ -39,6 +45,7 @@ import os import re import subprocess +import tempfile import unittest WORKFLOW = os.path.normpath( @@ -50,9 +57,16 @@ # site -> the expression that must be there. `None` for the step sites means # "the bare input", spelled out per-site so a copy-paste between jobs fails. CELL = "${{ inputs.cell_timeout_minutes }}" -CELL_JOB = "${{ inputs.cell_timeout_minutes + 15 }}" -JUDGE_STEP = "${{ inputs.cell_timeout_minutes * 2 }}" -JUDGE_JOB = "${{ inputs.cell_timeout_minutes * 2 + 10 }}" +CELL_JOB = "${{ fromJSON(needs.preflight.outputs.cell_job_timeout) }}" +JUDGE_STEP = "${{ fromJSON(needs.preflight.outputs.judge_step_timeout) }}" +JUDGE_JOB = "${{ fromJSON(needs.preflight.outputs.judge_job_timeout) }}" + +# preflight job output -> the step output it must forward. +PREFLIGHT_OUTPUTS = { + "cell_job_timeout": "${{ steps.cell_timeout.outputs.cell_job }}", + "judge_step_timeout": "${{ steps.cell_timeout.outputs.judge_step }}", + "judge_job_timeout": "${{ steps.cell_timeout.outputs.judge_job }}", +} # The values today's workflow had before the input existed. The default must # reproduce them exactly, or this stopped being a no-op refactor. @@ -133,6 +147,21 @@ def test_judge_step_and_job(self): self.assertEqual(job_level_timeout(blk), JUDGE_JOB) self.assertEqual(step_level_timeout(step_block(blk, "Run judge")), JUDGE_STEP) + def test_consumers_need_preflight(self): + # A `needs.preflight.outputs.*` read from a job that does not list + # preflight in `needs:` evaluates to '' and fromJSON('') fails the job. + for name in ("review", "review-openai-direct", "consolidate"): + blk = job_block(self.src, name) + m = re.search(r"^ needs: \[([^\]]*)\]", blk, re.M) + self.assertIsNotNone(m, name) + self.assertIn("preflight", [n.strip() for n in m.group(1).split(",")], name) + + def test_preflight_exposes_each_derived_cap(self): + blk = job_block(self.src, "preflight") + for out, expr in PREFLIGHT_OUTPUTS.items(): + self.assertRegex(blk, r"\n %s: %s\n" % (out, re.escape(expr))) + self.assertIn(" id: cell_timeout\n", step_block(blk, "Validate the cell timeout")) + def test_no_literal_cap_survives_in_the_three_coupled_jobs(self): # The regression is one site re-hardcoded while the others stay # expressions — green CI, skewed panel. @@ -146,6 +175,44 @@ def test_no_literal_cap_survives_in_the_three_coupled_jobs(self): ) +ARITH = re.compile(r"[+*/]|(?> "$GITHUB_OUTPUT" - name: Define panel models id: models @@ -2383,10 +2396,10 @@ jobs: # cap must sit above its step cap so a hung judge trips the step (degraded # fallback still posts) rather than the job (all panel cells discarded). A # cell cap that outgrew them recreated exactly the silent skew that fix was - # for — so the arithmetic lives in the workflow now, where it cannot be - # half-applied. The +15 is this job's non-agent overhead: checkout, CLI + # for — so the arithmetic lives in `preflight` now (expressions cannot do + # it), where it cannot be half-applied. The +15 is this job's non-agent overhead: checkout, CLI # install, artifact download and upload, none of which the step cap covers. - timeout-minutes: ${{ inputs.cell_timeout_minutes + 15 }} + timeout-minutes: ${{ fromJSON(needs.preflight.outputs.cell_job_timeout) }} permissions: contents: read strategy: @@ -2897,7 +2910,7 @@ jobs: # `review`) holds for this cell too; the agent step caps lower, at 15. # DERIVED from `cell_timeout_minutes`, the same +15 as the Cursor cells # above and for the same overhead; see that job's note. - timeout-minutes: ${{ inputs.cell_timeout_minutes + 15 }} + timeout-minutes: ${{ fromJSON(needs.preflight.outputs.cell_job_timeout) }} permissions: contents: read strategy: @@ -3301,7 +3314,7 @@ jobs: # # DERIVED from `cell_timeout_minutes` (step x 2, + 10 slack), so raising a # cell's cap carries the judge with it instead of leaving it behind. - timeout-minutes: ${{ inputs.cell_timeout_minutes * 2 + 10 }} + timeout-minutes: ${{ fromJSON(needs.preflight.outputs.judge_job_timeout) }} # contents: read ONLY. This job checks out PR-authored code and runs the # judge agent over it, so it must carry nothing write-scoped — `post-review` # below holds pull-requests: write and the bot credentials instead. @@ -3575,7 +3588,7 @@ jobs: # DERIVED: twice `cell_timeout_minutes`. The judge adjudicates EVERY # cell's findings at the same reasoning tier, so a budget equal to one # cell's is the floor, not the target. - timeout-minutes: ${{ inputs.cell_timeout_minutes * 2 }} + timeout-minutes: ${{ fromJSON(needs.preflight.outputs.judge_step_timeout) }} env: CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }} run: | From 9288f873be7bb658c744d8b3c7e858c61115b161 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 18:27:09 +0000 Subject: [PATCH 3/3] fix(cursor-review): floor cell_timeout_minutes at 10, keep the judge step at 30+, reject overlong values - Floor 5 -> 10: the stop-on-submit wrapper may start its last resource_exhausted retry as late as 300s in, so at 5-7 minutes a retried cell was all but certain to hit the cap and redden Panel integrity. - Judge step = max(30, 2 x cell): its workload scales with the panel, not the cell cap, so tightening cells to fail fast must not starve the judge. The default still reproduces 15 / 30 / 30 / 40. - Reject values longer than two digits before the numeric test: an all-digit value past 64 bits makes `[ -lt ]` error, which inside `if` reads as false and let it through. - Refresh comments that still quoted the old fixed 15-minute caps. Co-Authored-By: Claude Opus 5.5 --- .github/cursor-review/README.md | 2 +- .../tests/test_workflow_cell_timeout.py | 37 ++++++++-- .github/workflows/cursor-review.yml | 73 ++++++++++++------- docs/callers/cursor-review.md | 2 +- 4 files changed, 80 insertions(+), 34 deletions(-) diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 07166f68..b73c2b00 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -376,7 +376,7 @@ All optional except `workflows_ref` (required, no default) — pass them under | `panel_models` | `''` (built-in list) | JSON array of Cursor model ids that **replaces** the panel list; each still runs both review types and is validated against the live catalog by preflight. For per-repo experiments (e.g. `-xhigh` vs `-max` tiers). | | `openai_direct_model` | `''` (off) | OPT-IN. An OpenAI model id reviewed through the OpenAI API directly (pinned Codex CLI, `package.json` here) as two cells (**adversarial** + **edge-case**, the Cursor cells' prompts) on GitHub-hosted runners, by default **replacing** the Cursor `gpt-*` cells (next row). Needs the `OPENAI_API_KEY` secret and a plain id (`[A-Za-z0-9._-]`); without either, preflight warns and leaves the cells off and the panel unchanged. Not part of `panel_models` (not in the Cursor catalog). Whether the cells gate depends on the next row. | | `openai_direct_replaces_cursor_openai` | `true` | When the direct cells are in effect, drop the Cursor `gpt-*` cell(s) so the OpenAI lane is reviewed once — the direct cells then **gate** like the Cursor cells they replace: counted in `Panel integrity` and the auto-approve gate, a failed one red in the rollup and named `(direct, )` in the review's "did not contribute" line. `false` runs both for a side-by-side backend comparison, with the direct cells **advisory** (findings reach the judge; never counted, never red). Preflight fails rather than leave the Cursor panel empty. | -| `cell_timeout_minutes` | `15` | Wall-clock cap on one reviewer cell's agent step (Cursor or direct). Over it the cell exits 130, uploads `status: error` with no findings, reddens its leg and fails `Panel integrity`. Shared by the cell's in-step retries, so it bounds attempts + backoff. Derives three more caps that cannot be set alone: cell job = this + 15, `Run judge` step = this × 2, judge job = that + 10. At the default: 15 / 30 / 30 / 40, today's values exactly. Preflight bounds it 5–45. | +| `cell_timeout_minutes` | `15` | Wall-clock cap on one reviewer cell's agent step (Cursor or direct). Over it the cell exits 130, uploads `status: error` with no findings, reddens its leg and fails `Panel integrity`. Shared by the cell's in-step retries, so it bounds attempts + backoff. Derives three more caps that cannot be set alone: cell job = this + 15, `Run judge` step = max(30, this × 2), judge job = that + 10. At the default: 15 / 30 / 30 / 40, today's values exactly. Preflight bounds it 10–45. | | `skip_bot_branch_prefixes` | `ci/bump- chore/refresh- auto/refresh-` | Skip the panel when the PR author is a Bot **and** the head branch starts with one of these prefixes (machine pin bumps / catalog refreshes). `''` reviews every bot PR. | | `diff_size_cap` | `5000` | Max counted changed lines (after generated-file exclusion and comment discounting); larger PRs are skipped. | | `ignore_comments` | `true` | Discount blank/comment-only lines from the size count (count-only; the panel still sees them). | diff --git a/.github/cursor-review/tests/test_workflow_cell_timeout.py b/.github/cursor-review/tests/test_workflow_cell_timeout.py index 97e83cda..3ad8b71f 100644 --- a/.github/cursor-review/tests/test_workflow_cell_timeout.py +++ b/.github/cursor-review/tests/test_workflow_cell_timeout.py @@ -52,7 +52,12 @@ os.path.join(os.path.dirname(__file__), "..", "..", "workflows", "cursor-review.yml") ) -FLOOR, CEILING, DEFAULT = 5, 45, 15 +# The floor is 10, not lower: `Run cursor review` may start its last +# `resource_exhausted` retry as late as `--retry-window 300` (5 min) in, and +# that attempt still needs room for a healthy review. +FLOOR, CEILING, DEFAULT = 10, 45, 15 +RETRY_WINDOW_S = 300 +JUDGE_STEP_FLOOR = 30 # site -> the expression that must be there. `None` for the step sites means # "the bare input", spelled out per-site so a copy-paste between jobs fails. @@ -104,8 +109,8 @@ def derive(cell): return { "cell_step": cell, "cell_job": cell + 15, - "judge_step": cell * 2, - "judge_job": cell * 2 + 10, + "judge_step": max(JUDGE_STEP_FLOOR, cell * 2), + "judge_job": max(JUDGE_STEP_FLOOR, cell * 2) + 10, } @@ -229,6 +234,18 @@ def test_every_admitted_value_keeps_the_ordering(self): self.assertGreater(d["judge_job"], d["judge_step"]) # The judge must never have less model budget than one cell. self.assertGreaterEqual(d["judge_step"], d["cell_step"]) + # ...nor less than today's, whatever the cell cap: its + # workload scales with the panel, not with the cell cap. + self.assertGreaterEqual(d["judge_step"], HISTORICAL["judge_step"]) + + def test_floor_leaves_a_late_retry_room_for_a_full_review(self): + # A retry can start as late as the retry window; the attempt it starts + # must still get at least as long as the window itself. + self.assertGreaterEqual(FLOOR * 60 - RETRY_WINDOW_S, RETRY_WINDOW_S) + + def test_floor_constant_tracks_the_wrappers_retry_window(self): + step = step_block(job_block(text(), "review"), "Run cursor review") + self.assertIn("--retry-window %d " % RETRY_WINDOW_S, step) class PreflightBoundCheck(unittest.TestCase): @@ -280,7 +297,7 @@ def test_reports_every_derived_cap_it_accepted(self): def test_writes_the_derived_caps_the_jobs_consume(self): # These ARE the timeouts now; the notice above is only for humans. - for cell in (FLOOR, DEFAULT, CEILING): + for cell in (FLOOR, 14, DEFAULT, 16, CEILING): d = derive(cell) with self.subTest(cell=cell): r = self.run_guard(cell) @@ -299,7 +316,7 @@ def test_rejected_values_write_no_caps(self): self.assertEqual(self.run_guard(v).outputs, {}) def test_rejects_out_of_range(self): - for v in (0, FLOOR - 1, CEILING + 1, 600): + for v in (0, 5, FLOOR - 1, CEILING + 1, 600, "015", "08"): with self.subTest(v=v): r = self.run_guard(v) self.assertEqual(r.returncode, 1, r.stdout) @@ -313,6 +330,16 @@ def test_rejects_what_type_number_still_admits(self): self.assertEqual(r.returncode, 1, (v, r.stdout, r.stderr)) self.assertIn("::error::", r.stdout) + def test_rejects_digits_too_long_for_a_64_bit_integer(self): + # `[ -lt ]` errors on these, and an erroring test inside `if` reads as + # false, so without a length check both bounds would pass it. + for v in ("99999999999999999999", "0" * 30 + "15"): + with self.subTest(v=v): + r = self.run_guard(v) + self.assertEqual(r.returncode, 1, (v, r.stdout, r.stderr)) + self.assertIn("::error::", r.stdout) + self.assertEqual(r.outputs, {}) + def test_the_guard_names_the_value_it_rejected(self): r = self.run_guard(600) self.assertIn("600", r.stdout) diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index ac22bd57..320958aa 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -470,10 +470,12 @@ on: cannot be set separately, which is what keeps the panel and its judge in step: each cell's JOB cap is this + 15 (checkout, CLI install, artifact up/download — overhead outside the agent step), the judge's - `Run judge` step cap is this x 2 (it reads EVERY cell's findings at the - same reasoning tier, so it needs more than any one of them), and the - judge's job cap is its step + 10. At the default that reproduces - today's 30 / 15 / 30 / 40 exactly. Bounded 5-45 by preflight, which + `Run judge` step cap is this x 2 but never below 30 (it reads EVERY + cell's findings at the same reasoning tier, so it needs more than any + one of them, and its workload scales with the panel, not with this + cap), and the judge's job cap is its step + 10. At the default that + reproduces today's 30 / 15 / 30 / 40 exactly. Bounded 10-45 by + preflight, which fails before any cell spends rather than letting a typo mint an hours-long job. type: number @@ -2048,29 +2050,41 @@ jobs: # # Bounded rather than clamped, deliberately: silently correcting a # caller's 600 to 45 would review a long diff on a budget nobody chose - # and report success. The floor is 5 because a cell has never submitted - # faster than ~1 minute and the retry backoff alone spends 2; the - # ceiling is 45 because that is already a 100-minute judge job, and - # anything needing more is a diff-size problem, not a timeout problem. + # and report success. The floor is 10 because `Run cursor review`'s + # wrapper may start its last `resource_exhausted` retry as late as 300s + # in (`--retry-window 300`), and that attempt still needs room for a + # healthy 3-7 minute review; at 5 a retried cell was all but certain to + # hit the cap. The ceiling is 45 because that is already a 100-minute + # judge job, and anything needing more is a diff-size problem, not a + # timeout problem. The length check runs first because an all-digit + # value too long for a 64-bit integer makes `[ -lt ]` error, and an + # erroring test inside `if` reads as false, so it would pass. env: CELL_TIMEOUT: ${{ inputs.cell_timeout_minutes }} run: | set -euo pipefail case "$CELL_TIMEOUT" in ''|*[!0-9]*) - echo "::error::cell_timeout_minutes must be a whole number of minutes between 5 and 45; got '${CELL_TIMEOUT}'." + echo "::error::cell_timeout_minutes must be a whole number of minutes between 10 and 45; got '${CELL_TIMEOUT}'." exit 1 ;; esac - if [ "$CELL_TIMEOUT" -lt 5 ] || [ "$CELL_TIMEOUT" -gt 45 ]; then - echo "::error::cell_timeout_minutes must be between 5 and 45; got ${CELL_TIMEOUT}. The judge's job cap is twice this plus 10, so the ceiling is what keeps a typo from minting an hours-long job." + if [ "${#CELL_TIMEOUT}" -gt 2 ] || [ "$CELL_TIMEOUT" -lt 10 ] || [ "$CELL_TIMEOUT" -gt 45 ]; then + echo "::error::cell_timeout_minutes must be between 10 and 45; got ${CELL_TIMEOUT}. The judge's job cap is twice this plus 10, so the ceiling is what keeps a typo from minting an hours-long job." exit 1 fi - echo "::notice::Cell agent cap ${CELL_TIMEOUT}m; cell job $((CELL_TIMEOUT + 15))m; judge step $((CELL_TIMEOUT * 2))m; judge job $((CELL_TIMEOUT * 2 + 10))m." + # 10# forces base 10; a two-digit value with a leading zero is < 10 + # and already rejected above, but this keeps "08" from reading as octal. + cell=$((10#$CELL_TIMEOUT)) + judge_step=$((cell * 2)) + # Never below 30: the judge's workload grows with the panel, not with + # the cell cap, so tightening cells to fail fast must not starve it. + if [ "$judge_step" -lt 30 ]; then judge_step=30; fi + echo "::notice::Cell agent cap ${cell}m; cell job $((cell + 15))m; judge step ${judge_step}m; judge job $((judge_step + 10))m." { - echo "cell_job=$((CELL_TIMEOUT + 15))" - echo "judge_step=$((CELL_TIMEOUT * 2))" - echo "judge_job=$((CELL_TIMEOUT * 2 + 10))" + echo "cell_job=$((cell + 15))" + echo "judge_step=${judge_step}" + echo "judge_job=$((judge_step + 10))" } >> "$GITHUB_OUTPUT" - name: Define panel models @@ -2690,7 +2704,8 @@ jobs: # (~25s in, before recording anything) is re-run after 30s, then 90s. # Only a cell with no recorded findings is re-run, and no retry starts # later than 300s in, so the last attempt still has all but ~5 min of - # this cap for a full review — which is why the cap's floor is 5. Each retry is a `::warning::` naming the cell; a + # this cap for a full review — which is why the cap's floor is 10, not + # lower. Each retry is a `::warning::` naming the cell; a # cell that exhausts them fails its leg exactly as before. A retry # re-reads the whole prompt because the wrapper rewinds its stdin, so # keep feeding it as `< /tmp/prompt.txt`: a pipe cannot be rewound, @@ -2907,9 +2922,9 @@ jobs: # failed Cursor cell. continue-on-error: ${{ needs.preflight.outputs.openai_direct_counts != 'true' }} # Matches the `review` job's cap so the judge's budget reasoning (see - # `review`) holds for this cell too; the agent step caps lower, at 15. - # DERIVED from `cell_timeout_minutes`, the same +15 as the Cursor cells - # above and for the same overhead; see that job's note. + # `review`) holds for this cell too; the agent step caps lower, at + # `cell_timeout_minutes`. DERIVED from that input, the same +15 as the + # Cursor cells above and for the same overhead; see that job's note. timeout-minutes: ${{ fromJSON(needs.preflight.outputs.cell_job_timeout) }} permissions: contents: read @@ -3126,7 +3141,7 @@ jobs: continue-on-error: true # Capped well under the job so a hung agent trips the STEP and the # seeded artifact still uploads below, rather than tripping the job and - # discarding the cell. Same 15 minutes the Cursor cells get. + # discarding the cell. Same `cell_timeout_minutes` the Cursor cells get. timeout-minutes: ${{ inputs.cell_timeout_minutes }} env: MODEL: ${{ inputs.openai_direct_model }} @@ -3312,7 +3327,8 @@ jobs: # which a panel cell does not all pay, so the judge's effective MODEL # budget is the step cap, not this one. # - # DERIVED from `cell_timeout_minutes` (step x 2, + 10 slack), so raising a + # DERIVED from `cell_timeout_minutes` (step = max(30, x 2), job = step + # + 10 slack), so raising a # cell's cap carries the judge with it instead of leaving it behind. timeout-minutes: ${{ fromJSON(needs.preflight.outputs.judge_job_timeout) }} # contents: read ONLY. This job checks out PR-authored code and runs the @@ -3576,18 +3592,21 @@ jobs: # Time-boxed BELOW the job cap and absorbed, so a hung judge loses only # the adjudication: the steps below then read the `--init` seed left by # `Configure structured final review` (status=error), take the degraded - # panel-union path and still post what the cells found. Equal to the - # `review` panel cap so the judge is never given less model time than - # one of the cells it reads. `continue-on-error` is what makes the + # panel-union path and still post what the cells found. At least the + # `review` job cap at the default (30) and always at least twice the + # cell step cap, so the judge is never given less model time than one of + # the cells it reads. `continue-on-error` is what makes the # timeout absorbable — without it the step failure fails the job and # lands back on the all-cells-discarded outcome this is fixing. It # hides no real signal: the cursor-agent exit code is already captured # in-script, and a judge that did not submit is reported to the PR as # `judge_status` by `Post review`. continue-on-error: true - # DERIVED: twice `cell_timeout_minutes`. The judge adjudicates EVERY - # cell's findings at the same reasoning tier, so a budget equal to one - # cell's is the floor, not the target. + # DERIVED: twice `cell_timeout_minutes`, never below 30. The judge + # adjudicates EVERY cell's findings at the same reasoning tier, so a + # budget equal to one cell's is the floor, not the target — and its + # workload scales with the panel, so a tightened cell cap must not + # shrink it below today's 30. timeout-minutes: ${{ fromJSON(needs.preflight.outputs.judge_step_timeout) }} env: CURSOR_API_KEY: ${{ secrets.CURSOR_API_KEY }} diff --git a/docs/callers/cursor-review.md b/docs/callers/cursor-review.md index 5498b1b8..60ba53d9 100644 --- a/docs/callers/cursor-review.md +++ b/docs/callers/cursor-review.md @@ -116,7 +116,7 @@ pull-requests: write # posting the consolidated review (and, at the round cap, | `openai_direct_model` | `''` | Opt-in. An OpenAI model id reviewed directly through the OpenAI API (pinned Codex CLI) as two cells — **adversarial** and **edge-case**, with the same prompts the Cursor cells use — always on GitHub-hosted `ubuntu-latest`. By default they **replace** the Cursor-hosted OpenAI (`gpt-*`) panel cells — see the next row, which also decides whether they gate. Requires `secrets.OPENAI_API_KEY` and a plain id (`[A-Za-z0-9._-]`); without either, preflight warns, the cells stay off and the panel is unchanged. | | `openai_direct_replaces_cursor_openai` | `true` | Only with `openai_direct_model` (and its key) in effect: drop the Cursor `gpt-*` cell(s) from the panel so the OpenAI lane is reviewed once. The direct cells then **gate** exactly like the Cursor cells they replace: they count toward `Panel integrity` and the auto-approve gate (`approve_max_failed_reviewers`), a failed one turns its leg red, and the review's "did not contribute" line names it, e.g. `gpt-5.6-sol:edge-case (direct, error)`. `false` runs both, for a side-by-side comparison of the two backends; the direct cells are then **advisory** — their findings reach the judge, but they are never counted and their legs never fail the run. Preflight fails rather than leave the Cursor panel empty. | | `skip_bot_branch_prefixes` | `ci/bump- chore/refresh- auto/refresh-` | Skip the panel for Bot-authored PRs on these branch prefixes (machine pin bumps / refreshes). `''` to review every bot PR. | -| `cell_timeout_minutes` | `15` | Wall-clock cap on ONE reviewer cell's agent step (`Run cursor review`, or `Run direct-API review` for a direct cell). A cell that runs out of clock exits 130 and uploads `status: error` with no findings, which fails its leg and so fails `Panel integrity` — so this is a guillotine, not a safety net. Raise it when cells are dying at the cap rather than hanging: compare how long the cells that DID submit took, and if a meaningful share finish within a few minutes of the cap you are clipping a tail, not catching hangs. It is also the budget a cell's retries share (`resource_exhausted` is retried in-step), so it bounds attempts plus backoff. Three further caps derive from this one and cannot be set separately — each cell's job cap is this + 15 (checkout, CLI install, artifacts), the judge's `Run judge` step cap is this × 2 (it reads every cell's findings at the same tier), and the judge's job cap is its step + 10. Bounded 5–45; preflight fails loud outside that, before any cell spends. | +| `cell_timeout_minutes` | `15` | Wall-clock cap on ONE reviewer cell's agent step (`Run cursor review`, or `Run direct-API review` for a direct cell). A cell that runs out of clock exits 130 and uploads `status: error` with no findings, which fails its leg and so fails `Panel integrity` — so this is a guillotine, not a safety net. Raise it when cells are dying at the cap rather than hanging: compare how long the cells that DID submit took, and if a meaningful share finish within a few minutes of the cap you are clipping a tail, not catching hangs. It is also the budget a cell's retries share (`resource_exhausted` is retried in-step), so it bounds attempts plus backoff. Three further caps derive from this one and cannot be set separately — each cell's job cap is this + 15 (checkout, CLI install, artifacts), the judge's `Run judge` step cap is this × 2 but never below 30 (it reads every cell's findings at the same tier, and its workload scales with the panel, not this cap), and the judge's job cap is its step + 10. Bounded 10–45 (the floor leaves a late in-step retry room for a full review); preflight fails loud outside that, before any cell spends. | | `diff_size_cap` | `5000` | Skip review above this diff size. An over-cap PR is not silent — see the gotcha below. Under `blocking: true` it is also not green: an unreviewed PR cannot pass the gate. | | `ignore_comments` | `true` | Discount blank/comment-only lines from the size count (count-only — the panel still sees them). | | `review_label` | `cursor-review` | The label that triggers a run. |