From 1175e244817267da89d4839e4777cb17a7b30f7d Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 11:21:06 -0700 Subject: [PATCH 1/3] fix(cursor-review): move the default Moonshot cell back to kimi-k3-max MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `kimi-k3-high` is capacity-bound, so the built-in panel is currently one lane short on every consumer that has not overridden it — which is most of them. The cells do not run slow, they die: ~25 seconds in, `RetriableError: [resource_exhausted]` after three internal reconnects, zero findings, leg red, `Panel integrity` red with it. Measured on one consumer over 56 panels, 24 of 76 `kimi-k3-high` cells red (32%), 19 of those this error, and 12 of 22 on the last day. Not a property of one repo: the same model threw the same error the same day on two unrelated consumers. And #388's in-step retry does not rescue it, because the condition is sustained rather than transient — both cells on this repo's own panel for #392 exhausted all three attempts. `-max` rather than dropping the lane, because the lane earns its place. The 2026-08-27 attribution that cut Gemini was re-run over 1,158 judge-kept findings (82 PRs), normalised per finding raised so panel size flatters nobody: model raised sole distinct sole/PR c/h sole kimi-k3-max 229 39 17% 1.05 3/58 kimi-k3-high 81 22 27% 1.05 0/11 gemini-3.1-pro 99 12 12% 0.33 0/58 Kimi contributes 1.05 unique findings per PR at BOTH tiers and uniquely caught 3 of 58 critical/high at `-max`. Gemini — cut for exactly this reason — managed 0.33 and 0 of 58. Dropping Kimi would cost about three times what dropping Gemini did, so the fix is the tier, not removal. This deliberately gives back the spend the `-high` notch in #243 was for; it is a partial reversal of that change, not an oversight. And the reliability it buys is NOT established at this tier under present conditions: `-max` was 0 of 36 cells red, but that was 2026-10-07, before the capacity trouble, and the two tiers may share a pool. Recorded in the workflow so the next person does not have to rediscover it, along with what the evidence says to do if `-max` goes the same way: drop to a two-lab Cursor panel rather than substitute Gemini or Grok. Grok `4.7-high` was tried on 2026-10-09 and raised 0 unique findings in 17 while timing out on 11 of 20 cells. Default change, so it reaches every consumer on the next SHA bump; a repo that wants the old tier back sets CURSOR_PANEL_MODELS. README panel table and history updated alongside. Tests: cursor-review 1257, workflow-pins 551 + pin lint, agents-md — all green. The historical-era fixtures and `test_catalog_drift.py`'s REAL_PANEL pin their own eras by design and are deliberately untouched. Co-Authored-By: Claude Opus 5 --- .github/cursor-review/README.md | 17 +++++++++++- .github/workflows/cursor-review.yml | 40 ++++++++++++++++++++++++++--- 2 files changed, 52 insertions(+), 5 deletions(-) diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index 6350cd32..a2164fa0 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -183,13 +183,28 @@ The build is then **verified, not trusted**, and against the reviewed diff's own |---|---| | OpenAI | `gpt-5.6-sol-xhigh` | | Anthropic | `claude-opus-5-5-xhigh` | -| Moonshot | `kimi-k3-high` | +| Moonshot | `kimi-k3-max` | Gemini 3.1 Pro was dropped on 2026-08-27 after a spend review: across 1,450 judge-kept findings its two cells were the sole raiser of 2.5% of findings and 3 of 213 critical/high ones, at ~9% of every run's cost. Kimi moved from `-max` to `-high` and the judge from `-max` to `-xhigh` in the same review. +**Kimi moved back to `-max` on 2026-10-09.** The `-high` tier turned +capacity-bound — cells dying in ~25s with `RetriableError: +[resource_exhausted]`, 32% of them on one consumer, and the in-step retry does +not rescue a sustained condition. Re-running the attribution above over 1,158 +judge-kept findings (82 PRs), normalised per finding raised: Kimi is the sole +raiser of 1.05 findings per PR at **both** tiers and uniquely caught 3 of 58 +critical/high at `-max`, against Gemini's 0.33 per PR and 0 of 58 — so the lane +is worth about three times what the dropped Gemini lane was, and the fix is the +tier, not removal. This gives back the spend the `-high` notch was for. +Reliability at `-max` is not re-established under current conditions (0 of 36 +red, but measured 2026-10-07, and the tiers may share a pool); if it goes the +same way, the attribution says drop to a two-lab Cursor panel rather than +substitute Gemini or Grok — Grok `4.7-high`, tried 2026-10-09, raised 0 unique +findings in 17 while timing out on 11 of 20 cells. + The Anthropic cell moved from Opus 5 `-thinking-max` to **Opus 5.5 `-xhigh`** on 2026-10-07 — a newer model one reasoning notch down, on the same spend reasoning. Note that Opus 5.5 ships no `-thinking-` variants in the Cursor diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index f6467082..1a1209cf 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -2029,9 +2029,9 @@ jobs: # nearest is a plain tier id. That plain line IS the thinking model: # Cursor labels non-thinking explicitly where it applies (every plain # Haiku 5.5 entry reads "No Thinking") and no Opus 5.5 or Sonnet 5.5 - # entry carries that marker. Moonshot: Kimi K3 at the `high` tier — - # same model as the former `kimi-k3-max` pin, one reasoning notch - # lower; no NO-ZDR marker. + # entry carries that marker. Moonshot: Kimi K3 at the `max` tier — + # RESTORED 2026-10-09 after the `-high` tier proved capacity-bound; see + # the note below. No NO-ZDR marker. # # Gemini 3.1 Pro was removed 2026-08-27. Over 1,450 judge-kept # findings (172 PRs), its two cells were the sole raiser of 2.5% of @@ -2043,6 +2043,38 @@ jobs: # 2026-10-07 for the same reason: a newer model one reasoning notch # down. That is deliberately TWO axes at once (version + tier) — it # is a default change, not an attribution experiment. The + # Kimi went BACK to `-max` on 2026-10-09. The `-high` tier turned + # capacity-bound, not slow: cells died in ~25 seconds with + # `RetriableError: [resource_exhausted]` after three internal + # reconnects, zero findings, leg red, `Panel integrity` red with it. + # Measured on one consumer over 56 panels: 24 of 76 `kimi-k3-high` + # cells red (32%), 19 of them this error, and 12 of 22 on the last day. + # It is not a property of one repo — the same model threw the same error + # the same day on two unrelated consumers — and #388's in-step retry + # does not rescue it, because the condition is sustained rather than + # transient: both cells on github-workflows#392's own panel exhausted + # all three attempts. + # + # `-max` rather than dropping the lane, because the lane EARNS its + # place. Re-running the 2026-08-27 attribution over 1,158 judge-kept + # findings (82 PRs), normalised per finding raised so panel size does + # not flatter anyone: Kimi was the sole raiser of 17% of what it raised + # at `-max` and 27% at `-high` — 1.05 unique findings per PR at BOTH + # tiers — and uniquely caught 3 of 58 critical/high at `-max`. Compare + # Gemini, cut for exactly this reason: 12% and 0.33 per PR, 0 of 58 + # critical/high. Dropping Kimi would cost roughly three times what + # dropping Gemini did. + # + # What this trades away is the spend the tier drop was for, so it is a + # deliberate reversal of part of #243, not an oversight. The reliability + # it buys is also NOT established at this tier under present conditions: + # `-max` was 0 of 36 cells red, but that was measured on 2026-10-07, + # before the capacity trouble, and the two tiers may well share a pool. + # If `-max` goes the same way, the evidence above says drop to a + # two-lab Cursor panel rather than substitute Gemini or Grok — Grok + # `4.7-high` was tried on 2026-10-09 and raised 0 unique findings in 17 + # while timing out on 11 of 20 cells. + # # `panel_models` input — or the CURSOR_PANEL_MODELS variable, see the # env block below — overrides this list for A/B experiments. # @@ -2087,7 +2119,7 @@ jobs: [ "gpt-5.6-sol-xhigh", "claude-opus-5-5-xhigh", - "kimi-k3-high" + "kimi-k3-max" ] JSON if [ -n "$(printf '%s' "$PANEL_MODELS_OVERRIDE" | tr -d '[:space:]')" ]; then From 90e4e388e5ebdda637491af4235f527cf3871bd4 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 18:25:04 +0000 Subject: [PATCH 2/3] docs(cursor-review): rejoin the panel_models sentence the Kimi note split Co-Authored-By: Claude Opus 5.5 --- .github/workflows/cursor-review.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index 1a1209cf..bcad968f 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -2042,7 +2042,8 @@ jobs: # moved from Opus 5 `-thinking-max` to Opus 5.5 `-xhigh` on # 2026-10-07 for the same reason: a newer model one reasoning notch # down. That is deliberately TWO axes at once (version + tier) — it - # is a default change, not an attribution experiment. The + # is a default change, not an attribution experiment. + # # Kimi went BACK to `-max` on 2026-10-09. The `-high` tier turned # capacity-bound, not slow: cells died in ~25 seconds with # `RetriableError: [resource_exhausted]` after three internal @@ -2075,7 +2076,7 @@ jobs: # `4.7-high` was tried on 2026-10-09 and raised 0 unique findings in 17 # while timing out on 11 of 20 cells. # - # `panel_models` input — or the CURSOR_PANEL_MODELS variable, see the + # The `panel_models` input — or the CURSOR_PANEL_MODELS variable, see the # env block below — overrides this list for A/B experiments. # # The OpenAI cell moved from `gpt-5.6-sol-max` to `gpt-5.6-sol-xhigh`: From 2dadf0eb448ec82f69b06b3c2625b397e5877c53 Mon Sep 17 00:00:00 2001 From: Matt Miller Date: Fri, 9 Oct 2026 18:33:44 +0000 Subject: [PATCH 3/3] docs(cursor-review): move the remaining kimi-k3-high examples to the -max pin The CURSOR_PANEL_MODELS copy-paste examples would otherwise change two axes and silently put a fleet back on the capacity-bound -high tier. Also reword the README's 32% figure to match the source data. Co-Authored-By: Claude Opus 5.5 --- .github/cursor-review/README.md | 2 +- .github/cursor-review/prompt-judge.md | 2 +- .github/workflows/cursor-review.yml | 2 +- docs/callers/cursor-review.md | 6 +++--- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/cursor-review/README.md b/.github/cursor-review/README.md index a2164fa0..a77a3b27 100644 --- a/.github/cursor-review/README.md +++ b/.github/cursor-review/README.md @@ -192,7 +192,7 @@ to `-high` and the judge from `-max` to `-xhigh` in the same review. **Kimi moved back to `-max` on 2026-10-09.** The `-high` tier turned capacity-bound — cells dying in ~25s with `RetriableError: -[resource_exhausted]`, 32% of them on one consumer, and the in-step retry does +[resource_exhausted]`, 32% of cells red on one consumer, and the in-step retry does not rescue a sustained condition. Re-running the attribution above over 1,158 judge-kept findings (82 PRs), normalised per finding raised: Kimi is the sole raiser of 1.05 findings per PR at **both** tiers and uniquely caught 3 of 58 diff --git a/.github/cursor-review/prompt-judge.md b/.github/cursor-review/prompt-judge.md index 8991162f..27185ea7 100644 --- a/.github/cursor-review/prompt-judge.md +++ b/.github/cursor-review/prompt-judge.md @@ -75,7 +75,7 @@ two fields on no other finding. - `body`: concise (1-3 sentences). Do NOT prefix the body with a severity word or emoji; the severity field drives the rendered badge. END with attribution like - `_Raised by 3 of N reviewers (gpt-5.6-sol-xhigh adversarial, claude-opus-5-5-xhigh edge-case, kimi-k3-high adversarial)._` + `_Raised by 3 of N reviewers (gpt-5.6-sol-xhigh adversarial, claude-opus-5-5-xhigh edge-case, kimi-k3-max adversarial)._` where N is the number of panel cells listed in the PANEL FINDINGS block. Order findings most-severe first. If no findings rise to the bar, submit an diff --git a/.github/workflows/cursor-review.yml b/.github/workflows/cursor-review.yml index bcad968f..fe78b301 100644 --- a/.github/workflows/cursor-review.yml +++ b/.github/workflows/cursor-review.yml @@ -4491,7 +4491,7 @@ jobs: # not submit` in the `review` job is the per-cell half. This job is what a # gate should read, because a matrix leg's context name carries the model id # and therefore CHANGES whenever the panel list does — a required check on - # `edge-case (kimi-k3-high)` bricks every PR the day that pin moves. + # `edge-case (kimi-k3-max)` bricks every PR the day that pin moves. # # ADVISORY by default, exactly like `Blocking gate`: red here fails no other # job IN THIS WORKFLOW, gates no merge, and does not gate `post-review` — a diff --git a/docs/callers/cursor-review.md b/docs/callers/cursor-review.md index 60d58c1a..29092c7b 100644 --- a/docs/callers/cursor-review.md +++ b/docs/callers/cursor-review.md @@ -143,11 +143,11 @@ pull-requests: write # posting the consolidated review (and, at the round cap, ```bash # one repo gh variable set CURSOR_PANEL_MODELS --repo / \ - --body '["gpt-5.6-sol-xhigh","claude-opus-5-5-max","kimi-k3-high"]' + --body '["gpt-5.6-sol-xhigh","claude-opus-5-5-max","kimi-k3-max"]' # or the whole fleet at once (visibility must include private repos) gh variable set CURSOR_PANEL_MODELS --org --visibility all \ - --body '["gpt-5.6-sol-xhigh","claude-opus-5-5-max","kimi-k3-high"]' + --body '["gpt-5.6-sol-xhigh","claude-opus-5-5-max","kimi-k3-max"]' ``` These examples deliberately differ from the built-in list by exactly one axis: @@ -382,7 +382,7 @@ The remaining shapes to expect before you require it: `cancel-in-progress` caller above that red lands on the head SHA that was superseded, not on the new one. * **Do not require a leg check instead.** A panel cell's context name carries - the model id (`edge-case (kimi-k3-high)`), so it changes whenever the panel + the model id (`edge-case (kimi-k3-max)`), so it changes whenever the panel list does — and a required check whose name no longer exists blocks every PR in the repo. `Panel integrity` is stable by design. * **It is red, not skipped, when the decision itself failed.** Panel integrity