From de587dc9d7e0e0921044bbd6dc54b8b017e442ec Mon Sep 17 00:00:00 2001 From: Artem Frolov Date: Tue, 13 Oct 2026 13:57:34 +0300 Subject: [PATCH] feat: add a sonnet task-reviewer for plan-loop reviews and drop "when in doubt, opus" Superpowers subagent-driven-development scales the model of its per-task reviews and scoped re-reviews to the diff, with a mid-tier floor, while the anchor sent every review to the opus-pinned reviewer. Sessions resolved the conflict by dispatching reviewer with model=sonnet, which the stats report flags as below pin: 69 of about 230 reviewer dispatches in 11 days on one machine, 6 of 38 on another. task-reviewer (sonnet/high) takes those seats and flags diffs that need reviewer. reviewer stays opus for final, whole-branch, high-risk and pre-PR review, so a bare reviewer dispatch still fails toward the stronger model. The implementer line loses "when in doubt, opus". In 199 implementer dispatches over the same window, 104 ran on opus and about 98 of those named no escalation reason; sonnet tasks showed no sign of extra fix rounds (0.16 fix dispatches per task against 0.21 on opus, confounded by task size). The explicit escalation categories stay. --- .claude-plugin/plugin.json | 2 +- README.md | 22 ++++++++++++++-------- agents/task-reviewer.md | 32 ++++++++++++++++++++++++++++++++ hooks/dispatch-counter.mjs | 1 + hooks/routing-anchor.md | 4 ++-- skills/model-routing/SKILL.md | 16 +++++++++++----- 6 files changed, 61 insertions(+), 16 deletions(-) create mode 100644 agents/task-reviewer.md diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index e207c6a..3259e2f 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "model-routing", - "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", + "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer, task-reviewer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", "version": "0.16.6", "author": { "name": "Artem Frolov" diff --git a/README.md b/README.md index e2f4ca4..6a8b072 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,8 @@ reason written down in `evals/forced-dispatch-tier/NOTES.md`.) | `agents/test-runner.md` | haiku | low | Run tests/builds/linters, report failures compactly. Never fixes anything. | | `agents/e2e-runner.md` | sonnet | medium | Drive Playwright/E2E scenarios, interpret failures (product bug vs test bug vs flake). | | `agents/implementer.md` | sonnet | medium | Implement one well-defined task from an approved plan. Verifies its own work. Dispatch with `model=opus` for multi-file/architectural/subtle work. | -| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. | +| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. Final/whole-branch, high-risk and pre-PR review. | +| `agents/task-reviewer.md` | sonnet | high | Per-task reviews and scoped re-reviews inside a plan loop (Superpowers SDD), following the caller's review template. Flags diffs that need `reviewer` instead. | | `agents/verifier.md` | haiku | medium | Cheap gate on a subagent's diff: does it match the task (scope, completeness, obvious breakage)? Not a code review. | | `skills/model-routing/` | - | - | The routing table and delegation rules Claude follows when deciding where work goes. | | `hooks/routing-anchor.md` | - | - | Short routing anchor auto-injected at session start - zero config. | @@ -199,6 +200,7 @@ flowchart LR C["'Implement tasks 1-3 from the plan'"] --> IM["implementer
sonnet / medium"] D["'Refactor the payment pipeline'"] --> IMO["implementer
model=opus / medium"] E["'Review the diff'"] --> RV["reviewer
opus / high"] + G["'Review Task 2' (plan loop)"] --> TRV["task-reviewer
sonnet / high"] F["'Design the architecture'"] --> MS["main session
your strongest model"] ``` @@ -238,7 +240,7 @@ For local development: clone the repo and - Continuing a stuck agent uses `SendMessage` when the harness provides it; where it is absent the skill falls back to re-dispatching with the packaged state. Neither path is a hard requirement. -- What "read-only" means here: scout, reviewer, verifier and test-runner +- What "read-only" means here: scout, reviewer, task-reviewer, verifier and test-runner block the file-edit tools (`Edit`, `Write`, `NotebookEdit`) at the tool level via `disallowedTools`. `Bash` (needed to run tests and inspect repos) and MCP tools remain available and could in principle mutate @@ -269,6 +271,7 @@ For local development: clone the repo and | "Run the tests" | `test-runner` | haiku / low | | "Implement tasks from the plan" | `implementer` | sonnet / medium (`model=opus` for complex work) | | "Review the diff" | `reviewer` | opus / high | + | Per-task review in a plan loop | `task-reviewer` | sonnet / high | | "Walk through the flow in the browser" | `e2e-runner` | sonnet / medium | The main session spends tokens only on planning, decisions, final @@ -284,9 +287,10 @@ Works with any plan-driven process (superpowers or similar): 2. Plan execution goes to `implementer` with a batch of related tasks per dispatch (one agent per batch, not per task - every fresh agent re-reads files from scratch). -3. Verification: `test-runner` after each batch, `reviewer` on the - completed chunk, and for high-risk diffs a final review in the main - session. +3. Verification: `test-runner` after each batch, `task-reviewer` + (sonnet) for the loop's per-task reviews and scoped re-reviews, + `reviewer` (opus) on the completed chunk or branch, and for high-risk + diffs a final review in the main session. ### "I don't want the expensive model" @@ -316,7 +320,8 @@ let Claude route via the skill: work goes out with `model=opus`. - "Review the diff" - Claude dispatches `reviewer` (opus). For high-risk diffs, ask for review in the main session instead - one expensive pass - is cheaper than a missed bug. + is cheaper than a missed bug. Per-task reviews inside a plan loop go to + `task-reviewer` (sonnet). - "Walk through the checkout flow in the browser" - Claude dispatches `e2e-runner` (sonnet). @@ -339,8 +344,9 @@ earns its cost (the knobs themselves: | --------- | ----- | ------ | -------------- | --------------- | | Exploration (`scout`) | sonnet | low | Finding and tracing code is retrieval, not reasoning - a cheap tier reports as well as a costly one, and the file volume stays in the subagent regardless. | The work is mechanical lookup; extra thinking buys nothing. | | Ordinary implementation (`implementer`) | sonnet | medium | Sonnet 5.5 pairs speed with intelligence at half the opus price ($2/$10 against $4/$20) - for work whose approach the plan already decided, the margin never changes the outcome. | The plan already decided the approach; the agent executes real logic, not design, and medium is where Sonnet 5.5's own guidance starts well-specified agentic coding. | -| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less), so escalate when in doubt. This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | +| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less). This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | | Code review (`reviewer`) | opus | high | Review is an asymmetric bet - one pass guards against a bug that costs far more if it ships, so it is the one place to prefer the top tier by default. | High: subtle correctness bugs hide from shallow reading. The case for `medium` grew with Opus 5.5 - its `medium` matches or exceeds Opus 5 at `high` on coding and it thinks more per turn at a given level, so the same pin costs more - but that is vendor evidence, not this plugin's own review eval; the pin stays high until a medium-vs-high review run on Opus 5.5 is measured. | +| Per-task review in a plan loop (`task-reviewer`) | sonnet | high | Superpowers SDD scales the model of its per-task reviews and scoped re-reviews to the diff, with a mid-tier floor; a sonnet-pinned agent takes those seats instead of `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and pre-PR review stays on `reviewer`. | High: same correctness job as `reviewer`, on a smaller diff. | | Tests / builds (`test-runner`) | haiku | low | Running a command and summarizing output is mechanical; the value is keeping raw logs out of the main context, not the model doing it. | Low: no reasoning, just report. | | Diff sanity gate (`verifier`) | haiku | medium | Checking a diff matches its task (scope, completeness, obvious breakage) is a cheap spot-check, not a quality judgment. | Medium: completeness is the job, and at `low` Haiku 5.5 "is more likely to skip a search, stop early, or skip a check". | | E2E / failure interpretation (`e2e-runner`) | sonnet | medium | Driving a browser and telling a product bug from a flake needs some judgment, but not top-tier reasoning. | Medium: real interpretation, clear method. | @@ -442,7 +448,7 @@ step up on evidence. | ------ | ------------ | ------------------------ | | low | Most efficient: significant token savings with some capability reduction. On the Opus 5 generation low/medium punch well above their weight. | Pins: `scout`, `test-runner` (on the cloud platforms `haiku` is Haiku 4.5, which has no effort knob, so there the haiku pins only document intent). | | medium | Balanced: real logic whose approach is already decided, at moderate savings. On Opus 5.5, Sonnet 5.5 and Haiku 5.5 this is the model's own default, so a pin at medium steps down only from a session running high or above. | Pins: `implementer`, `e2e-runner`, `surveyor`, `verifier`. Recommended session setting. | -| high | Full capability - Claude Code's per-model default except on Opus 5.5, Sonnet 5.5 and Haiku 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pin: `reviewer`. Main-session planning and final review. | +| high | Full capability - Claude Code's per-model default except on Opus 5.5, Sonnet 5.5 and Haiku 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pins: `reviewer`, `task-reviewer`. Main-session planning and final review. | | xhigh | Extended capability for long-horizon agentic/coding runs (multi-hour, token budgets in the millions). | Session-level or Workflow `effort` opt only - never an agent pin. | | max | Unconstrained token spend for frontier-grade problems; Anthropic notes diminishing returns and overthinking risk on routine work. | Not used by any pin - reach for it deliberately or not at all. | diff --git a/agents/task-reviewer.md b/agents/task-reviewer.md new file mode 100644 index 0000000..01cebb6 --- /dev/null +++ b/agents/task-reviewer.md @@ -0,0 +1,32 @@ +--- +name: task-reviewer +description: Per-task reviews and scoped re-reviews inside a plan-driven loop such as Superpowers subagent-driven-development, and any review explicitly asked for at the sonnet tier. NOT for final or whole-branch review, security/money/migrations/concurrency/public-contract changes, or pre-PR review - those go to reviewer. +model: sonnet +effort: high +disallowedTools: Edit, Write, NotebookEdit +--- + +You review one task's changes, or one fix round, for defects. + +The caller's prompt defines scope and output format - usually a review +template from the plan loop. Follow it exactly. When it gives no format, +use the reviewer agent's focus (correctness, regressions, security, +design, tests) and its report format: file:line, one-sentence defect, +concrete failure scenario, fix direction; end with safe to merge / needs +fixes / needs rework. + +Rules: + +- Escalate first: if the diff touches security, money, migrations, + concurrency or public contracts, or is too large to review well in one + pass, say so in the first line of the report so the caller can + re-dispatch `reviewer`. Then review what you can. +- Verify before reporting: for each suspected bug, trace the actual code + path that triggers it. Report only findings you can defend with a + concrete failure scenario. +- Review the whole diff before you report. Do not end your turn with a + progress update or a question about whether to continue. Stop early + only when nothing can move without the caller. +- Style nits only if the caller asks. Do not pad the report. +- Do not dispatch subagents. +- Never edit code. Review only. diff --git a/hooks/dispatch-counter.mjs b/hooks/dispatch-counter.mjs index 23b5d53..c46f74d 100644 --- a/hooks/dispatch-counter.mjs +++ b/hooks/dispatch-counter.mjs @@ -87,6 +87,7 @@ const AGENT_PINS = { "model-routing:verifier": { model: "haiku", effort: "medium" }, "model-routing:implementer": { model: "sonnet", effort: "medium" }, "model-routing:reviewer": { model: "opus", effort: "high" }, + "model-routing:task-reviewer": { model: "sonnet", effort: "high" }, }; // Model pins for OTHER plugins' agents, curated by hand from their frontmatter // the same way AGENT_PINS is - the difference is these files live outside this diff --git a/hooks/routing-anchor.md b/hooks/routing-anchor.md index 595dbdd..666cd14 100644 --- a/hooks/routing-anchor.md +++ b/hooks/routing-anchor.md @@ -2,8 +2,8 @@ MODEL ROUTING ACTIVE - expensive model thinks, cheap models grind. You cannot sw - Main session: planning, specs, docs, architecture, small interactive edits. Trivial questions answerable from context or general knowledge: answer inline - a dispatch costs more than the answer. - Exploration, by what has to come back: a judgement about what code does -> `scout`; a complete list or ordering verified by following the code -> `surveyor` (haiku); unverified candidate files -> built-in Explore with `model=haiku` (bare, it inherits the session model). With a code-graph MCP connected, query the graph first; scout verifies file:line. -- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result; when in doubt, opus. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. -- Review -> `reviewer`; high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. +- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. +- Review -> `reviewer`; plan-loop per-task reviews and scoped re-reviews (Superpowers SDD) -> `task-reviewer` (sonnet); high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. - Effort (low/medium/high/xhigh/max; Claude Code default medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on Opus 4.7, high elsewhere): low mechanical, medium implementation, high architecture/debugging/review, xhigh/max only long-horizon or frontier work. On xhigh/max review only what the plan, the user or these rules ask - no self-started second round, no extra reviewer subagents. A dispatch carries no effort param: bundled agents pin theirs, Workflow `agent()` takes an `effort` opt. - Stuck on the approach: the agent hands back; decide, then continue the same agent (SendMessage, else re-dispatch with its packaged state). Unfinished return with no blocker named: continue it the same way at the same tier. Failed or weak RESULT: fresh dispatch one step up (next tier via `model`, or higher Workflow `effort` when the miss is shallow thinking); a second failure comes to the main session. `stop_reason: "refusal"` is a classifier, not weakness: rephrase or drop, never re-dispatch a tier up. - Pins: a pin above the session model gets capped at the session model via `model` - a bare dispatch runs the pin. Never push a dispatch below the lower of pin and session model; when a cheaper tier fits, pick an agent pinned for it. CLAUDE_CODE_SUBAGENT_MODEL_FORCE overrides `model` and pins; plain CLAUDE_CODE_SUBAGENT_MODEL only decides bare unpinned dispatches. diff --git a/skills/model-routing/SKILL.md b/skills/model-routing/SKILL.md index 810b5ee..5352750 100644 --- a/skills/model-routing/SKILL.md +++ b/skills/model-routing/SKILL.md @@ -150,7 +150,8 @@ carries no effort param: | Complex implementation: multi-file refactor, subtle concurrency/security | subagent | `implementer` with `model=opus` | medium (pinned) | | Trivial mechanical tasks: renames, boilerplate, mirrored constants | subagent | sonnet | low | | Small interactive edits, quick fixes | main session | strongest | low | -| Code review of implemented work | subagent | `reviewer` (opus) | high | +| Per-task review and scoped re-review inside a plan loop (Superpowers SDD) | subagent | `task-reviewer` (sonnet) | high | +| Final/whole-branch, high-risk or pre-PR review of implemented work | subagent | `reviewer` (opus) | high | | Final review of high-risk or large diffs | main session | strongest | high | | Run tests/builds/linters, report failures | subagent | `test-runner` (haiku) | low | | Sanity-check a subagent's diff against its task | subagent | `verifier` (haiku) | medium | @@ -191,8 +192,7 @@ actually earns its cost: Opus 5 was a step-change over Opus 4.8 at unchanged price, and Opus 5.5 is stronger again at 20% less ($4/$20, cache reads 60% less), so the opus tier buys strictly more per dollar than when - this table was tuned - when in doubt between sonnet and opus for - implementation, take opus. That holds on every session model, Fable + this table was tuned. That holds on every session model, Fable included - see the fable-to-opus bullet below. - **Opus 5.5 to Sonnet 5.5 is a flat 50% off.** Both read cache at 0.05x of their own input, so every token type halves: the scout eval profile @@ -206,7 +206,12 @@ actually earns its cost: Opus 5 at `high` on coding, and at `high` it thinks more per turn than Opus 5 did, so the same pin now costs more. That is vendor evidence on model-level evals, not this plugin's review eval - the pin stays at - high until a medium-vs-high review run on Opus 5.5 is measured. + high until a medium-vs-high review run on Opus 5.5 is measured. The + per-task reviews and scoped re-reviews of a plan loop are the + exception: Superpowers SDD scales their model to the diff with a + mid-tier floor, so they go to `task-reviewer` (sonnet/high) rather than + to `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and + pre-PR review stays on `reviewer`. - **Tests / verification -> haiku.** Running a command and summarizing output is mechanical: `test-runner` stays at low effort, and the value is keeping raw output out of the main context, not the model doing it. @@ -414,7 +419,8 @@ with `/model-routing:stats`. dispatch whose session model cannot be read is left unjudged rather than guessed at. When the cheap tier genuinely fits the work, pick an agent pinned for it - (`test-runner`, `verifier`) instead of overriding a role agent downward; the + (`test-runner`, `verifier`, `task-reviewer` for a sonnet-tier review) + instead of overriding a role agent downward; the dispatch report lists below-pin dispatches in their own section. - Against the session model, a pin is a ceiling. A pin says "this task never needs more than X"; the session model says what the user is willing to pay.