diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index e207c6a..3259e2f 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "model-routing", - "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", + "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer, task-reviewer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", "version": "0.16.6", "author": { "name": "Artem Frolov" diff --git a/README.md b/README.md index e2f4ca4..6a8b072 100644 --- a/README.md +++ b/README.md @@ -95,7 +95,8 @@ reason written down in `evals/forced-dispatch-tier/NOTES.md`.) | `agents/test-runner.md` | haiku | low | Run tests/builds/linters, report failures compactly. Never fixes anything. | | `agents/e2e-runner.md` | sonnet | medium | Drive Playwright/E2E scenarios, interpret failures (product bug vs test bug vs flake). | | `agents/implementer.md` | sonnet | medium | Implement one well-defined task from an approved plan. Verifies its own work. Dispatch with `model=opus` for multi-file/architectural/subtle work. | -| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. | +| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. Final/whole-branch, high-risk and pre-PR review. | +| `agents/task-reviewer.md` | sonnet | high | Per-task reviews and scoped re-reviews inside a plan loop (Superpowers SDD), following the caller's review template. Flags diffs that need `reviewer` instead. | | `agents/verifier.md` | haiku | medium | Cheap gate on a subagent's diff: does it match the task (scope, completeness, obvious breakage)? Not a code review. | | `skills/model-routing/` | - | - | The routing table and delegation rules Claude follows when deciding where work goes. | | `hooks/routing-anchor.md` | - | - | Short routing anchor auto-injected at session start - zero config. | @@ -199,6 +200,7 @@ flowchart LR C["'Implement tasks 1-3 from the plan'"] --> IM["implementer
sonnet / medium"] D["'Refactor the payment pipeline'"] --> IMO["implementer
model=opus / medium"] E["'Review the diff'"] --> RV["reviewer
opus / high"] + G["'Review Task 2' (plan loop)"] --> TRV["task-reviewer
sonnet / high"] F["'Design the architecture'"] --> MS["main session
your strongest model"] ``` @@ -238,7 +240,7 @@ For local development: clone the repo and - Continuing a stuck agent uses `SendMessage` when the harness provides it; where it is absent the skill falls back to re-dispatching with the packaged state. Neither path is a hard requirement. -- What "read-only" means here: scout, reviewer, verifier and test-runner +- What "read-only" means here: scout, reviewer, task-reviewer, verifier and test-runner block the file-edit tools (`Edit`, `Write`, `NotebookEdit`) at the tool level via `disallowedTools`. `Bash` (needed to run tests and inspect repos) and MCP tools remain available and could in principle mutate @@ -269,6 +271,7 @@ For local development: clone the repo and | "Run the tests" | `test-runner` | haiku / low | | "Implement tasks from the plan" | `implementer` | sonnet / medium (`model=opus` for complex work) | | "Review the diff" | `reviewer` | opus / high | + | Per-task review in a plan loop | `task-reviewer` | sonnet / high | | "Walk through the flow in the browser" | `e2e-runner` | sonnet / medium | The main session spends tokens only on planning, decisions, final @@ -284,9 +287,10 @@ Works with any plan-driven process (superpowers or similar): 2. Plan execution goes to `implementer` with a batch of related tasks per dispatch (one agent per batch, not per task - every fresh agent re-reads files from scratch). -3. Verification: `test-runner` after each batch, `reviewer` on the - completed chunk, and for high-risk diffs a final review in the main - session. +3. Verification: `test-runner` after each batch, `task-reviewer` + (sonnet) for the loop's per-task reviews and scoped re-reviews, + `reviewer` (opus) on the completed chunk or branch, and for high-risk + diffs a final review in the main session. ### "I don't want the expensive model" @@ -316,7 +320,8 @@ let Claude route via the skill: work goes out with `model=opus`. - "Review the diff" - Claude dispatches `reviewer` (opus). For high-risk diffs, ask for review in the main session instead - one expensive pass - is cheaper than a missed bug. + is cheaper than a missed bug. Per-task reviews inside a plan loop go to + `task-reviewer` (sonnet). - "Walk through the checkout flow in the browser" - Claude dispatches `e2e-runner` (sonnet). @@ -339,8 +344,9 @@ earns its cost (the knobs themselves: | --------- | ----- | ------ | -------------- | --------------- | | Exploration (`scout`) | sonnet | low | Finding and tracing code is retrieval, not reasoning - a cheap tier reports as well as a costly one, and the file volume stays in the subagent regardless. | The work is mechanical lookup; extra thinking buys nothing. | | Ordinary implementation (`implementer`) | sonnet | medium | Sonnet 5.5 pairs speed with intelligence at half the opus price ($2/$10 against $4/$20) - for work whose approach the plan already decided, the margin never changes the outcome. | The plan already decided the approach; the agent executes real logic, not design, and medium is where Sonnet 5.5's own guidance starts well-specified agentic coding. | -| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less), so escalate when in doubt. This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | +| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less). This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | | Code review (`reviewer`) | opus | high | Review is an asymmetric bet - one pass guards against a bug that costs far more if it ships, so it is the one place to prefer the top tier by default. | High: subtle correctness bugs hide from shallow reading. The case for `medium` grew with Opus 5.5 - its `medium` matches or exceeds Opus 5 at `high` on coding and it thinks more per turn at a given level, so the same pin costs more - but that is vendor evidence, not this plugin's own review eval; the pin stays high until a medium-vs-high review run on Opus 5.5 is measured. | +| Per-task review in a plan loop (`task-reviewer`) | sonnet | high | Superpowers SDD scales the model of its per-task reviews and scoped re-reviews to the diff, with a mid-tier floor; a sonnet-pinned agent takes those seats instead of `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and pre-PR review stays on `reviewer`. | High: same correctness job as `reviewer`, on a smaller diff. | | Tests / builds (`test-runner`) | haiku | low | Running a command and summarizing output is mechanical; the value is keeping raw logs out of the main context, not the model doing it. | Low: no reasoning, just report. | | Diff sanity gate (`verifier`) | haiku | medium | Checking a diff matches its task (scope, completeness, obvious breakage) is a cheap spot-check, not a quality judgment. | Medium: completeness is the job, and at `low` Haiku 5.5 "is more likely to skip a search, stop early, or skip a check". | | E2E / failure interpretation (`e2e-runner`) | sonnet | medium | Driving a browser and telling a product bug from a flake needs some judgment, but not top-tier reasoning. | Medium: real interpretation, clear method. | @@ -442,7 +448,7 @@ step up on evidence. | ------ | ------------ | ------------------------ | | low | Most efficient: significant token savings with some capability reduction. On the Opus 5 generation low/medium punch well above their weight. | Pins: `scout`, `test-runner` (on the cloud platforms `haiku` is Haiku 4.5, which has no effort knob, so there the haiku pins only document intent). | | medium | Balanced: real logic whose approach is already decided, at moderate savings. On Opus 5.5, Sonnet 5.5 and Haiku 5.5 this is the model's own default, so a pin at medium steps down only from a session running high or above. | Pins: `implementer`, `e2e-runner`, `surveyor`, `verifier`. Recommended session setting. | -| high | Full capability - Claude Code's per-model default except on Opus 5.5, Sonnet 5.5 and Haiku 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pin: `reviewer`. Main-session planning and final review. | +| high | Full capability - Claude Code's per-model default except on Opus 5.5, Sonnet 5.5 and Haiku 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pins: `reviewer`, `task-reviewer`. Main-session planning and final review. | | xhigh | Extended capability for long-horizon agentic/coding runs (multi-hour, token budgets in the millions). | Session-level or Workflow `effort` opt only - never an agent pin. | | max | Unconstrained token spend for frontier-grade problems; Anthropic notes diminishing returns and overthinking risk on routine work. | Not used by any pin - reach for it deliberately or not at all. | diff --git a/agents/task-reviewer.md b/agents/task-reviewer.md new file mode 100644 index 0000000..01cebb6 --- /dev/null +++ b/agents/task-reviewer.md @@ -0,0 +1,32 @@ +--- +name: task-reviewer +description: Per-task reviews and scoped re-reviews inside a plan-driven loop such as Superpowers subagent-driven-development, and any review explicitly asked for at the sonnet tier. NOT for final or whole-branch review, security/money/migrations/concurrency/public-contract changes, or pre-PR review - those go to reviewer. +model: sonnet +effort: high +disallowedTools: Edit, Write, NotebookEdit +--- + +You review one task's changes, or one fix round, for defects. + +The caller's prompt defines scope and output format - usually a review +template from the plan loop. Follow it exactly. When it gives no format, +use the reviewer agent's focus (correctness, regressions, security, +design, tests) and its report format: file:line, one-sentence defect, +concrete failure scenario, fix direction; end with safe to merge / needs +fixes / needs rework. + +Rules: + +- Escalate first: if the diff touches security, money, migrations, + concurrency or public contracts, or is too large to review well in one + pass, say so in the first line of the report so the caller can + re-dispatch `reviewer`. Then review what you can. +- Verify before reporting: for each suspected bug, trace the actual code + path that triggers it. Report only findings you can defend with a + concrete failure scenario. +- Review the whole diff before you report. Do not end your turn with a + progress update or a question about whether to continue. Stop early + only when nothing can move without the caller. +- Style nits only if the caller asks. Do not pad the report. +- Do not dispatch subagents. +- Never edit code. Review only. diff --git a/hooks/dispatch-counter.mjs b/hooks/dispatch-counter.mjs index 23b5d53..c46f74d 100644 --- a/hooks/dispatch-counter.mjs +++ b/hooks/dispatch-counter.mjs @@ -87,6 +87,7 @@ const AGENT_PINS = { "model-routing:verifier": { model: "haiku", effort: "medium" }, "model-routing:implementer": { model: "sonnet", effort: "medium" }, "model-routing:reviewer": { model: "opus", effort: "high" }, + "model-routing:task-reviewer": { model: "sonnet", effort: "high" }, }; // Model pins for OTHER plugins' agents, curated by hand from their frontmatter // the same way AGENT_PINS is - the difference is these files live outside this diff --git a/hooks/routing-anchor.md b/hooks/routing-anchor.md index 595dbdd..666cd14 100644 --- a/hooks/routing-anchor.md +++ b/hooks/routing-anchor.md @@ -2,8 +2,8 @@ MODEL ROUTING ACTIVE - expensive model thinks, cheap models grind. You cannot sw - Main session: planning, specs, docs, architecture, small interactive edits. Trivial questions answerable from context or general knowledge: answer inline - a dispatch costs more than the answer. - Exploration, by what has to come back: a judgement about what code does -> `scout`; a complete list or ordering verified by following the code -> `surveyor` (haiku); unverified candidate files -> built-in Explore with `model=haiku` (bare, it inherits the session model). With a code-graph MCP connected, query the graph first; scout verifies file:line. -- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result; when in doubt, opus. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. -- Review -> `reviewer`; high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. +- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. +- Review -> `reviewer`; plan-loop per-task reviews and scoped re-reviews (Superpowers SDD) -> `task-reviewer` (sonnet); high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. - Effort (low/medium/high/xhigh/max; Claude Code default medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on Opus 4.7, high elsewhere): low mechanical, medium implementation, high architecture/debugging/review, xhigh/max only long-horizon or frontier work. On xhigh/max review only what the plan, the user or these rules ask - no self-started second round, no extra reviewer subagents. A dispatch carries no effort param: bundled agents pin theirs, Workflow `agent()` takes an `effort` opt. - Stuck on the approach: the agent hands back; decide, then continue the same agent (SendMessage, else re-dispatch with its packaged state). Unfinished return with no blocker named: continue it the same way at the same tier. Failed or weak RESULT: fresh dispatch one step up (next tier via `model`, or higher Workflow `effort` when the miss is shallow thinking); a second failure comes to the main session. `stop_reason: "refusal"` is a classifier, not weakness: rephrase or drop, never re-dispatch a tier up. - Pins: a pin above the session model gets capped at the session model via `model` - a bare dispatch runs the pin. Never push a dispatch below the lower of pin and session model; when a cheaper tier fits, pick an agent pinned for it. CLAUDE_CODE_SUBAGENT_MODEL_FORCE overrides `model` and pins; plain CLAUDE_CODE_SUBAGENT_MODEL only decides bare unpinned dispatches. diff --git a/skills/model-routing/SKILL.md b/skills/model-routing/SKILL.md index 810b5ee..5352750 100644 --- a/skills/model-routing/SKILL.md +++ b/skills/model-routing/SKILL.md @@ -150,7 +150,8 @@ carries no effort param: | Complex implementation: multi-file refactor, subtle concurrency/security | subagent | `implementer` with `model=opus` | medium (pinned) | | Trivial mechanical tasks: renames, boilerplate, mirrored constants | subagent | sonnet | low | | Small interactive edits, quick fixes | main session | strongest | low | -| Code review of implemented work | subagent | `reviewer` (opus) | high | +| Per-task review and scoped re-review inside a plan loop (Superpowers SDD) | subagent | `task-reviewer` (sonnet) | high | +| Final/whole-branch, high-risk or pre-PR review of implemented work | subagent | `reviewer` (opus) | high | | Final review of high-risk or large diffs | main session | strongest | high | | Run tests/builds/linters, report failures | subagent | `test-runner` (haiku) | low | | Sanity-check a subagent's diff against its task | subagent | `verifier` (haiku) | medium | @@ -191,8 +192,7 @@ actually earns its cost: Opus 5 was a step-change over Opus 4.8 at unchanged price, and Opus 5.5 is stronger again at 20% less ($4/$20, cache reads 60% less), so the opus tier buys strictly more per dollar than when - this table was tuned - when in doubt between sonnet and opus for - implementation, take opus. That holds on every session model, Fable + this table was tuned. That holds on every session model, Fable included - see the fable-to-opus bullet below. - **Opus 5.5 to Sonnet 5.5 is a flat 50% off.** Both read cache at 0.05x of their own input, so every token type halves: the scout eval profile @@ -206,7 +206,12 @@ actually earns its cost: Opus 5 at `high` on coding, and at `high` it thinks more per turn than Opus 5 did, so the same pin now costs more. That is vendor evidence on model-level evals, not this plugin's review eval - the pin stays at - high until a medium-vs-high review run on Opus 5.5 is measured. + high until a medium-vs-high review run on Opus 5.5 is measured. The + per-task reviews and scoped re-reviews of a plan loop are the + exception: Superpowers SDD scales their model to the diff with a + mid-tier floor, so they go to `task-reviewer` (sonnet/high) rather than + to `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and + pre-PR review stays on `reviewer`. - **Tests / verification -> haiku.** Running a command and summarizing output is mechanical: `test-runner` stays at low effort, and the value is keeping raw output out of the main context, not the model doing it. @@ -414,7 +419,8 @@ with `/model-routing:stats`. dispatch whose session model cannot be read is left unjudged rather than guessed at. When the cheap tier genuinely fits the work, pick an agent pinned for it - (`test-runner`, `verifier`) instead of overriding a role agent downward; the + (`test-runner`, `verifier`, `task-reviewer` for a sonnet-tier review) + instead of overriding a role agent downward; the dispatch report lists below-pin dispatches in their own section. - Against the session model, a pin is a ceiling. A pin says "this task never needs more than X"; the session model says what the user is willing to pay.