diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index e207c6a..3259e2f 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "model-routing", - "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", + "description": "Tiered model routing for token economy: the strongest model thinks, cheaper models grind. Ready-made subagents (sonnet scout, implementer, task-reviewer and E2E runner, haiku surveyor, test-runner and verifier, opus reviewer) plus a routing skill and dispatch stats.", "version": "0.16.6", "author": { "name": "Artem Frolov" diff --git a/README.md b/README.md index 5820879..6a8b072 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,7 @@ Read this before installing, because the honest answer has two halves and **A subagent is not free.** It starts with an empty context, so everything it reads is a cache *write*, while your main session pays cache *read* - 12.5x -cheaper, 25x on Opus 5.5, 50x on Fable 5.1 and Mythos 5.1 - for what it already has. Delegation converts cheap re-reads into +cheaper, 25x on Opus 5.5 and Sonnet 5.5, 50x on Fable 5.1 and Mythos 5.1 - for what it already has. Delegation converts cheap re-reads into expensive first-reads, and that bill lands whether or not this plugin is installed. @@ -91,12 +91,13 @@ reason written down in `evals/forced-dispatch-tier/NOTES.md`.) | Component | Model | Effort | Purpose | | --------- | ----- | ------ | ------- | | `agents/scout.md` | sonnet | low | Read-only codebase exploration: conclusions and file:line refs come back, file dumps stay out. | -| `agents/surveyor.md` | haiku | low | Read-only breadth: enumerate, list, trace a chain end to end. Measured correct on the cheap tier where `scout`'s judgement work is not. | +| `agents/surveyor.md` | haiku | medium | Read-only breadth: enumerate, list, trace a chain end to end. Measured correct on the cheap tier where `scout`'s judgement work is not. | | `agents/test-runner.md` | haiku | low | Run tests/builds/linters, report failures compactly. Never fixes anything. | | `agents/e2e-runner.md` | sonnet | medium | Drive Playwright/E2E scenarios, interpret failures (product bug vs test bug vs flake). | | `agents/implementer.md` | sonnet | medium | Implement one well-defined task from an approved plan. Verifies its own work. Dispatch with `model=opus` for multi-file/architectural/subtle work. | -| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. | -| `agents/verifier.md` | haiku | low | Cheap gate on a subagent's diff: does it match the task (scope, completeness, obvious breakage)? Not a code review. | +| `agents/reviewer.md` | opus | high | Review a diff for correctness bugs, ranked by severity. Final/whole-branch, high-risk and pre-PR review. | +| `agents/task-reviewer.md` | sonnet | high | Per-task reviews and scoped re-reviews inside a plan loop (Superpowers SDD), following the caller's review template. Flags diffs that need `reviewer` instead. | +| `agents/verifier.md` | haiku | medium | Cheap gate on a subagent's diff: does it match the task (scope, completeness, obvious breakage)? Not a code review. | | `skills/model-routing/` | - | - | The routing table and delegation rules Claude follows when deciding where work goes. | | `hooks/routing-anchor.md` | - | - | Short routing anchor auto-injected at session start - zero config. | | `hooks/dispatch-counter.mjs` | - | - | Logs every Agent dispatch; `stats`/`report`/`tokens` modes measure what stayed off the session model. | @@ -172,9 +173,9 @@ The honest reading: the automatic half is fully attributable, the behavioural ha For the behavioural half there is now a control group, just a much smaller one: [`evals/`](evals/README.md) runs the same question with and without the plugin loaded. The behavioural claim holds - on a codebase question the plugin arm delegated to `scout` in 3 of 3 runs and the plugin-free arm in 0 of 3, grepping inline despite having `Explore` available. -The money is a more interesting answer than "it saves". On a wide-reading question across a four-turn session, the expensive tier drops either way - opus spend down 14%, its input volume down about 25% - but the shipped configuration ends up **23% more expensive in total than not having the plugin at all**, because a fresh subagent pays cache *write* for everything it reads while a main session pays cache *read*, 12.5x cheaper (25x on Opus 5.5, 50x on Fable 5.1 and Mythos 5.1), for what it already has. So what this plugin reliably buys is room in the expensive model's context, not a smaller bill; where the bill also falls is a narrower claim, and it depends on the tier. +The money is a more interesting answer than "it saves". On a wide-reading question across a four-turn session, the expensive tier drops either way - opus spend down 14%, its input volume down about 25% - but the shipped configuration ends up **23% more expensive in total than not having the plugin at all**, because a fresh subagent pays cache *write* for everything it reads while a main session pays cache *read*, 12.5x cheaper (25x on Opus 5.5 and Sonnet 5.5, 50x on Fable 5.1 and Mythos 5.1), for what it already has. So what this plugin reliably buys is room in the expensive model's context, not a smaller bill; where the bill also falls is a narrower claim, and it depends on the tier. -Running that same question with `scout` pinned to haiku lands about a tenth *below* the no-plugin baseline - which looked like the fix until the tier was asked something subtler. On a question whose code contains a confident-looking wrong answer, haiku took the bait once in three runs and sonnet in none. Three runs cannot pin a failure rate, and the pin did not stay on sonnet because of the rate: it stayed because the failure is asymmetric, since a wrong answer sends the main session back to read the files itself and costs more than the cheaper tier saves. What the two results together support is a split rather than a pin change - breadth to `surveyor` (haiku), judgement to `scout` (sonnet). [`evals/README.md`](evals/README.md) carries the numbers, the arithmetic, and the three isolation attempts that produced numbers worth throwing away. +Running that same question with `scout` pinned to haiku (Haiku 4.5) lands about a tenth *below* the no-plugin baseline - which looked like the fix until the tier was asked something subtler. On a question whose code contains a confident-looking wrong answer, Haiku 4.5 took the bait once in three runs and sonnet in none. Three runs cannot pin a failure rate, and the pin did not stay on sonnet because of the rate: it stayed because the failure is asymmetric, since a wrong answer sends the main session back to read the files itself and costs more than the cheaper tier saves. What the two results together support is a split rather than a pin change - breadth to `surveyor` (haiku), judgement to `scout` (sonnet). [`evals/README.md`](evals/README.md) carries the numbers, the arithmetic, and the three isolation attempts that produced numbers worth throwing away. Where that volume actually ran with routing active: @@ -199,6 +200,7 @@ flowchart LR C["'Implement tasks 1-3 from the plan'"] --> IM["implementer
sonnet / medium"] D["'Refactor the payment pipeline'"] --> IMO["implementer
model=opus / medium"] E["'Review the diff'"] --> RV["reviewer
opus / high"] + G["'Review Task 2' (plan loop)"] --> TRV["task-reviewer
sonnet / high"] F["'Design the architecture'"] --> MS["main session
your strongest model"] ``` @@ -238,7 +240,7 @@ For local development: clone the repo and - Continuing a stuck agent uses `SendMessage` when the harness provides it; where it is absent the skill falls back to re-dispatching with the packaged state. Neither path is a hard requirement. -- What "read-only" means here: scout, reviewer, verifier and test-runner +- What "read-only" means here: scout, reviewer, task-reviewer, verifier and test-runner block the file-edit tools (`Edit`, `Write`, `NotebookEdit`) at the tool level via `disallowedTools`. `Bash` (needed to run tests and inspect repos) and MCP tools remain available and could in principle mutate @@ -255,7 +257,7 @@ For local development: clone the repo and plan offers). The plugin never changes it - the main session is where planning and decisions happen, so give it the strongest tier you are willing to pay for. Session effort: left unset, you get Claude Code's - per-model default - medium on Opus 5.5 and Sonnet 5.5, xhigh on Opus 4.7, high on every + per-model default - medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on Opus 4.7, high on every other model that has the knob, and some models have no effort knob at all (see [the effort ladder](#model-tiers-and-effort-ladder)); dropping the session to medium is the cost-conscious pick when the main @@ -265,10 +267,11 @@ For local development: clone the repo and | You ask | Who runs it | Model / effort | | ------- | ----------- | -------------- | | "Where is X handled?" | `scout` | sonnet / low | - | "List every stage this order goes through" | `surveyor` | haiku / low | + | "List every stage this order goes through" | `surveyor` | haiku / medium | | "Run the tests" | `test-runner` | haiku / low | | "Implement tasks from the plan" | `implementer` | sonnet / medium (`model=opus` for complex work) | | "Review the diff" | `reviewer` | opus / high | + | Per-task review in a plan loop | `task-reviewer` | sonnet / high | | "Walk through the flow in the browser" | `e2e-runner` | sonnet / medium | The main session spends tokens only on planning, decisions, final @@ -284,9 +287,10 @@ Works with any plan-driven process (superpowers or similar): 2. Plan execution goes to `implementer` with a batch of related tasks per dispatch (one agent per batch, not per task - every fresh agent re-reads files from scratch). -3. Verification: `test-runner` after each batch, `reviewer` on the - completed chunk, and for high-risk diffs a final review in the main - session. +3. Verification: `test-runner` after each batch, `task-reviewer` + (sonnet) for the loop's per-task reviews and scoped re-reviews, + `reviewer` (opus) on the completed chunk or branch, and for high-risk + diffs a final review in the main session. ### "I don't want the expensive model" @@ -316,7 +320,8 @@ let Claude route via the skill: work goes out with `model=opus`. - "Review the diff" - Claude dispatches `reviewer` (opus). For high-risk diffs, ask for review in the main session instead - one expensive pass - is cheaper than a missed bug. + is cheaper than a missed bug. Per-task reviews inside a plan loop go to + `task-reviewer` (sonnet). - "Walk through the checkout flow in the browser" - Claude dispatches `e2e-runner` (sonnet). @@ -339,10 +344,11 @@ earns its cost (the knobs themselves: | --------- | ----- | ------ | -------------- | --------------- | | Exploration (`scout`) | sonnet | low | Finding and tracing code is retrieval, not reasoning - a cheap tier reports as well as a costly one, and the file volume stays in the subagent regardless. | The work is mechanical lookup; extra thinking buys nothing. | | Ordinary implementation (`implementer`) | sonnet | medium | Sonnet 5.5 pairs speed with intelligence at half the opus price ($2/$10 against $4/$20) - for work whose approach the plan already decided, the margin never changes the outcome. | The plan already decided the approach; the agent executes real logic, not design, and medium is where Sonnet 5.5's own guidance starts well-specified agentic coding. | -| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less), so escalate when in doubt. This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | +| Complex implementation (`implementer` `model=opus`) | opus | medium (pinned) | Multi-file or cross-layer changes, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result - a wrong approach is expensive, and Opus 5.5 is stronger again at 20% less than Opus 5 ($4/$20, cache reads 60% less). This holds on every session model, Fable included: Opus 5.5 is cheaper than Fable 5.1 on every token type. (Ambiguous tasks are not an escalation case: implementer stops on ambiguity by contract - clarify first.) | The `model=opus` dispatch changes the model only - the Agent tool has no effort param, so the pinned medium stays; the escalation buys the tier, not extra thinking. | | Code review (`reviewer`) | opus | high | Review is an asymmetric bet - one pass guards against a bug that costs far more if it ships, so it is the one place to prefer the top tier by default. | High: subtle correctness bugs hide from shallow reading. The case for `medium` grew with Opus 5.5 - its `medium` matches or exceeds Opus 5 at `high` on coding and it thinks more per turn at a given level, so the same pin costs more - but that is vendor evidence, not this plugin's own review eval; the pin stays high until a medium-vs-high review run on Opus 5.5 is measured. | +| Per-task review in a plan loop (`task-reviewer`) | sonnet | high | Superpowers SDD scales the model of its per-task reviews and scoped re-reviews to the diff, with a mid-tier floor; a sonnet-pinned agent takes those seats instead of `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and pre-PR review stays on `reviewer`. | High: same correctness job as `reviewer`, on a smaller diff. | | Tests / builds (`test-runner`) | haiku | low | Running a command and summarizing output is mechanical; the value is keeping raw logs out of the main context, not the model doing it. | Low: no reasoning, just report. | -| Diff sanity gate (`verifier`) | haiku | low | Checking a diff matches its task (scope, completeness, obvious breakage) is a cheap spot-check, not a quality judgment. | Low: pattern-matching against the task, not deep analysis. | +| Diff sanity gate (`verifier`) | haiku | medium | Checking a diff matches its task (scope, completeness, obvious breakage) is a cheap spot-check, not a quality judgment. | Medium: completeness is the job, and at `low` Haiku 5.5 "is more likely to skip a search, stop early, or skip a check". | | E2E / failure interpretation (`e2e-runner`) | sonnet | medium | Driving a browser and telling a product bug from a flake needs some judgment, but not top-tier reasoning. | Medium: real interpretation, clear method. | | Planning, architecture, high-risk final review | main session (strongest) | high | These set the direction everything else follows - the one place raw capability changes the outcome most. | High: a wrong call here is the most expensive kind to unwind. | @@ -393,15 +399,13 @@ exact numbers live on | ------ | ------------ | ------------------- | | fable | 2.5x | Frontier reasoning. A main-session choice for all-hard-reasoning days (architecture, subtle debugging hunts) - not a dispatch target. | | opus | 1x | The escalation tier: code review, multi-file/cross-layer implementation, security/money/concurrency-sensitive changes. Opus 5 made this tier a step stronger at the same price; Opus 5.5 stronger again at a lower one ($4/$20). | -| sonnet | 0.5x (permanent $2/$10) | The workhorse: ordinary implementation from an approved plan, exploration, E2E driving. Sonnet 5.5 is the best speed-intelligence combination on offer; cache reads cost the same as opus ($0.20/MTok), so the tier saves on first reads, cache writes and output - 42% off on the one profile measured here, less as re-reads take over the bill. | -| haiku | 0.25x | Mechanical grind: test/build runs, diff sanity checks, trivial sweeps. | +| sonnet | 0.5x (permanent $2/$10) | The workhorse: ordinary implementation from an approved plan, exploration, E2E driving. Sonnet 5.5 is the best speed-intelligence combination on offer, at half of Opus 5.5 on every token type, cache reads included ($0.10 against $0.20/MTok). | +| haiku | 0.025x (Haiku 5.5, prompts up to 100K tokens; 0.125x above), 0.25x (Haiku 4.5) | Mechanical grind: test/build runs, diff sanity checks, trivial sweeps. | Generation notes (as of the Sonnet 5.5 launch, 28 September 2026 - these rot fastest, re-verify against Anthropic's model overview): Sonnet 5.5 keeps -Sonnet 5's prices exactly, including a $0.20/MTok cache read - the rate -Opus 5.5 also charges, so the two tiers cost the same per re-read token -and differ only on first reads, cache writes and output - a parity Opus -5.5 already had with Sonnet 5. It +Sonnet 5's $2/$10 but reads cache at 0.05x, $0.10/MTok, so it costs half +of Opus 5.5 on every token type. It defaults to `medium` effort in Claude Code and starts well-specified agentic coding there. Opus 5.5 is cheaper than Opus 5 ($4/$20 vs $5/$25), defaults to `medium` too, and per Anthropic is stronger at that lower @@ -416,17 +420,17 @@ work - a session-model choice, not a dispatch target. The effort ladder - the second knob. Unset effort means Claude Code's per-model default: `high` on every model that supports effort, except -`medium` on Opus 5.5 and Sonnet 5.5 and `xhigh` on Opus 4.7 - a recommendation is +`medium` on Opus 5.5, Sonnet 5.5 and Haiku 5.5 and `xhigh` on Opus 4.7 - a recommendation is a value you pass, and on Opus 4.7 that recommendation IS what runs when you pass nothing; Opus 4.8 still defaults to `high`. Levels are saved per model under `modelSettings`, and a top-level `effortLevel` in the user settings file no longer reaches Opus 5.5 or anything released after it, Sonnet 5.5 included. Support itself is an explicit list rather than a -version cutoff: Fable 5, Mythos 5, Opus 5 and Sonnet 5 with their 5.5 successors, Opus 4.8 and Opus 4.7 +version cutoff: Fable 5, Mythos 5, Opus 5 and Sonnet 5 with their 5.5 successors, Haiku 5.5, Opus 4.8 and Opus 4.7 take the whole ladder, Opus 4.6 and Sonnet 4.6 take everything but `xhigh`, and a -model absent from that list - Haiku 4.5 among them - has no effort knob at -all. Setting a level a model does not support runs the highest supported +model absent from that list - Haiku 4.5 among them, still what `haiku` +resolves to on the cloud platforms - has no effort knob at all. Setting a level a model does not support runs the highest supported level at or below it, so `xhigh` becomes `high` on Opus 4.6. The vendor recommendation is per-generation ([effort @@ -442,9 +446,9 @@ step up on evidence. | Effort | What it buys | Where the plugin uses it | | ------ | ------------ | ------------------------ | -| low | Most efficient: significant token savings with some capability reduction. On the Opus 5 generation low/medium punch well above their weight. | Pins: `scout`, `surveyor`, `test-runner`, `verifier` (haiku has no effort knob, so those three pins only document intent). | -| medium | Balanced: real logic whose approach is already decided, at moderate savings. On Opus 5.5 and Sonnet 5.5 this is the model's own default, so a pin at medium steps down only from a session running high or above. | Pins: `implementer`, `e2e-runner`. Recommended session setting. | -| high | Full capability - Claude Code's per-model default except on Opus 5.5 and Sonnet 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pin: `reviewer`. Main-session planning and final review. | +| low | Most efficient: significant token savings with some capability reduction. On the Opus 5 generation low/medium punch well above their weight. | Pins: `scout`, `test-runner` (on the cloud platforms `haiku` is Haiku 4.5, which has no effort knob, so there the haiku pins only document intent). | +| medium | Balanced: real logic whose approach is already decided, at moderate savings. On Opus 5.5, Sonnet 5.5 and Haiku 5.5 this is the model's own default, so a pin at medium steps down only from a session running high or above. | Pins: `implementer`, `e2e-runner`, `surveyor`, `verifier`. Recommended session setting. | +| high | Full capability - Claude Code's per-model default except on Opus 5.5, Sonnet 5.5 and Haiku 5.5 (medium) and Opus 4.7 (xhigh). Complex reasoning, subtle debugging, high-risk review. | Pins: `reviewer`, `task-reviewer`. Main-session planning and final review. | | xhigh | Extended capability for long-horizon agentic/coding runs (multi-hour, token budgets in the millions). | Session-level or Workflow `effort` opt only - never an agent pin. | | max | Unconstrained token spend for frontier-grade problems; Anthropic notes diminishing returns and overthinking risk on routine work. | Not used by any pin - reach for it deliberately or not at all. | @@ -464,8 +468,8 @@ coordination, and final review. That makes the balanced default: where the whole session sits in the seat that tier is worth. Drop to a **Sonnet** session for pure-implementation days with no hard decisions. - **Effort: medium** as the everyday session setting - a deliberate step - DOWN from Claude Code's per-model default of high (Opus 5.5 and Sonnet - 5.5 excepted: there medium already IS the default). Use **high** for sessions + DOWN from Claude Code's per-model default of high (Opus 5.5, Sonnet 5.5 + and Haiku 5.5 excepted: there medium already IS the default). Use **high** for sessions built around architecture or subtle debugging. Session effort mainly governs main-session work - the bundled agents pin their own - so raise it when the thinking you keep in the main seat is genuinely hard, not @@ -480,7 +484,8 @@ haiku), so when a new family member ships (Opus 5 -> Opus 5.5, then Sonnet 5 -> Sonnet 5.5), reviewer, the sonnet-pinned agents and every `model=opus` escalation upgrade automatically on the Anthropic API - no plugin update, no config change. The cloud platforms resolve the same -aliases to older models, so the figures below are first-party ones. Prices and every +aliases to older models (`haiku` to Haiku 4.5 on all four, `opus` to +Opus 4.6 on Foundry), so the figures below are first-party ones. Prices and every rule derived from them do not follow the alias, though: the stats price table and the cost-driven exceptions above need a check on each launch. `/model-routing:stats` shows which @@ -666,7 +671,7 @@ At API list prices (rates as of 2026-09-09), this is what the volume above would Read it as a counterfactual, which is exactly what it is: on a subscription you pay none of this, and the difference carries the same assumption as the volume figures above - that every subagent would otherwise have inherited the session model, which agents pinned by other plugins would not. -The arithmetic is per token type rather than per token, because they are priced an order of magnitude apart: base input at the model rate, cache reads at 0.05x on Opus 5.5, 0.025x on Fable 5.1 and Mythos 5.1, 0.1x elsewhere, five-minute cache writes at 1.25x, one-hour writes at 2x, and output at its own rate. Cache reads dominate real transcripts, so a flat volume multiply would overstate the bill several times over. Transcripts break cache writes down by TTL; a line carrying only a total is charged at the cheaper five-minute rate. +The arithmetic is per token type rather than per token, because they are priced an order of magnitude apart: base input at the model rate, cache reads at 0.05x on Opus 5.5 and Sonnet 5.5, 0.025x on Fable 5.1 and Mythos 5.1, 0.1x elsewhere, five-minute cache writes at 1.25x, one-hour writes at 2x, and output at its own rate. Haiku 5.5 prices each request on its own card: above 100K prompt tokens, cache reads and writes included, the whole request bills at $0.50/$2.50 instead of $0.10/$0.50. Cache reads dominate real transcripts, so a flat volume multiply would overstate the bill several times over. Transcripts break cache writes down by TTL; a line carrying only a total is charged at the cheaper five-minute rate. Several documented modifiers push these figures DOWNWARD, so treat them as conservative rather than optimistic. Models from Opus 4.7, Sonnet 5 and Fable 5 onward use a tokenizer producing roughly 30% more tokens for the same text than Sonnet 4.6 and earlier, so re-pricing a cheap model's token count at an expensive model's rate understates what that work would really have cost there. Nothing in a usage line reveals whether fast mode (double rates on Opus 5.5, Opus 5 and Opus 4.8) or US-only inference (1.1x from 4.6 onward) applied, so neither is added. And a model absent from the price table is excluded from every figure and declared on its own line, never counted as free - a window where nothing can be priced prints that declaration instead of figures, because silence would read as "this cost nothing". diff --git a/agents/surveyor.md b/agents/surveyor.md index 11deb3c..fddee8b 100644 --- a/agents/surveyor.md +++ b/agents/surveyor.md @@ -2,7 +2,7 @@ name: surveyor description: Read-only breadth sweeps over a codebase - enumerate, list, trace a chain end to end ("list every stage in order", "which files import X", "where does this pipeline end"). Returns the list or the ordering, never file dumps. Use scout instead when the answer needs judgement about what code does. Never modifies anything. model: haiku -effort: low +effort: medium disallowedTools: Agent, SendMessage, Edit, Write, NotebookEdit --- diff --git a/agents/task-reviewer.md b/agents/task-reviewer.md new file mode 100644 index 0000000..01cebb6 --- /dev/null +++ b/agents/task-reviewer.md @@ -0,0 +1,32 @@ +--- +name: task-reviewer +description: Per-task reviews and scoped re-reviews inside a plan-driven loop such as Superpowers subagent-driven-development, and any review explicitly asked for at the sonnet tier. NOT for final or whole-branch review, security/money/migrations/concurrency/public-contract changes, or pre-PR review - those go to reviewer. +model: sonnet +effort: high +disallowedTools: Edit, Write, NotebookEdit +--- + +You review one task's changes, or one fix round, for defects. + +The caller's prompt defines scope and output format - usually a review +template from the plan loop. Follow it exactly. When it gives no format, +use the reviewer agent's focus (correctness, regressions, security, +design, tests) and its report format: file:line, one-sentence defect, +concrete failure scenario, fix direction; end with safe to merge / needs +fixes / needs rework. + +Rules: + +- Escalate first: if the diff touches security, money, migrations, + concurrency or public contracts, or is too large to review well in one + pass, say so in the first line of the report so the caller can + re-dispatch `reviewer`. Then review what you can. +- Verify before reporting: for each suspected bug, trace the actual code + path that triggers it. Report only findings you can defend with a + concrete failure scenario. +- Review the whole diff before you report. Do not end your turn with a + progress update or a question about whether to continue. Stop early + only when nothing can move without the caller. +- Style nits only if the caller asks. Do not pad the report. +- Do not dispatch subagents. +- Never edit code. Review only. diff --git a/agents/verifier.md b/agents/verifier.md index a1918d9..2b3cedd 100644 --- a/agents/verifier.md +++ b/agents/verifier.md @@ -2,7 +2,7 @@ name: verifier description: Cheap gate before accepting another agent's diff - does it match the task (scope, completeness, obvious breakage)? Returns PASS/FAIL with reasons. NOT a code review - it catches "did the wrong thing"; real reviews go to reviewer. model: haiku -effort: low +effort: medium disallowedTools: Edit, Write, NotebookEdit --- diff --git a/hooks/dispatch-counter.mjs b/hooks/dispatch-counter.mjs index aca5ca5..c46f74d 100644 --- a/hooks/dispatch-counter.mjs +++ b/hooks/dispatch-counter.mjs @@ -81,12 +81,13 @@ function sessionFilterFromArgs(argv) { // Keep in sync with agents/*.md; one sync test guards both columns. const AGENT_PINS = { "model-routing:scout": { model: "sonnet", effort: "low" }, - "model-routing:surveyor": { model: "haiku", effort: "low" }, + "model-routing:surveyor": { model: "haiku", effort: "medium" }, "model-routing:test-runner": { model: "haiku", effort: "low" }, "model-routing:e2e-runner": { model: "sonnet", effort: "medium" }, - "model-routing:verifier": { model: "haiku", effort: "low" }, + "model-routing:verifier": { model: "haiku", effort: "medium" }, "model-routing:implementer": { model: "sonnet", effort: "medium" }, "model-routing:reviewer": { model: "opus", effort: "high" }, + "model-routing:task-reviewer": { model: "sonnet", effort: "high" }, }; // Model pins for OTHER plugins' agents, curated by hand from their frontmatter // the same way AGENT_PINS is - the difference is these files live outside this @@ -190,21 +191,21 @@ const EFFORT_SUPPORT = [ // mythos-5 rides with fable-5 here for the same reason it does in PRICES and // TIER_PATTERNS: same generation, same knobs. Leaving it out of this one table // was the only way the three could disagree, and it did. - [/fable-5|mythos-5|opus-5|sonnet-5|opus-4-8|opus-4-7/, ["low", "medium", "high", "xhigh", "max"]], + [/fable-5|mythos-5|opus-5|sonnet-5|haiku-5-5|opus-4-8|opus-4-7/, ["low", "medium", "high", "xhigh", "max"]], [/opus-4-6|sonnet-4-6/, ["low", "medium", "high", "max"]], ]; const EFFORT_LADDER = ["low", "medium", "high", "xhigh", "max"]; const effortLevelsFor = (m) => (m ? EFFORT_SUPPORT.find(([re]) => re.test(m))?.[1] ?? null : null); // Claude Code's own model default (Model configuration, "Adjust effort level"): -// "Opus 5.5 and Sonnet 5.5 default to `medium`", Opus 4.7 to `xhigh`, `high` on -// every other model that supports effort. This is Claude Code's default, not the -// API's - the API runs both Opus 4.7 and Sonnet 5.5 at `high` - and a session's -// effort is what Claude Code sends. No support, no default - an unlisted or +// "`high` on every model that supports effort, except that Opus 5.5, Sonnet 5.5, +// and Haiku 5.5 default to `medium`, Opus 4.7 defaults to `xhigh`". This is +// Claude Code's default, not the API's - the API runs both Opus 4.7 and Sonnet +// 5.5 at `high` - and a session's effort is what Claude Code sends. No support, no default - an unlisted or // unrecognized session model never receives a fabricated level. function defaultEffortFor(sessionModel) { if (!effortLevelsFor(sessionModel)) return null; - if (/opus-5-5|sonnet-5-5/.test(sessionModel)) return "medium"; + if (/opus-5-5|sonnet-5-5|haiku-5-5/.test(sessionModel)) return "medium"; if (/opus-4-7/.test(sessionModel)) return "xhigh"; return "high"; } @@ -312,7 +313,13 @@ const shortModel = (m) => m ? m.replace(/^claude-/, "").replace(/-\d{8}$/, "") : // (Opus 4.1 bills at three times Opus 4.5) and a loose pattern would quietly // misprice a retired model. A model absent from this table is reported as // unpriced volume, never as zero. -const PRICES_ASOF = "2026-09-29"; +const PRICES_ASOF = "2026-10-13"; +// A row's optional fourth element is the long-prompt card [inRate, outRate]: +// "A request's prompt length counts all of its input tokens, including cache +// reads and cache writes. Each request is priced on its own." Above +// LONG_PROMPT tokens the request bills on that card; exactly LONG_PROMPT is +// still the base card. +const LONG_PROMPT = 100000; const PRICES = [ // Retired families first: a looser pattern below must not claim them. [/opus-4-1-|opus-4-20/, () => [15, 75]], @@ -325,19 +332,22 @@ const PRICES = [ // Opus 5.5 cache reads at 0.05x ($0.20/MTok), ahead of the looser opus-5 row. [/opus-5-5/, () => [4, 20, 0.05]], [/opus-5|opus-4-8|opus-4-7|opus-4-6|opus-4-5/, () => [5, 25]], + // "Cache hits and refreshes on Claude Opus 5.5 and Claude Sonnet 5.5 are + // priced at 0.05x the base input price": $0.10/MTok, ahead of the sonnet-5 row. + [/sonnet-5-5/, () => [2, 10, 0.05]], // Permanent $2/$10: the scheduled increase to $3/$15 on 2026-09-01 was - // called off. Sonnet 5.5 rides this row - "the same prices as Claude Sonnet 5, - // including prompt caching and batch processing rates" - so its cache reads - // stay at the standard 0.1x, $0.20/MTok, the rate Opus 5.5 also charges. + // called off. [/sonnet-5/, () => [2, 10]], [/sonnet-4-6|sonnet-4-5|sonnet-4-20/, () => [3, 15]], + // $0.10/$0.50 up to LONG_PROMPT, $0.50/$2.50 above it. + [/haiku-5-5/, () => [0.1, 0.5, CACHE_READ, [0.5, 2.5]]], [/haiku-4-5/, () => [1, 5]], ]; // Prompt-caching multipliers, quoted from the same page: a 5-minute cache write -// costs 1.25x base input, a 1-hour write 2x. Cache reads are 0.1x base input -// for every model except where the page states otherwise (Fable 5.1 and -// Mythos 5.1 read at 0.025x, Opus 5.5 at 0.05x - the row's third element -// overrides CACHE_READ). +// costs 1.25x base input, a 1-hour write 2x, of whichever card the request bills +// on. Cache reads are 0.1x base input for every model except where the page +// states otherwise (Fable 5.1 and Mythos 5.1 read at 0.025x, Opus 5.5 and +// Sonnet 5.5 at 0.05x - the row's third element overrides CACHE_READ). const CACHE_WRITE_5M = 1.25, CACHE_WRITE_1H = 2, CACHE_READ = 0.1; // Billable input volume: everything the model read, however it was cached. @@ -346,17 +356,24 @@ const volOf = (v) => v.in + v.cr + v.cw5 + v.cw1h; // Dollars for one model's token counts, or null when the model is not on the // price table. `at` is kept as the hook for a future dated rate; no row on -// the table uses it today. +// the table uses it today. `v.long` holds the share of `v` that came from +// requests above LONG_PROMPT; a row without a long card prices all of `v` on +// its base card. function costOf(model, v, at) { const row = model ? PRICES.find(([re]) => re.test(model)) : null; if (!row) return null; - const [inRate, outRate, crMul = CACHE_READ] = row[1](at); - const perTok = inRate / 1e6; - return (v.in * perTok) - + (v.cw5 * perTok * CACHE_WRITE_5M) - + (v.cw1h * perTok * CACHE_WRITE_1H) - + (v.cr * perTok * crMul) - + (v.out * outRate / 1e6); + const [inRate, outRate, crMul = CACHE_READ, long] = row[1](at); + const card = (c, inR, outR) => { + const perTok = inR / 1e6; + return (c.in * perTok) + + (c.cw5 * perTok * CACHE_WRITE_5M) + + (c.cw1h * perTok * CACHE_WRITE_1H) + + (c.cr * perTok * crMul) + + (c.out * outR / 1e6); + }; + if (!long || !v.long) return card(v, inRate, outRate); + const base = Object.fromEntries(Object.keys(v.long).map((k) => [k, v[k] - v.long[k]])); + return card(base, inRate, outRate) + card(v.long, ...long); } // Bounded fd reads, NOT readFileSync - these run inside the PostToolUse @@ -629,7 +646,7 @@ if (process.argv[2] === "stats" || process.argv[2] === "report") { `Effort: ${inherited.length} of ${withEffort.length} dispatches ran on an agent type carrying no EFFORT pin this plugin knows about, and so inherited the session level${byLevel ? ` (${byLevel})` : ""}. Only the bundled agents pin effort here - a foreign agent with a known model pin still counts as inheriting effort.`, ` The bundled agents pin theirs in frontmatter, so routing a mechanical errand through a role agent can buy a cheaper model tier and, where that model supports effort and the pinned level is lower than the one it would otherwise inherit, a lower effort too. An agent from anywhere else may pin its own effort, which is invisible here and counted as inherited.`, ...(inferred ? [` ${inferred} of these levels are the documented model default rather than an observed setting.`] : []), - ` Source order is CLAUDE_CODE_EFFORT_LEVEL, then the level saved for the model in modelSettings or a settings effortLevel, then the model default (medium on Opus 5.5 and Sonnet 5.5, xhigh on Opus 4.7, high elsewhere). Four states can override that and none are visible here: a /effort or --effort choice inside a running session, ultracode, an organization effort cap, and a per-model maxEffortLevel cap.`, + ` Source order is CLAUDE_CODE_EFFORT_LEVEL, then the level saved for the model in modelSettings or a settings effortLevel, then the model default (medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on Opus 4.7, high elsewhere). Four states can override that and none are visible here: a /effort or --effort choice inside a running session, ultracode, an organization effort cap, and a per-model maxEffortLevel cap.`, ); } // Grouped sections instead of per-row v/- markers: the reader should not @@ -747,7 +764,7 @@ if (process.argv[2] === "tokens") { // lines. Dedup happens BEFORE the window filter so a response whose lines // straddle a boundary counts in exactly one window, the one holding its // last line, rather than in both or neither. - const fileVols = new Map(); // model -> { in, out, cr, cw5, cw1h } + const fileVols = new Map(); // model -> { in, out, cr, cw5, cw1h, long: same shape } let text; try { text = readFileSync(p, "utf-8"); } catch { unreadablePaths.add(p); return fileVols; } const lastById = new Map(); // message.id -> [line's obj, its message] @@ -768,9 +785,11 @@ if (process.argv[2] === "tokens") { if (Number.isFinite(lts)) { if (lts < win.start || lts >= win.end) continue; } else if (!mtimeInWindow) continue; - const v = fileVols.get(m.model) ?? { in: 0, out: 0, cr: 0, cw5: 0, cw1h: 0 }; - v.in += u.input_tokens ?? 0; v.out += u.output_tokens ?? 0; - v.cr += u.cache_read_input_tokens ?? 0; + const zero = () => ({ in: 0, out: 0, cr: 0, cw5: 0, cw1h: 0 }); + const v = fileVols.get(m.model) ?? { ...zero(), long: zero() }; + const req = zero(); + req.in = u.input_tokens ?? 0; req.out = u.output_tokens ?? 0; + req.cr = u.cache_read_input_tokens ?? 0; // Cache writes are split by TTL because they are priced differently // (1.25x base input at 5 minutes, 2x at an hour). Whatever the breakdown // does not account for - a line carrying only the flat total, or a TTL @@ -781,8 +800,13 @@ if (process.argv[2] === "tokens") { const cc = u.cache_creation; const e5 = cc?.ephemeral_5m_input_tokens ?? 0; const e1h = cc?.ephemeral_1h_input_tokens ?? 0; - v.cw1h += e1h; - v.cw5 += e5 + Math.max(0, (u.cache_creation_input_tokens ?? 0) - e5 - e1h); + req.cw1h = e1h; + req.cw5 = e5 + Math.max(0, (u.cache_creation_input_tokens ?? 0) - e5 - e1h); + // One deduped line is one request, so this is where its rate card is + // decided. v keeps the totals; v.long repeats the long-prompt share. + for (const b of volOf(req) > LONG_PROMPT ? [v, v.long] : [v]) { + for (const k of Object.keys(req)) b[k] += req[k]; + } fileVols.set(m.model, v); } return fileVols; @@ -994,6 +1018,8 @@ if (process.argv[2] === "tokens") { // tracked separately: the two need different sentences, and reporting // both as "ran on an unpriced model" was false for the session-side case, // where the tokens ran on a perfectly priceable model. + // v.long was split with the tokenizer of the model that ran, so the + // inherited side's long-prompt share is an approximation. const ran = costOf(model, v, priceAt); const inherited = costOf(sessionModel, v, priceAt); if (ran == null) unpricedVol += vol; diff --git a/hooks/dispatch-counter.test.mjs b/hooks/dispatch-counter.test.mjs index 7e9879d..fd81043 100644 --- a/hooks/dispatch-counter.test.mjs +++ b/hooks/dispatch-counter.test.mjs @@ -1008,6 +1008,17 @@ test("an unset effortLevel records the documented model default", () => { assert.equal(sonnet55.effortFrom, "default"); const sonnet5 = dispatchWithSettings({ sessionModel: "claude-sonnet-5" }); assert.equal(sonnet5.effort, "high"); + const haiku55 = dispatchWithSettings({ sessionModel: "claude-haiku-5-5" }); + assert.equal(haiku55.effort, "medium"); + assert.equal(haiku55.effortFrom, "default"); +}); + +test("haiku-5.5 supports the full effort ladder, unlike haiku-4.5", () => { + const xhigh = dispatchWithSettings({ sessionModel: "claude-haiku-5-5", settings: { project: { effortLevel: "xhigh" } } }); + assert.equal(xhigh.effort, "xhigh"); + assert.equal(xhigh.effortFrom, "settings"); + const max = dispatchWithSettings({ sessionModel: "claude-haiku-5-5", env: { CLAUDE_CODE_EFFORT_LEVEL: "max" } }); + assert.equal(max.effort, "max"); }); test("the effort footer names the same per-model defaults the log records", () => { @@ -1020,7 +1031,7 @@ test("the effort footer names the same per-model defaults the log records", () = const out = run(["report"], cfg); // The footer explains where a recorded level came from; a model missing // from it contradicts the level printed above it. - assert.match(out, /the model default \(medium on Opus 5\.5 and Sonnet 5\.5, xhigh on Opus 4\.7, high elsewhere\)/); + assert.match(out, /the model default \(medium on Opus 5\.5, Sonnet 5\.5 and Haiku 5\.5, xhigh on Opus 4\.7, high elsewhere\)/); // The sentence above it promises what a pin buys; an unconditional claim // there is wrong for a haiku agent and for a session already at the pin. assert.match(out, /can buy a cheaper model tier and, where that model supports effort and the pinned level is lower than the one it would otherwise inherit, a lower effort too/); @@ -1279,6 +1290,43 @@ test("opus-5.5 prices at $4/$20 with cache reads at 0.05x, not the opus-5 row", } finally { rmSync(cfg, { recursive: true, force: true }); } }); +// Haiku 5.5 bills each request on the $0.10/$0.50 card up to 100K prompt tokens +// and on $0.50/$2.50 above. 100 identical lines with no message.id are 100 +// requests, which lifts per-request cents to a figure the report prints exactly. +const haiku55Requests = (n, usage) => Array(n).fill(costLine("claude-haiku-5-5", usage)).join("\n") + "\n"; +const tokensForHaiku55 = (body) => { + const cfg = freshConfigDir(); + const dir = join(cfg, "projects", "proj", "sess-1", "subagents"); + mkdirSync(dir, { recursive: true }); + writeFileSync(join(cfg, "projects", "proj", "sess-1.jsonl"), usageLine("claude-opus-5", 1) + "\n"); + writeFileSync(join(dir, "agent-a.jsonl"), body); + try { return run(["tokens"], cfg); } finally { rmSync(cfg, { recursive: true, force: true }); } +}; + +test("haiku-5.5 prices each request of a loop on its own card by prompt length", () => { + // 100 x 60K in + 20K out on the base card: $0.006 + $0.010 = $1.60 + // 100 x 150K in + 20K out on the long card: $0.075 + $0.050 = $12.50 + const out = tokensForHaiku55( + haiku55Requests(100, { input: 60000, out: 20000 }) + haiku55Requests(100, { input: 150000, out: 20000 })); + assert.match(out, /as it ran\s+\$14\.10/); + // Opus 5 has no long card: 21M in x $5 + 4M out x $25 = $205.00, the totals + // untouched by the split. + assert.match(out, /had every subagent inherited its session model\s+\$205\.00/); +}); + +test("haiku-5.5 at exactly 100K prompt tokens stays on the base card", () => { + // 100 x 100K in at $0.10/MTok = $1.00 (the long card would be $5.00). + assert.match(tokensForHaiku55(haiku55Requests(100, { input: 100000 })), /as it ran\s+\$1\.00/); +}); + +test("haiku-5.5 cache reads and writes count toward the 100K threshold", () => { + // 10K input alone is under the threshold; with 100K cache reads and 8K 5m + // writes the prompt is 118K, so all three bill on the long card: + // 10K x $0.50 + 100K x $0.05 + 8K x $0.625 = $0.015 per request, $1.50 for 100. + // On the base card it would be $0.30. + assert.match(tokensForHaiku55(haiku55Requests(100, { input: 10000, cacheRead: 100000, cw5: 8000 })), /as it ran\s+\$1\.50/); +}); + test("a cache-write bucket this code has never heard of is still charged", () => { const cfg = freshConfigDir(); const dir = join(cfg, "projects", "proj", "sess-1", "subagents"); @@ -1645,33 +1693,32 @@ test("a routed-down agent priced above its session model is named with both amou } finally { rmSync(cfg, { recursive: true, force: true }); } }); -test("sonnet-5.5 prices exactly as sonnet-5, cache reads included", () => { +test("sonnet-5.5 prices at $2/$10 with cache reads at 0.05x, not the sonnet-5 row", () => { const cfg = freshConfigDir(); const dir = join(cfg, "projects", "proj", "sess-1", "subagents"); mkdirSync(dir, { recursive: true }); writeFileSync(join(cfg, "projects", "proj", "sess-1.jsonl"), usageLine("claude-fable-5-1", 1) + "\n"); - // 1M input $2 + 1M cache read $0.20 + 1M 5m write $2.50 + 1M 1h write $4 + 1M output $10 = $18.70 + // 1M input $2 + 1M cache read $0.10 + 1M 5m write $2.50 + 1M 1h write $4 + 1M output $10 = $18.60 writeFileSync(join(dir, "agent-a.jsonl"), costLine("claude-sonnet-5-5", { input: 1e6, cacheRead: 1e6, cw5: 1e6, cw1h: 1e6, out: 1e6 }) + "\n"); try { const out = run(["tokens"], cfg); - assert.match(out, /as it ran\s+\$18\.70/); + assert.match(out, /as it ran\s+\$18\.60/); assert.match(out, /sonnet-5-5/); } finally { rmSync(cfg, { recursive: true, force: true }); } }); -test("sonnet-5.5 re-reading under an opus-5.5 session costs the same and is not named", () => { +test("sonnet-5.5 re-reading under an opus-5.5 session costs half and is not named", () => { const cfg = freshConfigDir(); const dir = join(cfg, "projects", "proj", "sess-1", "subagents"); mkdirSync(dir, { recursive: true }); writeFileSync(join(cfg, "projects", "proj", "sess-1.jsonl"), '{"model":"claude-opus-5-5"}\n'); - // Both read cache at $0.20/MTok, so as-ran and inherited are equal: routed - // down on tier, level on price, and never a "priced higher" row. + // Both read cache at 0.05x of their input: $0.10/MTok against $0.20/MTok. writeFileSync(join(dir, "agent-a.jsonl"), costLine("claude-sonnet-5-5", { cacheRead: 10e6 }) + "\n"); metaFor(dir, "a", "model-routing:implementer", "sonnet"); try { const out = run(["tokens"], cfg); - assert.match(out, /as it ran\s+\$2\.00/); + assert.match(out, /as it ran\s+\$1\.00/); assert.match(out, /had every subagent inherited its session model\s+\$2\.00/); assert.match(out, /opus-5-5: 10\.0M across 1 agents - 100% below session tier/); assert.doesNotMatch(out, /Routed down but priced higher/); diff --git a/hooks/routing-anchor.md b/hooks/routing-anchor.md index 26d865b..666cd14 100644 --- a/hooks/routing-anchor.md +++ b/hooks/routing-anchor.md @@ -2,9 +2,9 @@ MODEL ROUTING ACTIVE - expensive model thinks, cheap models grind. You cannot sw - Main session: planning, specs, docs, architecture, small interactive edits. Trivial questions answerable from context or general knowledge: answer inline - a dispatch costs more than the answer. - Exploration, by what has to come back: a judgement about what code does -> `scout`; a complete list or ordering verified by following the code -> `surveyor` (haiku); unverified candidate files -> built-in Explore with `model=haiku` (bare, it inherits the session model). With a code-graph MCP connected, query the graph first; scout verifies file:line. -- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result; when in doubt, opus. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. -- Review -> `reviewer`; high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. -- Effort (low/medium/high/xhigh/max; Claude Code default medium on Opus 5.5 and Sonnet 5.5, xhigh on Opus 4.7, high elsewhere): low mechanical, medium implementation, high architecture/debugging/review, xhigh/max only long-horizon or frontier work. On xhigh/max review only what the plan, the user or these rules ask - no self-started second round, no extra reviewer subagents. A dispatch carries no effort param: bundled agents pin theirs, Workflow `agent()` takes an `effort` opt. +- Approved plan/spec -> `implementer` (sonnet). Pass `model=opus` (Fable sessions too; the pinned medium effort stays) for multi-file/cross-layer work, security/money/migrations/concurrency/public contracts, or a retry after a weak sonnet result. Ambiguous task or unclear root cause: clarify first (main session or scout) - implementer stops on ambiguity. No approved plan: have it return its PLAN and check it here before it writes code. Batch related tasks per agent. +- Review -> `reviewer`; plan-loop per-task reviews and scoped re-reviews (Superpowers SDD) -> `task-reviewer` (sonnet); high-risk or large diffs get final review in the main session. Batched implementer output you will not read in full -> `verifier`, never on the main session's own work. Tests/builds -> `test-runner`; E2E and failure interpretation -> `e2e-runner`. +- Effort (low/medium/high/xhigh/max; Claude Code default medium on Opus 5.5, Sonnet 5.5 and Haiku 5.5, xhigh on Opus 4.7, high elsewhere): low mechanical, medium implementation, high architecture/debugging/review, xhigh/max only long-horizon or frontier work. On xhigh/max review only what the plan, the user or these rules ask - no self-started second round, no extra reviewer subagents. A dispatch carries no effort param: bundled agents pin theirs, Workflow `agent()` takes an `effort` opt. - Stuck on the approach: the agent hands back; decide, then continue the same agent (SendMessage, else re-dispatch with its packaged state). Unfinished return with no blocker named: continue it the same way at the same tier. Failed or weak RESULT: fresh dispatch one step up (next tier via `model`, or higher Workflow `effort` when the miss is shallow thinking); a second failure comes to the main session. `stop_reason: "refusal"` is a classifier, not weakness: rephrase or drop, never re-dispatch a tier up. - Pins: a pin above the session model gets capped at the session model via `model` - a bare dispatch runs the pin. Never push a dispatch below the lower of pin and session model; when a cheaper tier fits, pick an agent pinned for it. CLAUDE_CODE_SUBAGENT_MODEL_FORCE overrides `model` and pins; plain CLAUDE_CODE_SUBAGENT_MODEL only decides bare unpinned dispatches. - Unpinned agents (general-purpose, custom types, Workflow `agent()` calls) inherit the session model and effort: set `model` (sonnet; haiku for trivial sweeps), and on Workflow stages `effort` (cheap stages low), unless the task needs the session's reasoning. diff --git a/skills/model-routing/SKILL.md b/skills/model-routing/SKILL.md index 25f356d..5352750 100644 --- a/skills/model-routing/SKILL.md +++ b/skills/model-routing/SKILL.md @@ -12,8 +12,8 @@ plugin). **Routing makes a dispatch cheaper; it does not make dispatching cheap.** A subagent starts empty, so everything it reads is a cache write, while -the main session pays cache read - 12.5x less, 25x on Opus 5.5, 50x on -Fable 5.1 and Mythos 5.1 - for context it already holds. That penalty is paid whether or not the tier is routed down, so the +the main session pays cache read - 12.5x less, 25x on Opus 5.5 and +Sonnet 5.5, 50x on Fable 5.1 and Mythos 5.1 - for context it already holds. That penalty is paid whether or not the tier is routed down, so the rules below only pay off on work that was going to be delegated anyway. On one wide-reading session, three runs: doing it inline cost $1.36 and delegating with the tier routed down $1.68, both measured; delegating at @@ -42,8 +42,8 @@ which model, and how hard it thinks. The full ladder is `low / medium / high / xhigh / max`. What runs when nothing sets a level is Claude Code's per-model default: `high` on every -model that supports effort, except `medium` on Opus 5.5 and Sonnet 5.5 -and `xhigh` on Opus 4.7. The API's own default can differ (it runs both +model that supports effort, except `medium` on Opus 5.5, Sonnet 5.5 and +Haiku 5.5 and `xhigh` on Opus 4.7. The API's own default can differ (it runs both Opus 4.7 and Sonnet 5.5 at `high`), but a session's effort is what Claude Code sends. Levels are saved per model under `modelSettings`, and a top-level `effortLevel` in the user settings file no longer reaches Opus @@ -87,9 +87,11 @@ example use case for `low` is subagents. default. So `implementer` at `medium` still steps down from a session running `high` or above - a Fable 5.1 session at its default, say - and buys nothing on a session already at `medium`, which is now the Claude - Code default for both Opus 5.5 and Sonnet 5.5 sessions. The three haiku - agents are a separate case: Haiku 4.5 has no effort knob at all, so - their `low` pins document intent and cost nothing either way. + Code default for Opus 5.5, Sonnet 5.5 and Haiku 5.5 sessions. The three + haiku agents' pins are live on the Anthropic API, where `haiku` is + Haiku 5.5 with the full ladder. On Claude Platform on AWS, Bedrock, + Google Cloud and Foundry `haiku` is still Haiku 4.5, which has no effort + knob at all, so there the pins document intent and cost nothing. - **high** - genuinely hard reasoning: architecture, subtle debugging, high-risk final review, anything where a wrong approach is expensive to unwind. @@ -143,15 +145,16 @@ carries no effort param: |------|-------|---------------|--------| | Planning, brainstorming, specs, docs, architecture | main session | strongest (user's /model choice) | high | | Codebase exploration ("where is X", "how does Y work") | subagent | `scout` (sonnet) | low | -| Breadth sweeps: enumerate, list, trace a chain end to end | subagent | `surveyor` (haiku) | low | +| Breadth sweeps: enumerate, list, trace a chain end to end | subagent | `surveyor` (haiku) | medium | | Implementing an approved plan/spec (ordinary: single-file, clear shape) | subagent | `implementer` (sonnet) | medium | | Complex implementation: multi-file refactor, subtle concurrency/security | subagent | `implementer` with `model=opus` | medium (pinned) | | Trivial mechanical tasks: renames, boilerplate, mirrored constants | subagent | sonnet | low | | Small interactive edits, quick fixes | main session | strongest | low | -| Code review of implemented work | subagent | `reviewer` (opus) | high | +| Per-task review and scoped re-review inside a plan loop (Superpowers SDD) | subagent | `task-reviewer` (sonnet) | high | +| Final/whole-branch, high-risk or pre-PR review of implemented work | subagent | `reviewer` (opus) | high | | Final review of high-risk or large diffs | main session | strongest | high | | Run tests/builds/linters, report failures | subagent | `test-runner` (haiku) | low | -| Sanity-check a subagent's diff against its task | subagent | `verifier` (haiku) | low | +| Sanity-check a subagent's diff against its task | subagent | `verifier` (haiku) | medium | | Playwright/E2E scenarios, failure interpretation | subagent | `e2e-runner` (sonnet) | medium | | Fresh external context, knowledge-cutoff gap (new APIs, recent releases) | subagent | mid-tier agent with web access | medium | @@ -189,29 +192,12 @@ actually earns its cost: Opus 5 was a step-change over Opus 4.8 at unchanged price, and Opus 5.5 is stronger again at 20% less ($4/$20, cache reads 60% less), so the opus tier buys strictly more per dollar than when - this table was tuned - when in doubt between sonnet and opus for - implementation, take opus. That holds on every session model, Fable + this table was tuned. That holds on every session model, Fable included - see the fable-to-opus bullet below. -- **Against Opus 5.5 the tier gap closed on cache reads and stayed open - everywhere else.** Opus 5.5 reads cache at $0.20/MTok and so does - Sonnet 5.5 (0.05x of $4 against the standard 0.1x of $2), so the tokens - a subagent re-reads turn by turn cost the same on either model - a - parity Opus 5.5 already had with Sonnet 5, not something Sonnet 5.5 - introduced. What routing down still buys is half price on everything - else: $2 against $4 for the first read of a file, $2.50 against $5 to - cache it, $10 against $20 for output. Count the dollars, not the - tokens: cache reads are most of the VOLUME in an agentic loop while - writes and output usually carry most of the BILL, and the saving falls - as that stops being true. For identical token usage it is 50% with no - cache reads at all; the one profile measured here, the scout eval (74k - written, 451k re-read, 6k out), costs $0.3352 on sonnet against $0.5802 - on opus, 42.2% off; at that same write-and-output mix it is about 30% - once cache reads reach 20 times the written-plus-output tokens; and on - a profile of 1k written, 1k out and 1M re-read it is about 5.6%, where - there is almost nothing left for the halved rates to halve. The share of the - bill decides, not an absolute token count. The parity is also specific to that pair: a - Fable 5.1 session reads at $0.25 and an Opus 5 one at $0.50, so against - those the cheaper tier still wins on every token type. +- **Opus 5.5 to Sonnet 5.5 is a flat 50% off.** Both read cache at 0.05x + of their own input, so every token type halves: the scout eval profile + (74k written, 451k re-read, 6k out) costs $0.2901 on sonnet against + $0.5802 on opus, whatever the mix. - **Review -> opus/high.** Review is one cheap pass guarding against expensive misses - an asymmetric bet where the strongest reasoning at high effort is worth it, because a bug that ships costs far more than @@ -220,11 +206,18 @@ actually earns its cost: Opus 5 at `high` on coding, and at `high` it thinks more per turn than Opus 5 did, so the same pin now costs more. That is vendor evidence on model-level evals, not this plugin's review eval - the pin stays at - high until a medium-vs-high review run on Opus 5.5 is measured. -- **Tests / verification -> haiku/low.** Running a command and - summarizing output, or checking a diff matches its task, is mechanical. - The cheapest tier at low effort suffices; the value is keeping raw - output out of the main context, not the model doing it. + high until a medium-vs-high review run on Opus 5.5 is measured. The + per-task reviews and scoped re-reviews of a plan loop are the + exception: Superpowers SDD scales their model to the diff with a + mid-tier floor, so they go to `task-reviewer` (sonnet/high) rather than + to `reviewer` with `model=sonnet`, which undercuts its pin. Final, whole-branch, high-risk and + pre-PR review stays on `reviewer`. +- **Tests / verification -> haiku.** Running a command and summarizing + output is mechanical: `test-runner` stays at low effort, and the value + is keeping raw output out of the main context, not the model doing it. + `verifier` and `surveyor` pin `medium`, because the Haiku 5.5 docs say + at `low` it "is more likely to skip a search, stop early, or skip a + check" - the failure these two agents exist to prevent. - **Effort tracks task shape, not tier** (levels: the ladder above). A strong model at low effort beats a weak model at high effort for a fraction of the cost. On the Opus 5 generation this is amplified: @@ -256,14 +249,16 @@ actually earns its cost: need. It does not replace the tier decision - dispatching that phase to sonnet is cheaper still. Sweep Workflow `effort` opts the same way. - **A refusal is a redirect, not a weak result.** Fable 5.1, Fable 5, - Opus 5.5, Opus 5 and Sonnet 5.5 ship safety classifiers that can + Opus 5.5, Opus 5, Sonnet 5.5 and Haiku 5.5 ship safety classifiers that can decline a request outright rather than answer it badly - Sonnet 5.5 included, which makes it the most common dispatch this can happen to, while Haiku 4.5 and Sonnet 5 carry no documented categories. A declined dispatch is the one failure the escalation ladder below does not fix, and a step up no longer reliably clears it: Opus 5.5 runs cybersecurity and biology classifiers like Fable 5.1's, and both it and Sonnet 5.5 add a `reasoning_extraction` - category whose declines a fallback model does not retry at all. Which classifier + category whose declines a fallback model does not retry at all. Haiku + 5.5 runs `cyber`, `bio`, `frontier_llm` and `general_harms` classifiers + with no server-side fallback. Which classifier fired is API-level and invisible from inside a dispatch, so do not re-dispatch a declined request on opus by reflex - hand it back to the main session to rephrase or drop. @@ -279,7 +274,8 @@ Model names in agent pins are FAMILY aliases (opus, sonnet, haiku), not versions - on the Anthropic API the harness resolves them to the current model of each family, while the cloud platforms lag (`sonnet` is Sonnet 4.6 on Claude Platform on AWS and Sonnet 4.5 on Bedrock, Google Cloud and -Foundry, where the parity figures above do not hold at all) - +Foundry, `haiku` is Haiku 4.5 on all four, and `opus` is Opus 4.6 on +Foundry, where the 5.5 price figures above do not hold at all) - so a generation jump (Opus 5 -> Opus 5.5, then Sonnet 5 -> Sonnet 5.5 on 2026-09-28) upgrades `reviewer`, every sonnet-pinned agent and every `model=opus` escalation automatically, with no plugin change. Prices, @@ -320,8 +316,8 @@ with `/model-routing:stats`. input - is not breadth, and the cheap tier fails it in a way that looks confident: that is `scout` (sonnet). Both halves are measured in `evals/`, on one generated fixture at three runs each: on a twelve-stage tracing - question haiku answered correctly every run at a third of sonnet's price; - on a question whose code contains an obvious wrong answer, haiku took the + question Haiku 4.5 answered correctly every run at a third of sonnet's + price; on a question whose code contains an obvious wrong answer, it took the bait once in three runs while sonnet took it in none. Three runs on one fixture cannot pin a failure rate, and the reason to keep judgement on sonnet is not the rate but the asymmetry: a cheap right answer saves @@ -423,7 +419,8 @@ with `/model-routing:stats`. dispatch whose session model cannot be read is left unjudged rather than guessed at. When the cheap tier genuinely fits the work, pick an agent pinned for it - (`test-runner`, `verifier`) instead of overriding a role agent downward; the + (`test-runner`, `verifier`, `task-reviewer` for a sonnet-tier review) + instead of overriding a role agent downward; the dispatch report lists below-pin dispatches in their own section. - Against the session model, a pin is a ceiling. A pin says "this task never needs more than X"; the session model says what the user is willing to pay. @@ -454,7 +451,8 @@ with `/model-routing:stats`. rather than mid-task: a thinking block is bound to the model that wrote it, and the families no longer read each other's - Sonnet 5.5 reads Sonnet 5, Opus 4.8 and Haiku 4.5 blocks but not Opus 5, Opus 5.5 - or any Fable one, and no other model reads Sonnet 5.5's. The API drops what + or any Fable one, and no other model reads Sonnet 5.5's; Opus 5.5 and + Sonnet 5.5 both read Haiku 5.5's. The API drops what the new model cannot read, unbilled and without an error, so the turns after a switch run without the reasoning that led to them. A dispatch does not have this problem: a subagent starts empty either way. @@ -472,8 +470,10 @@ what changes cost there. working the same task in conversation. - **Per-stage routing.** Every `agent()` call without `model`/`effort` opts inherits the session model at session effort, multiplied by the - fan-out. Mechanical finder stages get an explicit cheap model and - `effort: low`; a tier up only where the stage earns it. Precedence: + fan-out. Mechanical stages get an explicit cheap model and + `effort: low`; a haiku search stage that has to be complete gets + `medium`, for the same reason `surveyor` does; a tier up only where the + stage earns it. Precedence: the script opt wins, then a frontmatter pin, then the `CLAUDE_CODE_SUBAGENT_MODEL` env var, then the session model - unless `CLAUDE_CODE_SUBAGENT_MODEL_FORCE` is set, which ignores the opt and