Skip to content

fix: 5.5 prompt pass for the remaining agents and a routing-anchor diet - #56

Merged
AqueGen merged 4 commits into
mainfrom
fix/prompt-audit-5-5
Oct 2, 2026
Merged

AqueGen merged 4 commits into
mainfrom
fix/prompt-audit-5-5

Conversation

@AqueGen

@AqueGen AqueGen commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #54: the rest of the agent and skill prompts, checked against the Sonnet 5.5 and Opus 5.5 prompting guides, and a cut to the routing anchor. Two independent reviews (Fable, Sonnet) went over it in two rounds against the primary sources; every change below survived both, and the third reviewer (Codex) could not run because of a local sandbox error.

Prompt fixes

scout, reviewer, e2e-runner: finish before reporting, don't end the turn with a progress update or a question about whether to continue, and stop early only when nothing can move without the caller (e2e-runner, which may make test-only fixes, also before a risky step). #54 did the same for implementer. Sources: the Sonnet 5.5 guide, "At low and medium, on long agentic tasks, it's more likely to stop and check in with the user before it finishes" (scout runs sonnet at low); the Opus 5.5 guide, "some of those updates end the turn with text rather than a tool call" (reviewer runs opus), and "It also helps to name the stops you do want, for example when no work can advance without the user's input". The haiku agents (surveyor, test-runner, verifier) get no finish rule.

scout, surveyor: the "your value is that megabytes of source stay in your context" framing shrinks to one line, and surveyor's hand-back-to-scout paragraph keeps its rule and loses its explanation.

implementer: dropped "use that reasoning fully" for opus dispatches. It cannot raise the effort: medium pin, and the Opus 5.5 guide says "Lowering effort reduces thinking, and with it cost and latency, more reliably than prompt instructions do". (The best-practices advice to dial back anti-laziness prompts names Claude 4.6 models, so it is not the reason here.) Also "do NOT burn tokens" is now "don't burn tokens".

SKILL.md: the fallbackModel note said a bare opus falls back to a 200K window. model-config: Opus 4.7 and later, Sonnet 5 and later and Fable "run with the 1M window on every plan ... You don't select a [1m] variant". The note now says where a 200K entry still matters: Claude Code skips a fallback with a smaller window than the primary's during compaction.

SKILL.md: drops the Opus 4.7/4.8/Opus 5 effort-recommendation history and the retired Fable-implementer pricing story ($275.55 / $221.52 / ~$133), keeping the current guidance and the /model-routing:stats pointer.

SKILL.md, README.md: "beats" / "above" became "matches or exceeds" in all four places that compare Opus 5.5 at medium with Opus 5 at high, quoting the Opus 5.5 guide.

Checked and found clean: no prompt asks the model to write out its reasoning in the response (the reasoning_extraction refusal on both 5.5 models); the remaining capitals mark labels and contrasts.

Anchor cut

routing-anchor.md goes from 5114 to 3225 bytes. It had grown back past the 1509 bytes of the 0.4.2 cut. Measured on the same eval case and CLI, the new anchor writes about 630 fewer cache tokens per run (old 9866/9882, new 9254/9228, mean delta 633) and reads about 1.2k fewer (old 98633/98647, new 97414/97381; the anchor is written once and re-read on the two later calls of each run) - roughly 630 tokens off every session, and about $0.005 of opus cost per run on this case.

What left is rationale the skill and README still carry: the Fable vs Opus 5.5 cache-read prices, the 42% scout profile, why the main session self-verifies, the Explore cap at opus. Every actionable rule of the old anchor is in the new one; the reviews checked this rule by rule, and a first draft that had blurred six of them (the Opus 4.7 xhigh default, the cap target, effort as the alternative retry step, the clarify routes, the pinned medium on opus implementers, cheap Workflow stages) was corrected before this commit.

README's carry cost moves from "about 1.6k" to "about 1.8k": the old figure took the anchor at ~900 tokens, an estimate with no measurement behind it, while the anchor on main had grown to 5114 bytes. The new figure scales the anchor's 3225 bytes by the ~3 bytes per token its own cut measured, plus ~700 for the listings from claude plugin details.

Eval result, and what it no longer shows

delegates-codebase-question, Opus 5.5, 2 runs per anchor, both aggregates committed to evals/evidence/:

anchor answer correct delegated opus cost per run
old (main, 5114 bytes) 2/2 0/2 $0.1093, $0.1098
new (this PR, 3225 bytes) 2/2 0/2 $0.1044, $0.1042

The question needs a codebase search, so the anchor routes it to scout. On Opus 5.5 the session answers it with one grep and one read under both anchors, unlike the 3/3 delegation measured on 2026-08-17. The case therefore no longer separates the arms on Opus 5.5; it shows the cut changes nothing measurable in behaviour here and measures what it saves in tokens. README now says so next to the 3/3 figure.

Test plan

  • node --test hooks/dispatch-counter.test.mjs hooks/update-check.test.mjs: 123/123 pass.
  • Eval above, both arms run on 2026-10-02 with Claude Code 2.1.287; run totals including ~$0.001 of haiku per run are in the evidence files.
  • Agent definitions load on session restart; the agent-side changes get checked on the next real dispatches.

Sources:

@AqueGen AqueGen changed the title fix: finish-before-report for scout and reviewer, drop the opus anti-laziness line, correct the 1M fallback note fix: 5.5 prompt pass for the remaining agents and a routing-anchor diet Oct 2, 2026
…quote the 5.5 guides exactly, and commit the eval that measures the cut
@AqueGen
AqueGen merged commit c491224 into main Oct 2, 2026
2 checks passed
@AqueGen
AqueGen deleted the fix/prompt-audit-5-5 branch October 2, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant