analysis: what external review bots caught that the fleet did not (ADR-009 / #971 thread) #1009

Why

The owner pays for external reviewers (Codex, Copilot, CodeRabbit), and their credit runs out. If Hypatia and its fleet did their job, those reviewers would be nearly redundant: the fleet means gitbot-fleet, cicd-squabbler, .git-private-farm, proven-cicd and proven-tests-and-benches. They are not redundant yet. This issue takes one thread of work and records three things for each defect: what found it, when, and which part of the fleet should have found it first. The goal is to turn each gap into a rule, a harness or a gate, not a review comment.

The thread covered these PRs:

• metadatastician/standards #1352, #1354, #1359 and #1401;
• hyperpolymath/hypatia RE001: accept a deed egress declaration; stop counting secrets.GITHUB_TOKEN (standards ADR-009) #964, RE001: read workflows as YAML does (quoted keys, aliases, with: inputs) #975 and Hypatia.Kyaml: a port of kyq's KYAML reader, held to kyq in CI (#971, part 1) #1008;
• hyperpolymath/rsr-template-repo fix(ci): adopt canonical hypatia-scan.yml #259;
• hyperpolymath/choreographic-types chore(deps): bump actions/cache from 5.0.2 to 5.0.3 #37 and chore(deps): bump erlef/setup-beam from 1.17.5 to 1.20.4 #39.

Ledger

A defect found while writing this

The description of code_safety/elixir_atom_from_user (lib/rules/code_safety.ex:494) is backwards. It reads "String.to_existing_atom with user input exhausts atom table -- use to_existing_atom", but the pattern matches String.to_atom(. That message is what every alert shows, including code-scanning alert 1512 on #1008. It will be fixed in the next Hypatia PR from this thread.

Proposed conversions, rated

Ranked most elegant and correct first, then most dependable.

1. Local CI parity: one just ci-changed recipe. It runs exactly the workflow jobs a diff touches (gap 1), plus the Hypatia scan on the diff (gap 3), before push. The fleet already owns the rules, so this only moves when they run. Elegance: high. Dependability: high. Cost: a recipe and job-selection logic.
2. Grammar-driven differentials as a standard harness shape. Bump peter-evans/repository-dispatch from 3.0.0 to 4.0.1 #5 and Bump actions/checkout from 4.1.1 to 6.0.1 #6 were found by diffing an implementation against an independent reference, and Claude/review scm security t3e cl #2 escaped because both sides shared the bug. The harness should therefore also mutate toward spec classes (YAML 1.2 §5 character productions), not only toward the reference's behaviour. This is the natural remit of proven-tests-and-benches. Elegance: highest; it generalises to every format we parse (deed, k9, KYAML, I-JSON). Dependability: high, and deterministic by seed.
3. A rule against parsing a structured format with regexes or line scans where an estate parser exists (Hypatia.Kyaml, Hypatia.Deed). This is the class behind Bump webfactory/ssh-agent from 0.9.0 to 0.9.1 #4 and the whole of design: one parsed workflow model for every workflow rule #971. Elegance: medium, since it is heuristic. Dependability: medium.
4. Treat a paid reviewer's quota failure as "no review", visibly. Surface it in the fork-governance verdict grid, not as an approving silence. Dependability: high. This one is outside the repo: standards fork-governance.

Beyond this repo

• metadatastician/kyq: the patch for Claude/review scm security t3e cl #2 and Bump actions/checkout from 4.1.1 to 6.0.1 #6 exists locally (5 commits on fix/bom-and-block-scalar-comment; 328 pass). Pushing it was not possible from the session that found it. It needs to land upstream, and then KYQ_REF in ci.yml moves and KNOWN_DIVERGENCES.adoc shrinks.
• gitbot-fleet / proven-tests-and-benches / cicd-squabbler: I could not read these repositories from this session. The mapping above is by their stated remit and should be checked against what they actually run today.