feat(ci): an engine is wired into all of its lists, or none - #114
Conversation
Making a BYOA engine runnable means adding its id to about ten hand-kept lists across the server, the renderer and the tooling. Miss one and nothing shouts. The worst is BYOA_SOURCES: it is read through normalizeByoaSource(), which maps anything unrecognised to 'byoa-claude', so an engine missing from that list does not error — it quietly bills every one of its runs to Claude in the ledger. That failure mode is not hypothetical. yetone#102 was two identical blocks where one got the fix and the other did not, and cli-version.ts spent a while telling the next person to keep the catalog in sync with a list that had been deleted. Both are the same shape: a hand-maintained duplicate with no mechanical check. The guard anchors on ENGINE_IDS and, for each engine, asserts an ADAPTERS entry, all three EngineId unions, a label, a PATH binary, RUNNABLE_ENGINE_IDS, a version spec, BYOA_SOURCES, the four ledger source unions, and that the binary appears in the big-brain guard's direct-spawn rule (keyed on the BINARY, since cursor's is `cursor-agent`). Every anchor is required. A guard that quietly stops checking is worse than no guard, so a declaration that moves or is renamed reports "anchor not found" rather than passing vacuously — and a test pins that ENGINE_IDS really parsed, so scanRepo() cannot succeed by finding nothing to check. Verified by breaking it both ways: dropping 'byoa-gemini' from BYOA_SOURCES and dropping gemini from RUNNABLE_ENGINE_IDS each fail with the specific list and the cost of leaving it out.
The guard anchored on `RUNNABLE_ENGINE_IDS = new Set([...])`. yetone#116 collapses that duplicate by deriving the set from an ordered `RUNNABLE_ENGINES` tuple, and the guard then reported "anchor not found" — correctly, by its own design, but it would have made this the thing blocking a change that removes the very duplication it exists to police. It now reads whichever shape is present. Verified against all three open branches: main's Set literal, yetone#116's tuple, and yetone#113's added engine.
| return problems | ||
| } | ||
|
|
||
| if (import.meta.url === `file://${process.argv[1]}`) { |
There was a problem hiding this comment.
This comparison never matches on Windows: process.argv[1] is a filesystem path such as E:\..., while import.meta.url is a normalized file:///E:/... URL. As a result, npm run guard:engine-registry exits successfully without ever calling scanRepo(), so the local guard silently becomes a no-op.
Comparing filesystem paths keeps the CLI entry point portable:
| if (import.meta.url === `file://${process.argv[1]}`) { | |
| if (fileURLToPath(import.meta.url) === process.argv[1]) { |
It may also be worth adding a small test that launches the script as a CLI, since the current tests call scanRepo() directly and cannot catch a broken entry-point check.
There was a problem hiding this comment.
Right on both counts, thank you — applied in the latest commit.
What makes this one sting is that it only looks fine on POSIX by accident: argv[1] starts with /, so file:// + it lands on the same file:///… string. On Windows it cannot:
POSIX 'file:///Users/x/g.mjs' === 'file://' + '/Users/x/g.mjs' -> true
WIN32 'file:///E:/x/g.mjs' === 'file://' + 'E:\x\g.mjs' -> false
(built: file://E:\x\g.mjs)
So the guard exits 0 having checked nothing — which is precisely the failure mode the file argues against in its own header. And guard-big-brain.mjs already had the correct form:
const isMain = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]I've matched that exactly rather than inventing a third spelling. That is the third time in this batch that the right version already existed and a new copy diverged from it (#102, #116, now this), which is a fair thing to keep pointing out about my own patches.
Your test suggestion was the more valuable half, so I took it: two cases that launch the script as a CLI rather than calling scanRepo() — one asserts it prints a verdict, one sabotages an anchor in a scratch copy and asserts a non-zero exit. Verified they have teeth by replacing if (isMain) with if (false), which fails both (5/7 pass). I could not reproduce the Windows path semantics on macOS, so what these pin is the general property — an entry point that never fires — rather than that specific platform difference.
…ry point Review catch from @bingqilinweimaotai, and it was right. The entry check was if (import.meta.url === `file://${process.argv[1]}`) On POSIX that happens to match, because argv[1] starts with `/` and the concatenation lands on the same `file:///…` string. On Windows argv[1] is `E:\repo\scripts\guard-engine-registry.mjs`, so the built URL is `file://E:\repo\…` while import.meta.url is `file:///E:/repo/…`. They never match, the block never runs, and `npm run guard:engine-registry` exits 0 having checked nothing — the guard silently becoming a no-op is the exact failure this file was written to prevent. guard-big-brain.mjs already had the correct form: const isMain = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1] so this is once more a case of the right version existing and a new copy diverging from it. Matched to it exactly rather than inventing a third spelling. Two tests, also as suggested on the review: every earlier test calls scanRepo() directly and so cannot tell a working guard from one whose main check never fires. These launch the script as a CLI — one asserts it prints its verdict, one sabotages an anchor in a scratch copy and asserts a non-zero exit. Replacing `if (isMain)` with `if (false)` fails both.
Making a BYOA engine runnable means adding its id to about ten hand-kept lists across the server, the renderer and the tooling. Miss one and nothing shouts.
The worst one is
BYOA_SOURCES. It is read throughnormalizeByoaSource():An engine missing from that list does not error. It quietly bills every one of its runs to Claude in the ledger.
Why now
Two things this week were the same shape — a hand-maintained duplicate with no mechanical check:
SAVEPOINTand the other never did. Every sign-up whose email local part was already taken failed, for months, because the fix landed on one copy.cli-version.tscarried/** Keep in sync with electron/main.cjs LOCAL_CLIS … */afterLOCAL_CLIShad been deleted — pointing the next person at a list that no longer existed.Adding Gemini (#99) and Qwen (#113) meant walking that list by hand twice. I nearly left
byoa-qwenout, and nothing would have told me.What it checks
Anchored on
ENGINE_IDS— the engines the daemon actually probes and wakes — and for each id:ADAPTERSentryengine.tsEngineIdunionsengine.ts,computer/registry.ts,src/types.tssrc/lib/engines.tsRUNNABLE_ENGINE_IDSsrc/lib/engines.tscomputer/cli-version.tsbyoa-<id>in the shared source listruntime/byoa-source.tsbyoa-<id>in all four ledger unionsllm-ledger,observability,admin/api,api/clientguard-big-brain.mjsR4The last one is keyed on the binary, not the id — cursor's binary is
cursor-agent, so checking ids there would pass while the real binary went unguarded.Each failure names the list and what it costs to leave it out, rather than just "missing":
It fails loudly rather than vacuously
A guard that quietly stops checking is worse than no guard, so every anchor is required: if a refactor moves or renames a declaration,
extract()returns null and the guard reportsanchor not found — update this guard alongside the refactor. A test also pins thatENGINE_IDSreally parsed and found the engines that exist, soscanRepo()cannot pass by having nothing to check.Verified by breaking it
Dropping
'byoa-gemini'fromBYOA_SOURCES:Dropping
geminifromRUNNABLE_ENGINE_IDS:Both restore to green. Run against the #113 branch as well, which passes — an independent, mechanical confirmation that the Qwen wiring is complete rather than my having remembered every list.
Shape
Same as the two guards already in CI:
scripts/guard-engine-registry.mjs, annpm run guard:engine-registryscript, a.d.mtsbeside it (matchingguard-big-brain.d.mts), aguard-engine-registry.test.tssonpm testcatches it too, and its own fast-fail CI job.Runs in milliseconds — it reads eleven files and does string containment, no parsing of the TypeScript.