Skip to content

feat(ci): an engine is wired into all of its lists, or none - #114

Merged
WhichPaths merged 3 commits into
yetone:mainfrom
WhichPaths:feat/engine-registry-guard
Aug 30, 2026
Merged

feat(ci): an engine is wired into all of its lists, or none#114
WhichPaths merged 3 commits into
yetone:mainfrom
WhichPaths:feat/engine-registry-guard

Conversation

@WhichPaths

Copy link
Copy Markdown
Collaborator

Making a BYOA engine runnable means adding its id to about ten hand-kept lists across the server, the renderer and the tooling. Miss one and nothing shouts.

The worst one is BYOA_SOURCES. It is read through normalizeByoaSource():

export function normalizeByoaSource(value: unknown): ByoaSource {
  return typeof value === 'string' && BYOA_SOURCE_SET.has(value)
    ? value as ByoaSource
    : 'byoa-claude'
}

An engine missing from that list does not error. It quietly bills every one of its runs to Claude in the ledger.

Why now

Two things this week were the same shape — a hand-maintained duplicate with no mechanical check:

Adding Gemini (#99) and Qwen (#113) meant walking that list by hand twice. I nearly left byoa-qwen out, and nothing would have told me.

What it checks

Anchored on ENGINE_IDS — the engines the daemon actually probes and wakes — and for each id:

an ADAPTERS entry engine.ts
all three EngineId unions engine.ts, computer/registry.ts, src/types.ts
a label and a PATH binary src/lib/engines.ts
RUNNABLE_ENGINE_IDS src/lib/engines.ts
a version spec computer/cli-version.ts
byoa-<id> in the shared source list runtime/byoa-source.ts
byoa-<id> in all four ledger unions llm-ledger, observability, admin/api, api/client
the binary is spawn-guarded guard-big-brain.mjs R4

The last one is keyed on the binary, not the id — cursor's binary is cursor-agent, so checking ids there would pass while the real binary went unguarded.

Each failure names the list and what it costs to leave it out, rather than just "missing":

server/src/agents/runtime/byoa-source.ts → BYOA_SOURCES
  'gemini' is missing — add 'byoa-gemini' — without it every run of this
  engine is attributed to Claude in the ledger

It fails loudly rather than vacuously

A guard that quietly stops checking is worse than no guard, so every anchor is required: if a refactor moves or renames a declaration, extract() returns null and the guard reports anchor not found — update this guard alongside the refactor. A test also pins that ENGINE_IDS really parsed and found the engines that exist, so scanRepo() cannot pass by having nothing to check.

Verified by breaking it

Dropping 'byoa-gemini' from BYOA_SOURCES:

🚨 engine registry is half-wired:
  server/src/agents/runtime/byoa-source.ts → BYOA_SOURCES
    'gemini' is missing — add 'byoa-gemini' — without it every run of this engine
    is attributed to Claude in the ledger

Dropping gemini from RUNNABLE_ENGINE_IDS:

  src/lib/engines.ts → RUNNABLE_ENGINE_IDS
    'gemini' is missing — add the id to RUNNABLE_ENGINE_IDS, or the engine stays
    detect-only in the UI

Both restore to green. Run against the #113 branch as well, which passes — an independent, mechanical confirmation that the Qwen wiring is complete rather than my having remembered every list.

Shape

Same as the two guards already in CI: scripts/guard-engine-registry.mjs, an npm run guard:engine-registry script, a .d.mts beside it (matching guard-big-brain.d.mts), a guard-engine-registry.test.ts so npm test catches it too, and its own fast-fail CI job.

Runs in milliseconds — it reads eleven files and does string containment, no parsing of the TypeScript.

Making a BYOA engine runnable means adding its id to about ten hand-kept
lists across the server, the renderer and the tooling. Miss one and nothing
shouts. The worst is BYOA_SOURCES: it is read through
normalizeByoaSource(), which maps anything unrecognised to 'byoa-claude',
so an engine missing from that list does not error — it quietly bills every
one of its runs to Claude in the ledger.

That failure mode is not hypothetical. yetone#102 was two identical blocks where
one got the fix and the other did not, and cli-version.ts spent a while
telling the next person to keep the catalog in sync with a list that had
been deleted. Both are the same shape: a hand-maintained duplicate with no
mechanical check.

The guard anchors on ENGINE_IDS and, for each engine, asserts an ADAPTERS
entry, all three EngineId unions, a label, a PATH binary,
RUNNABLE_ENGINE_IDS, a version spec, BYOA_SOURCES, the four ledger source
unions, and that the binary appears in the big-brain guard's direct-spawn
rule (keyed on the BINARY, since cursor's is `cursor-agent`).

Every anchor is required. A guard that quietly stops checking is worse than
no guard, so a declaration that moves or is renamed reports "anchor not
found" rather than passing vacuously — and a test pins that ENGINE_IDS
really parsed, so scanRepo() cannot succeed by finding nothing to check.

Verified by breaking it both ways: dropping 'byoa-gemini' from BYOA_SOURCES
and dropping gemini from RUNNABLE_ENGINE_IDS each fail with the specific
list and the cost of leaving it out.
The guard anchored on `RUNNABLE_ENGINE_IDS = new Set([...])`. yetone#116 collapses
that duplicate by deriving the set from an ordered `RUNNABLE_ENGINES` tuple,
and the guard then reported "anchor not found" — correctly, by its own
design, but it would have made this the thing blocking a change that removes
the very duplication it exists to police.

It now reads whichever shape is present. Verified against all three open
branches: main's Set literal, yetone#116's tuple, and yetone#113's added engine.
Comment thread scripts/guard-engine-registry.mjs Outdated
return problems
}

if (import.meta.url === `file://${process.argv[1]}`) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comparison never matches on Windows: process.argv[1] is a filesystem path such as E:\..., while import.meta.url is a normalized file:///E:/... URL. As a result, npm run guard:engine-registry exits successfully without ever calling scanRepo(), so the local guard silently becomes a no-op.

Comparing filesystem paths keeps the CLI entry point portable:

Suggested change
if (import.meta.url === `file://${process.argv[1]}`) {
if (fileURLToPath(import.meta.url) === process.argv[1]) {

It may also be worth adding a small test that launches the script as a CLI, since the current tests call scanRepo() directly and cannot catch a broken entry-point check.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right on both counts, thank you — applied in the latest commit.

What makes this one sting is that it only looks fine on POSIX by accident: argv[1] starts with /, so file:// + it lands on the same file:///… string. On Windows it cannot:

POSIX  'file:///Users/x/g.mjs' === 'file://' + '/Users/x/g.mjs'   -> true
WIN32  'file:///E:/x/g.mjs'    === 'file://' + 'E:\x\g.mjs'       -> false
                                    (built: file://E:\x\g.mjs)

So the guard exits 0 having checked nothing — which is precisely the failure mode the file argues against in its own header. And guard-big-brain.mjs already had the correct form:

const isMain = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]

I've matched that exactly rather than inventing a third spelling. That is the third time in this batch that the right version already existed and a new copy diverged from it (#102, #116, now this), which is a fair thing to keep pointing out about my own patches.

Your test suggestion was the more valuable half, so I took it: two cases that launch the script as a CLI rather than calling scanRepo() — one asserts it prints a verdict, one sabotages an anchor in a scratch copy and asserts a non-zero exit. Verified they have teeth by replacing if (isMain) with if (false), which fails both (5/7 pass). I could not reproduce the Windows path semantics on macOS, so what these pin is the general property — an entry point that never fires — rather than that specific platform difference.

…ry point

Review catch from @bingqilinweimaotai, and it was right. The entry check was

  if (import.meta.url === `file://${process.argv[1]}`)

On POSIX that happens to match, because argv[1] starts with `/` and the
concatenation lands on the same `file:///…` string. On Windows argv[1] is
`E:\repo\scripts\guard-engine-registry.mjs`, so the built URL is
`file://E:\repo\…` while import.meta.url is `file:///E:/repo/…`. They never
match, the block never runs, and `npm run guard:engine-registry` exits 0
having checked nothing — the guard silently becoming a no-op is the exact
failure this file was written to prevent.

guard-big-brain.mjs already had the correct form:

  const isMain = process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]

so this is once more a case of the right version existing and a new copy
diverging from it. Matched to it exactly rather than inventing a third
spelling.

Two tests, also as suggested on the review: every earlier test calls
scanRepo() directly and so cannot tell a working guard from one whose main
check never fires. These launch the script as a CLI — one asserts it prints
its verdict, one sabotages an anchor in a scratch copy and asserts a
non-zero exit. Replacing `if (isMain)` with `if (false)` fails both.
@WhichPaths
WhichPaths merged commit 2843574 into yetone:main Aug 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants