Skip to content

test(cli): write oclif.manifest.json once per integration run, so a CLI spawn reads its command table - #26256

Merged
objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/project-thread-lxcldr
Oct 26, 2026
Merged

objectstack-fleet[bot] merged 3 commits into
mainfrom
claude/project-thread-lxcldr

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Leehom · project thread

Part of #23181 (slice 14). The card is a series and stays open.

Clause-②: no
Metadata-protocol: none

Test infrastructure only: skip-changeset. @objectstack/cli's files[] is dist, README.md and CHANGELOG.md, unchanged, and nothing in it changes. The diff touches packages/cli/vitest.config.ts, one new module under packages/cli/test/helpers/, one test file under packages/cli/test/, and the root .gitignore. No build step, no new gate, no product code.

What this does

Seat ruling A on slice 11's open question (report 6418998910 on #23181). A vitest globalSetup, packages/cli/test/helpers/oclif-manifest.global-setup.ts, writes packages/cli/oclif.manifest.json from source once per run of the integration project and removes it at teardown. The file is gitignored.

This package ships no oclif manifest, so every CLI process builds its command table by importing all 72 command modules. Through tsx bin/run-dev.js that is a floor of about 8 s per spawn. With the manifest beside package.json, oclif reads the table from it and imports only the command it runs. Outside the one file ruling A1 changed (below), spawn shape, the bin/run-dev.js entry and NODE_ENV are unchanged.

Why the written manifest is transparent

  • It is the object oclif's own root plugin builds in memory when no manifest exists (new Plugin({ isRoot: true, root }), loaded in a tsx child under NODE_ENV=development, as bin/run-dev.js runs), with errorOnManifestCreate on so that a command module that fails to load fails the generation loudly, naming the module.
  • Two keys are removed from every command: isESM and relativePath. This is required. When a cached command carries both, @oclif/core 5.1.2 imports root + relativePath verbatim, and a manifest generated from source records src/commands/ID.ts. Measured control: with those keys kept, node bin/run.js generate object order_line --dry-run imports packages/cli/src/commands/generate.ts and exits 1 (ModuleLoadError). With them removed, oclif takes its no-manifest path. Its debug log reads (import) CLI/dist/commands/generate.js under bin/run.js and (import) CLI/src/commands/generate.ts under bin/run-dev.js. So the manifest supplies the command table and nothing about where a command's code comes from.
  • Byte comparison, without and with the manifest, 9 invocations x 2 entries (tsx bin/run-dev.js and node bin/run.js): --version, --help, serve --help, generate object --help, generate object order_line --dry-run, generate object class (exit 1), generate object (missing argument, exit 1), validate in examples/app-todo, and an unknown command (exit 2). stdout, stderr and exit code are identical in 52 of 54 streams. The 2 that differ are validate's stdout on both entries, and they differ only in the two printed elapsed-time lines (Load time: 148ms against 164ms, and Validation passed (279ms) against (278ms)).

Scope: the queue population only

Under OS_TEST_TIERS=nightly neither project carries the setup. The e2e tier holds the suites that pin what a manifest changes, which is Config.load() importing every command module (run-dev-unbuilt-workspace.e2e, the two *-stderr-nonblocking.e2e files). The product ships no manifest, so that is the shape those pins keep seeing. The setup is wired per project, not at the root: with extends: true a root globalSetup would run for the root project and both tier projects, three times.

Ownership, nesting and cleanup (the order's three hypotheses)

  • Written once, before any worker spawns. vitest runs every project's globalSetup before pool.runTests. The file is written to a temporary sibling and renamed over the target, so a spawn reads a whole manifest or none.
  • Nested vitest runs leave it alone. The owning run sets OS_TEST_OCLIF_MANIFEST_OWNER in its own process. Workers and childEnv() children inherit it, and a vitest started inside the run does nothing. Measured with a throwaway integration test that spawned vitest run --project integration through childEnv(): the worker and the child env both carried the owner pid, the manifest's owner was the same before and after (3443), and the nested run took 3.5 s, with no 9 s generation.
  • Two runs on one checkout. Each run records its pid in the file (writtenBy.pid) and removes the file only while that pid is still its own. Measured with two --project integration runs started 4 s apart: the owner went 3691 then 3788, both runs were green (5/5 and 7/7), and after both the file was gone. The run left without a manifest continues at the old floor with the same output.
  • Crash. Every integration run regenerates unconditionally, and a unit-only run removes a file whose writer is dead. Measured: after a kill -9 of the owning vitest, the file stayed with a dead owner. The next --project unit run removed it. A planted dead-owner file (pid 999999) was regenerated by the next integration run (owners seen: 999999, then 4631) and removed at its teardown. After a SIGINT mid-run the file was gone (teardown, or the exit listener for a run that ends without one).

One test file changed, by seat ruling A1

test/published-entry-node-env-source-reroute.test.ts (#12271) spawned node bin/run.js --version. That reached the reroute only because, with no manifest, Config.load() imports every command module. With the manifest present, --version loads no command. In the full run below, its CONTROL stopped reproducing the card, and its three positive legs passed over a fixture that armed nothing. Its spawns now run explain object, a real command whose module imports the trapped chalk, with OCLIF_COLUMNS=100000 so oclif renders the load failure on one line. With a manifest that failure is the error handle() prints, hard-wrapped at 80 columns on a pipe, and a wrap can split the signature or the src/commands path. No assertion was changed or loosened.

Legs measured both ways, against the same armed fixture (exit / count of the card signature / count of packages/cli/src/commands):

leg no manifest (main's shape) manifest present
CONTROL, neutralised, NODE_ENV=development 1 / 4 / 3 1 / 1 / 1
CONTROL, neutralised, NODE_ENV=production 0 / 0 / 0 0 / 0 / 0
NODE_ENV=development 0 / 0 / 0 0 / 0 / 0
NODE_ENV=test 0 / 0 / 0 0 / 0 / 0
NODE_ENV=production 0 / 0 / 0 0 / 0 / 0

So the control still goes red on the trap on main's shape, and also with the manifest. The file under vitest: 5/5 passed twice under this config (manifest written) and 5/5 twice under the base config (no manifest).

Measurements

All on one shared 4-vCPU container under os-verify-lock. Per-file numbers are the default reporter's module line.

Per spawn (the byte-comparison runs above): tsx bin/run-dev.js 7.6 to 9.1 s without, and with the manifest 0.48 to 0.55 s for --version and help, 1.7 to 3.9 s for commands that run. node bin/run.js 2.1 to 2.6 s without, 0.15 to 2.1 s with. One generation: 8.8 to 9.9 s.

Slice 11's spawn family (PR #26174's ten files), interleaved A/B, --project integration --maxWorkers=2, BASE = c7aa9cde vitest.config.ts, HEAD = this branch, N=3:

file base, 3 runs (s) head, 3 runs (s) median ratio
test/build-callable-names.test.ts 87.6 / 87.4 / 80.9 35.2 / 35.2 / 37.3 2.5x
test/generate-refuses-unimportable-alias.test.ts 67.6 / 64.0 / 61.4 15.7 / 15.8 / 16.6 4.1x
test/build-runtime-bundle-bodyless.test.ts 63.2 / 59.2 / 54.8 24.4 / 23.1 / 24.9 2.4x
test/build-strict-body-count.test.ts 56.0 / 55.9 / 54.6 23.0 / 23.7 / 25.8 2.4x
test/generate-refuses-unparseable-name.test.ts 46.7 / 47.6 / 43.6 11.2 / 10.3 / 11.4 4.2x
test/generate-refuses-namespace-prefix.test.ts 48.2 / 46.0 / 43.7 11.8 / 10.6 / 12.6 3.9x
test/build-view-container-name.test.ts 48.2 / 46.0 / 45.2 19.4 / 18.9 / 21.6 2.4x
test/generate-refuses-retired-generator.test.ts 38.0 / 35.5 / 38.1 8.8 / 8.0 / 8.2 4.6x
test/generate-refuses-name-outside-charset.test.ts 35.6 / 37.8 / 37.7 9.8 / 8.3 / 9.6 3.9x
test/build-text-face-advisory-count.test.ts 28.5 / 27.8 / 26.6 12.5 / 11.1 / 12.1 2.3x
sum of file lines 519.5 / 507.1 / 486.5 172.0 / 164.9 / 180.1 2.9x

Run wall: base 285.3 / 279.6 / 258.2 s, head 110.8 / 106.8 / 115.7 s (generation included). Tests 91/91 every run, and no manifest was left after any run. The build-* files keep more of their wall because their build work is real, not spawn floor.

Budget headers: slice 11's // test-budget: headers (PR #26174) are not lowered here. A later refresh re-sizes them.

Proof

Acceptance notes

  • With both tiers in one run, the unit project's in-process Config.load({ root }) calls (for example exit-signal.pin, db/clean.operator-text-21418) also find the manifest, because every project's globalSetup runs before any test. They load commands through the same no-manifest path (no relativePath), and the full run above was green there. A unit-only run generates nothing; it only removes a file whose writer is dead.
  • A flag default computed at module load is frozen into the manifest at generation, and only help text reads it. One exists: serve --port's default reads OS_PORT / PORT. os serve --help therefore shows the generation-time value, which is the run's own environment. Parsing still uses the loaded class. No test runs --help with PORT set.
  • The setup's failure mode is loud by design. If a command module cannot load, the integration run fails at setup with the generator's stderr (the ModuleLoadError naming the module), instead of every spawn test failing on "command not found".

Generated by Claude Code

@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

2 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

What this run could not see
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 31 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 81bba6c3f5be5b6cbabf76511383940349f23b15 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 0c5ad00513bcfdddf69218bea1c9016d7e9136e7 — the merge of head 8e5065109d97177536dcda34469ea409520dbc80 into base 81bba6c3f5be5b6cbabf76511383940349f23b15, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 0c5ad00513bcfdddf69218bea1c9016d7e9136e7 && git checkout 0c5ad00513bcfdddf69218bea1c9016d7e9136e7
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 81bba6c3f5be5b6cbabf76511383940349f23b15 8e5065109d97177536dcda34469ea409520dbc80 && git checkout -B drift-repro 81bba6c3f5be5b6cbabf76511383940349f23b15 && git merge --no-ff 8e5065109d97177536dcda34469ea409520dbc80

node scripts/docs-audit/affected-docs.mjs --json 81bba6c3f5be5b6cbabf76511383940349f23b15

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT · domain:server seat 1 · os-project-manager · session_01R1q2xdtYgU6FKrZ7LgjPCC · 2026-10-26T17:41Z

Head 8e5065109d, report 6422753200 (on #23181, slice 14), claim 6417640032 and its amendments (newest 6420575177, which adds the A1 file). No contract review is owed: the change is test-only, touches no packages/spec/**, and nothing in files[] changes.

item reading
Shape Draft, base main, Part of #23181 (slice 14), skip-changeset, mergeable clean.
Files 4 files, +207/−9, all on slice 14's surface. The new test/helpers/oclif-manifest.global-setup.ts, the wiring in packages/cli/vitest.config.ts, two .gitignore lines, and the A1 file published-entry-node-env-source-reroute.test.ts.
Fix Seat ruling A. For a run, a test-only globalSetup writes oclif.manifest.json from source, using oclif's own root Plugin, and removes it at teardown. It strips isESM/relativePath from every command, so the built entry still loads dist/ and never source. The write is atomic (temp file then rename), ownership is tracked by pid, nested vitest children leave it alone, and the setup runs only for the queue population. The nightly e2e tier, which pins the unmanifested Config.load(), runs without it.
A1 The #12271 pin now runs explain object, a real command, instead of --version, with OCLIF_COLUMNS wide. Its control leg stays red on main's shape both without a manifest and with one. No assertion is weakened.
Evidence Spawn cost falls from 7.6–9.1 s to 0.5–3.9 s. The slice 11 family runs 285/280/258 s → 111/107/116 s (N=3, 91/91 tests each run). 52 of 54 output streams are byte-identical; the other 2 differ only in printed timings. The pin file passes 5/5 under both configs. 56/56 local gates exit 0. On this head, 36 check runs succeeded and 16 were expected skips, with 0 failed.
Carried Slices 10 and 12 took a declared budget, ruled as A then C. The next refresh re-reads those rows against this manifest, and their budgets tighten then.
Landing 「按照 pm-dispatch 规则复核过的都授权」 (Leehom, 2026-10-25T11:35Z): ready and auto-merge through the relay.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants