Skip to content

test(cli): declare measured budgets on the generate-refuses and build spawn files (#23181 slice 11) - #26174

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-mlto9g
Oct 26, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/project-thread-mlto9g

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Requested by Leehom · project thread

Part of #23181 (slice 11). The card is a series and stays open.

Clause-②: no
Metadata-protocol: none

Test-only, comment-only diff: skip-changeset (nothing in any package's files[] changes; packages/cli/test/** is not published).

Rows this PR addresses

The ten @objectstack/cli integration files of the per-case CLI spawn family (test/generate-refuses-*.test.ts and the five named test/build-*.test.ts). Reading used: the per-file list of shard-timings refresh run 40275747937 (PR #25960, the newest refresh with a per-file list; it was not merged, and PR #26089 used the same list for the spec/lint rows). The card's own table (run 38340176409) is older and lower.

file spawns slowest CI reading (run) runs over 10 s / executed local wall, 3 runs (s) budget declared
test/build-callable-names.test.ts 9 123.82 s (40200279306) 5 / 5 84.5 / 89.7 / 86.5 160
test/generate-refuses-unimportable-alias.test.ts 7 94.13 s (40204927516) 3 / 3 65.7 / 68.4 / 64.7 120
test/build-runtime-bundle-bodyless.test.ts 6 92.69 s (40200279306) 3 / 3 57.8 / 57.8 / 57.8 120
test/build-strict-body-count.test.ts 6 82.44 s (40204927516) 3 / 3 55.1 / 58.4 / 59.0 110
test/generate-refuses-namespace-prefix.test.ts 5 69.59 s (40204927516) 3 / 3 49.0 / 50.9 / 49.5 90
test/build-view-container-name.test.ts 5 66.67 s (40204927516) 4 / 4 49.3 / 48.8 / 46.8 90
test/generate-refuses-unparseable-name.test.ts 5 66.41 s (40200279306) 5 / 5 47.9 / 48.5 / 45.5 90
test/generate-refuses-name-outside-charset.test.ts 4 54.70 s (40204927516) 4 / 4 37.9 / 36.7 / 37.5 70
test/generate-refuses-retired-generator.test.ts 4 53.28 s (40204927516) 4 / 4 36.2 / 35.2 / 37.8 70
test/build-text-face-advisory-count.test.ts 3 41.41 s (40200279306) 5 / 5 25.3 / 25.0 / 27.2 60

Budget = slowest CI reading x 1.25, rounded up to the next 10 s (the grammar readTestBudget in scripts/measure-test-shard-timings.mjs enforces; all ten lines parse through it). Pin: the next refresh lists none of these ten in its over-budget table; each appears under "Held to the budget it declares" instead, and is listed again only if a run reads it over its own budget.

Why a declared budget, not a cheaper test

The order's hypothesis was a per-case boot that one beforeAll could share. Measured, it does not hold here:

  • The cost is the spawn floor, not the cases. One tsx bin/run-dev.js spawn costs 8.2 to 8.5 s locally, and os --version alone costs 8.4 to 8.8 s: the source entry has oclif import all 72 command modules through tsx on every start, because no oclif.manifest.json exists. Every file's local wall is its spawn count x 8.5 to 9.5 s (for example 7 spawns, 65 s; 3 spawns, 25 s).
  • There is nothing to share. The five generate-refuses-* files already run all their spawns in one beforeAll. Every spawn in all ten files is a different argv or a different fixture: each one is a single-cause case (a refusal, its dry-run form, the neighbouring layer's verdict, or a control). Cutting one removes a case.
  • So no file with even one spawn fits the 10 s line without changing the spawn entry, which the claim leaves out. These files are exactly the class ruling record 6181973363 letter B names ("spawns the real producer once per single-cause fixture"). Each now carries its one-line // test-budget: header, as eleven other cli spawn files already do.
  • Concurrency inside a file was looked at and not taken. Two parallel spawns cost the same 9.3 s as one, but four take 17.5 s and seven take 28.9 s on this 4-core box. That is packing, not less work, which is the reason packages/cli/vitest.config.ts records for rejecting maxWorkers: 4.

The lever that would make these budgets fall (a decision, not taken here)

Measured on this tree, with an oclif.manifest.json written into packages/cli for the probe and removed afterwards (git status clean): tsx bin/run-dev.js generate object order_line --dry-run drops from 8.2 to 8.5 s to 2.0 to 2.3 s, with byte-identical output (1412 bytes) and the same exit code. --version drops to 0.4 s. A refusal case (generate object class) still exits 1 in 2.2 s. Applied to this family, local wall would fall from about 520 s to about 130 s. It is reported to the seat as an open decision because it changes the spawn shape these files document.

Proof

  • Diff: 10 files, +10 lines, each a header comment at line 2. No case, assertion, timeout, tier or project changes.
  • Cases still isolated: shuffled run at 179c8078a, pnpm exec vitest run --project integration --maxWorkers=2 --sequence.shuffle --sequence.seed=23181 + the 10 files: Test Files 10 passed (10), Tests 91 passed (91), lock VERDICT command-exit 0.
  • Wall before: the table's three local runs, under os-verify-lock, --project integration --maxWorkers=2, the default reporter's per-file module line (the number CI's per-file timings read). The after wall is the same tree's behaviour (comment-only diff), so it was not re-run three times; the shuffled run above is the after reading.
  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at 179c8078a: 55 commands, all exit 0 (two first exited 3 on prerequisites: check-plugin-teardown-shape --self-test on a shallow clone, fixed by fetching 621a4876; check:dual-build-cjs-loads on eight unbuilt packages, fixed by building them). --ran: "55 derived famil(ies) accounted for, 55 run, 0 NOT-MEASURED".
  • pnpm --filter @objectstack/cli typecheck: exit 0 (check:test-typecheck: OK).
  • pnpm exec eslint --no-inline-config packages/cli: exit 0.

Acceptance notes


Generated by Claude Code

… spawn files

Ten packages/cli integration files spawn the source CLI once per
single-cause fixture. Each spawn's floor is the source entry loading all
72 commands (about 8.5 s locally, `os --version` included), so no file
with a spawn fits the 10 s line, and none spawns twice for one fixture.

Each file now carries its ruled `test-budget:` header line (class
"spawns the real producer once per single-cause fixture"), sized from
the slowest CI reading in shard-timings refresh run 40275747937 x 1.25,
rounded up to the next 10 s. Comment-only: no case, assertion, timeout
or project changes.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pq8DZ41armkBBdhQUufpjH
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json c7aa9cde24ca7d17563caa4e01037f3ab1b9e990 → packageMentionDocs.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT · domain:server seat 1 · os-project-manager · session_01R1q2xdtYgU6FKrZ7LgjPCC · 2026-10-26T15:12Z

Head 179c8078ad, report 6418998910 (on #23181, slice 11), claim 6417592580 and its amendments. No contract review is owed: the diff is comment-only.

item reading
Shape draft, base main, Part of #23181 (slice 11), skip-changeset, mergeable clean.
Files 10 files, +10/−0: one // test-budget: header each on the generate-refuses-* and build-* spawn files in packages/cli/test/.
Finding The per-case-boot hypothesis does not hold for this family. Every spawn is a separate single-cause case (the generate-refuses-* files already spawn inside one beforeAll), and each tsx run-dev spawn has an 8.2–8.5 s floor. So no file can get under 10 s within the claim.
Fix This is the ruled route for such files: ruling 6181973363 letter B ("spawns the real producer once per single-cause fixture"). Each header is sized at the slowest CI reading in refresh run 40275747937 (#25960) × 1.25, rounded up (60–160 s), and names its spawns. Departure (accepted): 10 files rather than about 8, because each change is one comment line.
Evidence 36 check runs success and 16 expected skips on this head, 0 failed. Shuffled run 10/10 files, 91/91 tests; 55/55 derived gates exit 0; eslint and typecheck clean.
Carried The open question (the 8.5 s spawn floor caused by the missing oclif.manifest.json) is ruled A by this seat and dispatched as slice 14 (amendment 6419075788): a test-only globalSetup writes the manifest per run. A later refresh re-sizes these headers. The duplicate spawns across generate-refuses-* are named in the PR's Acceptance notes and left in place.
Landing 「按照 pm-dispatch 规则复核过的都授权」 (Leehom, 2026-10-25T11:35Z): ready and auto-merge through the relay.

Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 26, 2026 15:13
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 26, 2026 15:13
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 26, 2026
Merged via the queue into main with commit 3f970b9 Oct 26, 2026
53 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/project-thread-mlto9g branch October 26, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants