Skip to content

Commit 7234f33

Browse files
test(cli): declare measured budgets on the eight os verify spawn files (#23181 slice 17) (#26507)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1X6RuqbjMYUre73wZPpaiDRN)_ Part of #23181 (slice 17). The card is a series and stays open. Clause-②: no Metadata-protocol: none Test-only, comment-only diff: `skip-changeset`. Eight lines under `packages/cli/test/`, which is not in `@objectstack/cli`'s `files[]` (`dist`, `README.md`, `CHANGELOG.md`); no `files[]` changes in any package. ## 维护者速读 - 改了什么:给 cli 里 8 个真实启动 `os verify` 的测试文件(`test/verify-*.test.ts`)各加一行第 2 行的 `// test-budget:` 声明,只是注释。 - 为什么改:这 8 个文件在 CI 上仍超 10 s;逐个数过,每个文件里每次 CLI 启动都是不同的(命令、参数、夹具)组合,没有重复可共享,而且已在 `beforeAll` 里各跑一次,所以按裁决 6181973363 B 声明实测预算。 - 风险与回滚:零运行时风险,不改断言、超时与分层;回滚即删这 8 行。 - 先看:`packages/cli/test/verify-parent-write-reach.test.ts`、`packages/cli/test/verify-reference-shapes.test.ts` 第 2 行(预算最大的一行,和第三次 CI 读数抬高了的一行)。 - 两分钟验证:用 `scripts/measure-test-shard-timings.mjs` 的 `readTestBudget` 解析这 8 个文件,预期 8 行都返回 `spawns the real producer once per single-cause fixture`、行号 2。 - 需要你做的:无,席位落地。 ## What changes One header line at line 2 of each file below, in the ruled grammar `// test-budget: N — spawns the real producer once per single-cause fixture: REASON` (ruling record 6181973363, letter B; grammar `readTestBudget` in `scripts/measure-test-shard-timings.mjs`). N is the slowest CI reading x 1.25, rounded up to the next 10 s. Nothing else changes: no assertion, timeout, tier or helper. | file | tests | CLI spawns (counted) | distinct (command, flags, fixture) | CI 40392443783 (s) | CI 40424746941 (s) | CI 40447801748 (s) | local (s) | budget | |:--|--:|--:|--:|--:|--:|--:|--:|--:| | `test/verify-parent-write-reach.test.ts` | 3 | 3, in one `beforeAll` | 3 | 55.76 | 36.47 | 35.05 | 40.9 | 70 | | `test/verify-delete-behaviors.test.ts` | 3 | 3, in one `beforeAll` | 3 | 52.68 | 34.05 | 31.86 | 35.6 | 70 | | `test/verify-dashboard-windows.test.ts` | 3 | 3, in one `beforeAll` | 3 | 50.49 | 34.23 | 32.75 | 35.3 | 70 | | `test/verify-unique-scopes.test.ts` | 3 | 3, in one `beforeAll` | 3 | 48.96 | 49.91 | 31.63 | 36.1 | 70 | | `test/verify-reference-shapes.test.ts` | 3 | 3, in one `beforeAll` | 3 | 37.07 | 43.50 | 45.37 | 32.1 | 60 | | `test/verify-author-time-stage.test.ts` | 3 | 4, one per case and step | 4 | 38.85 | 28.55 | 31.82 | 31.0 | 50 | | `test/verify-json-stdout.test.ts` | 4 | 2, in one `beforeAll` | 2 | 32.04 | 32.30 | 21.23 | 21.3 | 50 | | `test/verify-host-root.test.ts` | 2 | 1, in its `beforeAll` | 1 | 13.13 | 9.64 | 10.37 | 10.5 | 20 | ## Why **The count.** In the five flag files (`parent-write-reach`, `delete-behaviors`, `dashboard-windows`, `unique-scopes`, `reference-shapes`) the three spawns are `verify --json` on the app, `verify --FLAG --json` on the app, and `verify --FLAG --json` on the control fixture (in `reference-shapes`, the same directory under `OS_ALLOW_LAX_VALUE_SHAPES=1`). `author-time-stage` runs `validate --json`, `verify --json` and `verify` on the planted stack and `verify` on the clean control. `json-stdout` runs `verify --json` and `verify` once each. `host-root` runs one `verify --app` from a foreign cwd. No triple repeats inside a file, and every fixture is the file's own, so there is no spawn to share. Seven of the eight already spawn in one `beforeAll`; `author-time-stage` spawns inside its cases, one distinct triple each, so hoisting them would move the cost and save nothing. What is left is the spawn floor. **The readings.** No shard-timings refresh has run on `main` since slice 14 (PR #26256). The CI numbers are the vitest module lines (they include `beforeAll`) of the three scheduled full-battery `ci.yml` runs on `main` after it: slice 15's two, 40392443783 and 40424746941, and the newer 40447801748 (head `26e7c75e9a`; its Test Core shards and aggregator all passed, only a Test UI shard failed). The newer run raises one file's maximum: `verify-reference-shapes` 45.37 s, which still rounds to 60. **No file left out.** `verify-host-root` is under 10 s in one reading (9.64 s) but over in the other two, so it takes a header. No file needed another mechanism. ## Tests Comment-only diff, so behaviour and wall are the tree's. A/B of a comment line measures nothing (slice 15's precedent), so the after reading is one run on this head's tree. <details><summary>Local run of the family (one lock hold)</summary> `cd packages/cli && pnpm exec vitest run --project integration --maxWorkers=2 --reporter=default` plus the eight files, under `os-verify-lock`, on a shared container, manifest setup on (queue tier): `Test Files 8 passed (8)`, `Tests 24 passed (24)`, VERDICT command-exit 0, lock held 147 s. Every file reads under its declared budget. </details> <details><summary>Header parse</summary> All eight lines parse through `readTestBudget` (exit 0): each returns class `spawns the real producer once per single-cause fixture` at line 2, seconds 70 / 70 / 70 / 70 / 60 / 50 / 50 / 20. </details> <details><summary>Typecheck and lint</summary> `pnpm --filter @objectstack/cli typecheck`: VERDICT command-exit 0 (`check:test-typecheck: OK`). `pnpm exec eslint --no-inline-config` on the eight files: exit 0. `pnpm check:nul-bytes`: exit 0. </details> ## Acceptance notes - Pin: the next shard-timings refresh lists none of these eight files in its over-budget table. Each comes back only if a run reads it over its own budget. - The budgets rest on three CI readings. A refresh over its 20-run window may read one of them higher, and that is the refresh's job. - `packages/verify`'s own files over 10 s in slice 15's local reading were measured only, not changed; their causes are in the slice report on #23181 for a later slice. ## Verification Local gates at `dfec98106`: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 48 commands. Each ran with its exit code captured before any pipe. 46 exited 0 on the first pass. `check-plugin-teardown-shape.mjs --self-test` exited 3 on the shallow clone and 0 after `git fetch origin 621a487`. `check:dual-build-cjs-loads` exited 3 (ten packages outside the cli closure had no `dist/`) and 0 after building them. `--ran` reconciliation: `48 derived famil(ies) accounted for — 48 run, 0 NOT-MEASURED`, exit 0. The repo-wide `pnpm lint` is left to CI. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (not touched) - [x] 2. Every name I added uses the casing for its kind. (no names added) - [x] 3. No formatter ran; `pnpm lint` passes. (eslint on the touched files: exit 0) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (publishes nothing: `skip-changeset`) - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (not touched) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. (the maintainer brief is in Chinese, per AGENTS.md) - [x] The *Before Submitting* list in CONTRIBUTING.md is done. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01FEkQ5nuFC67LujGL3Fsyiy Co-authored-by: Claude <noreply@anthropic.com>
1 parent 162f668 commit 7234f33

8 files changed

Lines changed: 8 additions & 0 deletions

‎packages/cli/test/verify-author-time-stage.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 50 — spawns the real producer once per single-cause fixture: four spawns, one per (command, flags, fixture) triple, one per case and step (`validate --json`, `verify --json` and `verify` on the planted stack, `verify` on the clean control); slowest CI reading 38.85 s (run 40392443783) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify` runs the author-time rules FIRST, and a stack they refuse

‎packages/cli/test/verify-dashboard-windows.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 70 — spawns the real producer once per single-cause fixture: three `os verify` spawns, one per (flags, fixture) pair, already in one beforeAll (`--json` and `--dashboard-windows --json` on the app, `--dashboard-windows --json` on the control); slowest CI reading 50.49 s (run 40392443783) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --dashboard-windows` runs the dashboard date-window family

‎packages/cli/test/verify-delete-behaviors.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 70 — spawns the real producer once per single-cause fixture: three `os verify` spawns, one per (flags, fixture) pair, already in one beforeAll (`--json` and `--delete-behaviors --json` on the app, `--delete-behaviors --json` on the hook control); slowest CI reading 52.68 s (run 40392443783) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --delete-behaviors` runs the delete-behaviour family and

‎packages/cli/test/verify-host-root.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 20 — spawns the real producer once per single-cause fixture: one `os verify --app` spawn from a foreign cwd, in its beforeAll; slowest CI reading 13.13 s (run 40392443783) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --app <dir>/objectstack.config.mjs`, run from a working

‎packages/cli/test/verify-json-stdout.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 50 — spawns the real producer once per single-cause fixture: two `os verify` spawns, one per face (`--json` and text), already in one beforeAll; slowest CI reading 32.30 s (run 40424746941) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --json` on a stack that REACHES THE RUNTIME STAGE writes

‎packages/cli/test/verify-parent-write-reach.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 70 — spawns the real producer once per single-cause fixture: three `os verify` spawns, one per (flags, fixture) pair, already in one beforeAll (`--json` and `--parent-write-reach --json` on the app, `--parent-write-reach --json` on the hook control); slowest CI reading 55.76 s (run 40392443783) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --parent-write-reach` runs the parent-derived write reach

‎packages/cli/test/verify-reference-shapes.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 60 — spawns the real producer once per single-cause fixture: three `os verify` spawns, one per (flags, posture) pair, already in one beforeAll (`--json`, `--reference-shapes --json`, and `--reference-shapes --json` under OS_ALLOW_LAX_VALUE_SHAPES); slowest CI reading 45.37 s (run 40447801748) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --reference-shapes` runs the reference-value shape family

‎packages/cli/test/verify-unique-scopes.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 70 — spawns the real producer once per single-cause fixture: three `os verify` spawns, one per (flags, fixture) pair, already in one beforeAll (`--json` and `--unique-scopes --json` on the app, `--unique-scopes --json` on the control); slowest CI reading 49.91 s (run 40424746941) of the three scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941, 40447801748), x 1.25, rounded up
23

34
/**
45
* PIN — `os verify --unique-scopes` runs the unique-scope family and counts

0 commit comments

Comments
 (0)