Skip to content

Commit 68901b0

Browse files
test(cli): declare measured budgets on ten os-serve-path boot files (#23181 slice 15) (#26440)
<!-- claude-projects-attribution: {"github_login":"os-project-manager"} --> _Requested by **Leehom** · [project thread](https://claude.ai/code/project/chan_01TenzsRopZj4jMbvjX8uz1X?thread=cmsg_01TenzsRopZj4jMbvjX8uz1XG3Skuzg27tRY99NwTYbbs1)_ Part of #23181 (slice 15). The card is a series and stays open. Clause-②: no Metadata-protocol: none Test-only, comment-only diff: `skip-changeset`. Ten lines under `packages/cli/test/`, which is not in `@objectstack/cli`'s `files[]` (`dist`, `README.md`, `CHANGELOG.md`); no `files[]` changes in any package. ## 维护者速读 - 改了什么:给 cli 里 10 个「真实启动 `os serve` / `os dev` / `os start`」的测试文件各加一行第 2 行的 `// test-budget:` 声明,只是注释。 - 为什么改:#26256 把每次 CLI 启动从约 8 s 降到约 2 s 之后,这 10 个文件仍超 10 s;它们的耗时就是每个独立夹具各启动一次服务,没有可共享或重复的启动,按裁决 6181973363 B 声明实测预算。 - 风险与回滚:零运行时风险,不改断言、超时与分层;回滚即删这 10 行。 - 先看:`packages/cli/test/serve-config-boot-residual.integration.test.ts`、`packages/cli/test/artifact-flag-precedence.integration.test.ts` 第 2 行(预算最大的两行)。 - 两分钟验证:`node -e` 用 `scripts/measure-test-shard-timings.mjs` 的 `readTestBudget` 解析这 10 个文件,预期 10 行都返回 `spawns the real producer once per single-cause fixture`、行号 2。 - 需要你做的:无,席位落地。 ## What changes One header line at line 2 of each file below, the ruled grammar `// test-budget: N — spawns the real producer once per single-cause fixture: REASON` (ruling record 6181973363, letter B; grammar `readTestBudget` in `scripts/measure-test-shard-timings.mjs`). N = slowest CI reading x 1.25, rounded up to the next 10 s. Nothing else changes: no assertion, no timeout, no tier, no helper. | file | spawns (counted from the code) | CI run 40392443783 (s) | CI run 40424746941 (s) | local, 3 runs (s) | budget | |:--|:--|--:|--:|--:|--:| | `test/artifact-flag-precedence.integration.test.ts` | 10 boots: 3 `os dev`, 7 `os start` (each a supervisor plus its `serve` child) | 166.18 | 206.80 | 152.9 / 153.2 / 138.3 | 260 | | `test/serve-config-boot-residual.integration.test.ts` | 17: 8 served `os serve` boots, 2 refused boots, 5 `os build`, 2 `os validate` | 191.07 | 190.07 | 140.5 / 142.9 / 128.8 | 240 | | `test/serve-package-declared-capabilities.test.ts` | 5 `os serve` boots, one per case and fixture | 68.81 | 43.26 | 49.7 / 48.1 / 47.0 | 90 | | `test/serve-config-boot-flat-docs.integration.test.ts` | 3 `os serve --dev` boots, one per fixture | 47.95 | 30.34 | 35.6 / 33.9 / 33.2 | 60 | | `test/serve-view-container-read.integration.test.ts` | 3: config boot, `os build`, artifact boot | 35.48 | 36.70 | 29.3 / 26.3 / 25.6 | 50 | | `test/serve-host-config-metadata-door.integration.test.ts` | 2 `os serve` boots of one host config (source, compiled twin) | 26.55 | 34.85 | 29.1 / 25.0 / 25.1 | 50 | | `test/serve-no-auth-secret-refusal.integration.test.ts` | 10 `os serve` boots, one per case (5 refused before the kernel, 5 reach ready) | 25.87 | 31.59 | 24.7 / 22.8 / 24.2 | 40 | | `test/serve-migrate-exit-schema-parity.integration.test.ts` | 2 migrate-and-exit runs (server, `--no-server`) | 30.02 | 19.15 | 22.9 / 20.5 / 21.3 | 40 | | `test/dev-no-watch.pin.test.ts` | 3 `os dev` spawns, each a distinct argv and directory | 24.16 | 17.74 | 23.9 / 22.2 / 23.5 | 40 | | `test/dev-connector-env-parity.integration.test.ts` | 3 boots, one per port leg | 14.45 (old 3-case shape, not used) | 16.35 | 17.7 / 17.9 / 17.6 | 30 | ## Why **The reading.** No shard-timings refresh has run on `main` since slice 14 (PR #26256, `9c922b55c1`); the newest refresh is run 40275747937, which is older. So the CI numbers above are the vitest module lines (they include `beforeAll`) of the only two scheduled full-battery `ci.yml` runs on `main` after it: 40392443783 (head `69aa630b9f`) and 40424746941 (head `e843b5b320`). Both trees carry `packages/cli/test/helpers/oclif-manifest.global-setup.ts`, so every spawn there already has the manifest's 2 s floor. The full re-measured offender table (cli from both runs, plus a local run of the other 64 `domain:server` packages) is in the slice report on #23181. **Why these ten are one family.** Each file's wall is real served boots of the CLI, one per distinct fixture or flag, already sequential in `beforeAll` hooks or one per case. Read per file: no boot is repeated on the same fixture and flag, and no two cases could read one boot. So the two recipes that cut time do not apply. Hoisting boots into a shared `beforeAll` gains nothing because the cases are distinct fixtures. Sharing a fixture would merge distinct fixtures. What is left is the spawn floor. The local per-case lines show it: in `serve-package-declared-capabilities` each case is one boot of 7.3 to 12.4 s, and in `serve-no-auth-secret-refusal` each case is one boot of 1.6 to 3.6 s. **Why not the other boot files.** `dev-boot-output-order.integration.test.ts` boots one fixture several times on purpose (one order "on every boot"). `package-restart-acceptance.integration.test.ts` and `dev-standalone-self-heal.integration.test.ts` boot one database twice because the restart is what they test. The class text "once per single-cause fixture" would be false for those three, so they are left for their own reading. `serve-runtime-state-project-key.test.ts` spawns a writer child, not the CLI. ## Tests Comment-only diff, so behaviour and wall are the tree's. A/B of a comment line measures nothing, so the after reading is the same three runs on this head's tree. <details><summary>Local runs of the family (N=3, one lock hold)</summary> `cd packages/cli && pnpm exec vitest run --project integration --maxWorkers=2 --reporter=default` plus the ten files, three times in one `os-verify-lock` hold, on a shared 4-vCPU container, manifest setup on (queue tier). Each run: `Test Files 10 passed (10)`, `Tests 71 passed (71)`, exit 0. Run walls 297.2 / 285.2 / 270.8 s. Sum of file lines 526.4 / 512.8 / 484.7 s. </details> <details><summary>Header parse</summary> All ten lines parse through `readTestBudget` (exit 0): each returns class `spawns the real producer once per single-cause fixture` at line 2, seconds 260 / 240 / 90 / 60 / 50 / 50 / 40 / 40 / 40 / 30. </details> <details><summary>Typecheck and lint</summary> `pnpm --filter @objectstack/cli typecheck`: VERDICT command-exit 0 (`check:test-typecheck: OK`). `pnpm exec eslint --no-inline-config` on the ten files: exit 0. `node scripts/check-nul-bytes.mjs`: exit 0. </details> ## Acceptance notes - Pin: the next shard-timings refresh lists none of these ten files in its over-budget table. Each appears under the budgets they declare, and comes back only if a run reads it over its own budget. - Slice 11 (PR #26174) headers whose files read under 10 s in both runs above, for the next refresh to drop: `test/generate-refuses-retired-generator.test.ts` (9.95 / 9.67 s) and `test/generate-refuses-name-outside-charset.test.ts` (8.77 / 6.49 s). Under 10 s in one run only: `-namespace-prefix` (13.11 / 8.61) and `-unparseable-name` (12.37 / 7.82). Still over in both: `-unimportable-alias` and the five `build-*` files. Other slices' headers under 10 s in both runs: `serve-plugin-auth-load-failure-refusal` (9.86 / 7.15) and `serve-datasource-mapping-before-sync` (7.64 / 5.08). ⛔ Not lowered by hand, per the order: a refresh re-sizes them. - The budgets rest on two CI readings. A refresh over its 20-run window may read one of them higher, and that is the refresh's job. ## Verification Local gates at `0863e1aa3`: `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived 54 commands. Each one ran with its exit code captured before any pipe, and all 54 exit 0. One of them, `check-plugin-teardown-shape.mjs --self-test`, first exited 3 on the shallow clone; it exited 0 after `git fetch origin 621a487`. `--ran` reconciliation: `54 derived famil(ies) accounted for — 54 run, 0 NOT-MEASURED`, exit 0. The repo-wide `pnpm lint` is left to CI. ## Human Minimum - [x] 1. Schemas start as Zod; `packages/spec` holds no runtime logic. (not touched) - [x] 2. Every name I added uses the casing for its kind. (no names added) - [x] 3. No formatter ran; `pnpm lint` passes. (eslint on the touched files: exit 0) - [x] 4. One issue, one branch off an up-to-date `main`, one pull request. - [x] 5. A change that publishes carries a changeset, and both declaration lines are filled in here and in the changeset. (publishes nothing: `skip-changeset`) - [x] 6. `packages/spec` touched: its generated artifacts are regenerated and committed. (not touched) - [x] 7. No hand edits to `content/docs/references/`, `packages/*/CHANGELOG.md` or `content/docs/releases/`. - [x] 8. Title, body and comments are in English. (the maintainer brief is in Chinese, per AGENTS.md) - [x] The *Before Submitting* list in CONTRIBUTING.md is done. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01FEkQ5nzM2GqFAcHGd5cWx4 --- _Generated by [Claude Code](https://claude.ai/code/session_01FEkQ5nzM2GqFAcHGd5cWx4)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6746140 commit 68901b0

10 files changed

Lines changed: 10 additions & 0 deletions

‎packages/cli/test/artifact-flag-precedence.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 260 — spawns the real producer once per single-cause fixture: ten boots in one beforeAll, each a distinct precedence leg: three `os dev`, seven `os start`, each a supervisor plus its `serve` child read through the metadata door; slowest CI reading 206.80 s (run 40424746941) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #21501 — an explicitly named artifact is the stack that is SERVED, even from

‎packages/cli/test/dev-connector-env-parity.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 30 — spawns the real producer once per single-cause fixture: three boots in one beforeAll, each a distinct port leg (`os dev -p N`, `os serve --port N`, `os dev -p N` with N held); slowest CI reading 16.35 s (run 40424746941, the first run of its current six-case shape) after the oclif test manifest (PR #26256), x 1.25, rounded up
23

34
/**
45
* #26130 — on an `os dev` boot, a connector value the config reads from the

‎packages/cli/test/dev-no-watch.pin.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 40 — spawns the real producer once per single-cause fixture: three `os dev` spawns in one beforeAll, each a distinct argv and directory on its workspace-orchestration branch; slowest CI reading 24.16 s (run 40392443783) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* PIN (#20681) — `os dev`'s watch opt-out is reachable, and the argv that used

‎packages/cli/test/serve-config-boot-flat-docs.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 60 — spawns the real producer once per single-cause fixture: three `os serve --dev` boots, one per fixture (multi-package, single-package, no owner); slowest CI reading 47.95 s (run 40392443783) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #22405 — a CONFIG boot of a multi-package stack serves the stack's flat

‎packages/cli/test/serve-config-boot-residual.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 240 — spawns the real producer once per single-cause fixture: seventeen spawns of the source CLI in five row beforeAll hooks, each a distinct fixture or flag: eight `os serve` boots read through the metadata and data doors, two refused `os serve` boots, five `os build`, two `os validate`; slowest CI reading 191.07 s (run 40392443783) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #22521 — a CONFIG boot of a multi-package stack answers for the stack's

‎packages/cli/test/serve-host-config-metadata-door.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 50 — spawns the real producer once per single-cause fixture: two `os serve` boots of one host config in one beforeAll, the source leg and its compiled twin; slowest CI reading 34.85 s (run 40424746941) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #22989 — `os serve` of a HOST config serves the stack's declared `apis`

‎packages/cli/test/serve-migrate-exit-schema-parity.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 40 — spawns the real producer once per single-cause fixture: two `os serve` migrate-and-exit runs in one beforeAll, the server boot and the --no-server boot whose table sets are compared; slowest CI reading 30.02 s (run 40392443783) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* [#22202] A `--no-server` migrate-and-exit run provisions the SAME table set

‎packages/cli/test/serve-no-auth-secret-refusal.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 40 — spawns the real producer once per single-cause fixture: ten `os serve` boots, one per case, each a distinct posture (five refused before the kernel starts, five reach ready); slowest CI reading 31.59 s (run 40424746941) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
import { afterAll, beforeAll, describe, expect, it } from 'vitest';
45
import { spawn, type ChildProcessByStdio } from 'node:child_process';

‎packages/cli/test/serve-package-declared-capabilities.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 90 — spawns the real producer once per single-cause fixture: five `os serve` boots, one per case and per fixture (three reach the banner, two are refused); slowest CI reading 68.81 s (run 40392443783) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #22288 — `os serve` reads a multi-package config's PACKAGE-OWNED keys

‎packages/cli/test/serve-view-container-read.integration.test.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
// test-budget: 50 — spawns the real producer once per single-cause fixture: three spawns in one beforeAll: an `os serve` config boot, the `os build` of its artifact, and the artifact boot; slowest CI reading 36.70 s (run 40424746941) of the two scheduled CI runs after the oclif test manifest (PR #26256: 40392443783, 40424746941), x 1.25, rounded up
23

34
/**
45
* #22614 — the by-name read of a `defineView` container answers the same on an

0 commit comments

Comments
 (0)