You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit b50f5cf
Browse filesBrowse the repository at this point in the historyBrowse files
test(spec,lint): cheaper slow-file sweeps and re-measured test budgets (#26089)
<!-- claude-projects-attribution: {"github_login":"zhuangjianguo"} -->
_Requested by **Steedos** · [project
thread](https://claude.ai/code/project/chan_018ETKaNwMZCNRYD7D8Q9Ubb?thread=cmsg_018ETKaNwMZCNRYD7D8Q9UbbJquRbbKg1smQPg49yRES4i)_
Part of #23180. This PR covers the packages/spec and packages/lint rows
of refresh PR #25960's per-file list that are still over 10 s or over
their declared budget (12 rows, listed below). #23180 stays open for the
other packages' rows.
Clause-②: no
Metadata-protocol: none
## Summary
Tests only. Each row is either made cheaper (three files), or given a
budget re-measured from #25960's window, or both. Nothing is deleted, no
vitest timeout is raised, and no file moves to a slower or skipped
project.
- `packages/lint/src/non-record-object-entry.test.ts`: the junk-value
sweeps ran every rule once per assertion. They now run each rule once
per distinct fixture and memoize the outcome (findings count, or the
thrown message), and every assertion reads that record. 348/348 tests,
2.11 s to 1.00 s locally.
- `packages/spec/src/ui/app.test.ts`: the compiler-API case built its
program with the ES5 default lib (DOM declarations) and every ambient
`@types` package. Neither is an input of `app.zod.ts`, so the program
now uses `lib.es2022.d.ts` and `types: []`. The guard still reads the
full member list off the resolved type. 1.49 s to 1.25 to 1.37 s
locally.
- `packages/spec/scripts/build-schemas-check-mode.manifest.test.ts`: the
#12588 block ran the generator twice on one fixture for two read-only
cases. It now runs it once per block (`beforeAll` plus a lazy memo, the
same shape as `openapi-self-consistency.test.ts`'s shared run). That is
one generator spawn (about 7 s) fewer per run. 136.3 s to 125.1 s
locally (shared box).
## Rows covered, with their source lines
Source: refresh workflow run 40275747937, job 121418010419 (the #25960
refresh). Line numbers are that job log's.
| Row | Log line | Slowest CI reading (run) | Declared before | Now |
|:--|--:|:--|--:|:--|
| spec `scripts/build-schemas-check-mode.retired-keys.test.ts` | 1339 |
241.05 s (40204927516) | 190 | budget 310 |
| spec `scripts/build-schemas-check-mode.manifest.test.ts` | 1340 |
226.64 s (40204927516) | 180 | one spawn fewer, budget 290 |
| spec `scripts/build-schemas-check-mode.anchor.test.ts` | 1346 | 144.97
s (40204927516) | 140 | budget 190 |
| spec `src/eager-entry-import.test.ts` | 1394 | 41.61 s (40266979145) |
30 | budget 60 |
| spec `scripts/upgrade-guide-docs.test.ts` | 1408 | 30.07 s
(40214410982) | 30 | budget 40 |
| lint `src/lazy-deps.test.ts` | 1442 | 13.91 s (40200279306) | none |
budget 20 |
| lint `src/runtime-lazy-deps.test.ts` | 1452 | 13.05 s (40200279306) |
none | budget 20 |
| spec `scripts/root-entry-type-nameability.pin.test.ts` | 1453 | 12.92
s (40214410982) | none | budget 20 |
| lint `src/non-record-object-entry.test.ts` | 1456 | 12.49 s
(40214410982) | none | cheaper, about half the local time |
| spec `src/ui/app.test.ts` | 1459 | 11.90 s (40204927516) | none |
cheaper, budget 20 |
| spec `scripts/pure-schema-construction.test.ts` | 1460 | 11.86 s
(40214410982) | none | budget 20 |
| spec `scripts/openapi-self-consistency.test.ts` | 1473 | 10.48 s
(40204927516) | none | budget 20 |
The window is the hourly runs 40200279306 to 40272142430. Over the
window's executed readings, the declared budgets were exceeded in:
retired-keys 6 of 10, manifest 6 of 10, anchor 1 of 10, eager-entry 7 of
8, upgrade-guide 1 of 8.
## Re-measure arithmetic
Every budget follows rulings 6181973363 item 2: the slowest CI reading
in the window, x 1.25, rounded up to the next 10 s. Each header line
names its reading and run, and each was parsed with `readTestBudget`
from `scripts/measure-test-shard-timings.mjs`.
- retired-keys: 241.05 x 1.25 = 301.3, so 310
- manifest: 226.64 x 1.25 = 283.3, so 290
- anchor: 144.97 x 1.25 = 181.2, so 190
- eager-entry-import: 41.61 x 1.25 = 52.0, so 60
- upgrade-guide-docs: 30.07 x 1.25 = 37.6, so 40
- lazy-deps: 13.91 x 1.25 = 17.4, so 20
- runtime-lazy-deps: 13.05 x 1.25 = 16.3, so 20
- root-entry-type-nameability.pin: 12.92 x 1.25 = 16.2, so 20
- ui/app: 11.90 x 1.25 = 14.9, so 20
- pure-schema-construction: 11.86 x 1.25 = 14.8, so 20
- openapi-self-consistency: 10.48 x 1.25 = 13.1, so 20
`non-record-object-entry.test.ts` gets no budget. Its local time halved,
which puts its CI reading at about 6 s by the window's CI-to-local ratio
(12.49 / 2.11). Per c91af8e, a later CI reading over 10 s would be the
measurement a declaration needs.
## Why the bscm, eager-entry and upgrade-guide floors stay
One `build-schemas.ts` spawn costs about 7.3 s on a profile: zod to JSON
Schema 3.14 s, module load and script 2.73 s, writes 0.97 s, GC 0.40 s,
child_process 0.24 s. Each bscm case is one spawn on its own
single-cause fixture. retired-keys and anchor have no two cases on an
identical fixture, so no run can be shared. In manifest, the #12588 pair
was the one identical-fixture pair inside a block. Sharing across blocks
(for example a #16431 control with another block's run) would couple
fixtures that each block seeds itself, so it is left out. Batching
causes or redesigning the producer is the option seat ruling 6176284384
declined.
eager-entry-import spawns one fresh tsx child per published entry by
design, and upgrade-guide-docs spawns the real producer once per mode or
usage error it pins (about 0.7 s each). Both are kept and re-declared.
## Measurements
Local runs use the CI shape: `CI=true pnpm exec vitest run --project P
--maxWorkers=1` with the agent environment variables unset. They ran on
a shared box, so ratios carry more weight than absolutes.
| File | Before | After |
|:--|--:|--:|
| lint non-record-object-entry | 2111 ms | 999 ms |
| lint lazy-deps | 2312 ms | 2305 ms |
| lint runtime-lazy-deps | 1952 ms | 2030 ms |
| spec ui/app | 1490 ms | 1250 to 1372 ms |
| spec manifest (bscm) | 136.3 s | 125.1 s |
| spec retired-keys (bscm) | 124.0 s | header only |
| spec anchor (bscm) | 121.6 s | header only |
| spec eager-entry-import | 12.3 s | 12.1 s |
| spec upgrade-guide-docs | 6.7 s | 7.3 s |
| spec pure-schema-construction | 3.2 to 4.2 s | 3.7 s |
| spec openapi-self-consistency | 3.4 s | 3.6 s |
| spec root-entry-type-nameability.pin | 3.14 s | 3.11 s |
## Mutation proof (non-record-object-entry)
The memo must not hide a rule regression. I mutated `recordsOf` in
`packages/lint/src/object-graph.ts` to `return v as AnyRec[]` with
`scripts/ablation-replace.mjs`, then ran both the new file and the BASE
copy against the same mutant. Both failed the identical set, 121 of 348.
The restore was proven: the blob equals HEAD's (693c48a) and `git
diff HEAD` was empty.
## Acceptance notes
- Header-only files (retired-keys, anchor, eager-entry, upgrade-guide,
pure-schema, openapi, root-entry pin, both lazy-deps files) change only
their budget comment line.
- The bscm files run in the vitest `repo` project, which CI's Test Core
owns. Locally I ran only the manifest file after the change, 18/18
green.
- #23180 remains open. Rows outside packages/spec and packages/lint stay
with it.
## Verification
All runs below are on head 1a44227 (this branch merged with main
bbf7061), unless noted.
- `pnpm --filter @objectstack/lint test`: 164 files, 7356 tests passed
(exit 0).
- The six changed spec `local` files, run together after the merge: 6
files, 169 tests passed (exit 0).
- Before the second merge, on ea742f5: `pnpm --filter
@objectstack/spec test` (723 files, 21902 tests passed, 1 todo), `pnpm
--filter @objectstack/spec typecheck`, `pnpm --filter @objectstack/lint
typecheck`. All exit 0.
- Derived gates: `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` gives 84 commands. All 84 exit 0, and
`--ran` prints: "84 derived famil(ies) accounted for, 84 run, 0
NOT-MEASURED". One note: `check:dts-closure` first exited 1. After the
merge, `@object-ui/plugin-editor` could not build because main's new
`monaco-editor` dependency was not installed yet. After `pnpm install
--frozen-lockfile` and a rebuild of the two named UI packages, it exited
0 ("231/231 declared declaration file(s) present").
- The bscm files belong to the vitest `repo` project, which CI's Test
Core runs. Locally, the manifest file ran after its change, 18/18
passed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01LpCAbWQuprShJmC2GJXTp7
---------
Co-authored-by: Claude <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: packages/lint/src/lazy-deps.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 20 — whole-package import: the in-process case imports the whole @objectstack/lint barrel cold and loads each lazy dep on its trigger, and the two built-dist probes each spawn node once to cold-load the dist and every lazy dep (typescript among them); slowest CI reading 13.91 s (run 40200279306, refresh PR #25960's window (hourly runs 40200279306 to 40272142430)) x 1.25, rounded up
2
3
//
3
4
// Boot-path contract: importing @objectstack/lint must NOT load its heavy,
4
5
// gate-only dependencies. The package sits on the kernel boot path, while each
Copy file name to clipboardExpand all lines: packages/lint/src/runtime-lazy-deps.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 20 — whole-package import: the in-process cases import the @objectstack/lint/runtime entry cold, which reaches the whole gate graph, and the built-dist probes each spawn node once to cold-load it; slowest CI reading 13.05 s (run 40200279306, refresh PR #25960's window (hourly runs 40200279306 to 40272142430)) x 1.25, rounded up
2
3
//
3
4
// The kernel boot-path contract for `@objectstack/lint/runtime` (#4463).
Copy file name to clipboardExpand all lines: packages/spec/scripts/build-schemas-check-mode.anchor.test.ts
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
-
// test-budget: 140 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause; measured 2026-10-23 on its first CI reading after #25372 split build-schemas-check-mode.test.ts (run 39963105499, Test Core, head dce4c6406): 104.13 s x 1.25, rounded up; one reading only, its wall tracks the run's load, so re-measure once the window has more
2
+
// test-budget: 190 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause, and one generator run is about 7 s of JSON Schema emission and spec module load before any assertion; re-measured 2026-10-26 over the 10 executed readings in refresh PR #25960's window (hourly runs 40200279306 to 40272142430), slowest 144.97 s (run 40204927516) x 1.25, rounded up
3
3
//
4
4
// The in-tree anchor: it carries the deletion gate offline (#5235), and only
Copy file name to clipboardExpand all lines: packages/spec/scripts/build-schemas-check-mode.manifest.test.ts
+13-4Lines changed: 13 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
-
// test-budget: 180 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause; measured 2026-10-23 on its first CI reading after #25372 split build-schemas-check-mode.test.ts (run 39963105499, Test Core, head dce4c6406): 140.73 s x 1.25, rounded up; one reading only, its wall tracks the run's load, so re-measure once the window has more
2
+
// test-budget: 290 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause, and one generator run is about 7 s of JSON Schema emission and spec module load before any assertion; re-measured 2026-10-26 over the 10 executed readings in refresh PR #25960's window (hourly runs 40200279306 to 40272142430), slowest 226.64 s (run 40204927516) x 1.25, rounded up
3
3
//
4
4
// The manifest ratchet and its neighbours: a check reports, it does not write (#4711);
5
5
// an export that never published must be declared (#16431); the output clean spares a
@@ -557,7 +557,7 @@ describe('build-schemas.ts — the output clean spares a sibling generator (#537
557
557
describe('build-schemas.ts — the bundle counts the definitions it carries (#12588)',()=>{
558
558
constOUT=()=>path.join(sandbox,'json-schema');
559
559
560
-
beforeEach(()=>{
560
+
beforeAll(()=>{
561
561
// A current, self-consistent tree, so the run exits 0 and these assertions
562
562
// are about the bundle rather than about some ratchet upstream of it.
563
563
seedManifest((s)=>s);
@@ -566,11 +566,20 @@ describe('build-schemas.ts — the bundle counts the definitions it carries (#12
566
566
seedSurfaceBase(tip,(k)=>k);
567
567
});
568
568
569
+
/**
570
+
* One generation for the block. Both cases read the same fixture and only
571
+
* READ what the run printed and wrote, so the first case to need it runs the
572
+
* generator and the other reads that run, the way
573
+
* `openapi-self-consistency.test.ts` shares its shipped run.
Copy file name to clipboardExpand all lines: packages/spec/scripts/build-schemas-check-mode.retired-keys.test.ts
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
-
// test-budget: 190 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause; measured 2026-10-23 on its first CI reading after #25372 split build-schemas-check-mode.test.ts (run 39963105499, Test Core, head dce4c6406): 147.5 s x 1.25, rounded up; one reading only, its wall tracks the run's load, so re-measure once the window has more
2
+
// test-budget: 310 — spawns the real producer once per single-cause fixture: every case runs the real schema generator (build-schemas.ts --check) in its own sandbox git tree, one fixture per cause, and one generator run is about 7 s of JSON Schema emission and spec module load before any assertion; re-measured 2026-10-26 over the 10 executed readings in refresh PR #25960's window (hourly runs 40200279306 to 40272142430), slowest 241.05 s (run 40204927516) x 1.25, rounded up
3
3
//
4
4
// RETIRED_KEYS_BY_MAJOR rows, judged by exact key: check (b) matches the retired key,
5
5
// not its leaf (#4659); a nested retirement row is judged, not ignored (#17969).
Copy file name to clipboardExpand all lines: packages/spec/scripts/openapi-self-consistency.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 20 — spawns the real producer once per single-cause fixture: the subprocess group runs the real build-openapi.ts once per distinct fixture, the shipped source and each mutation, and cases that share a fixture read its one run; slowest CI reading 10.48 s (run 40204927516, refresh PR #25960's window (hourly runs 40200279306 to 40272142430)) x 1.25, rounded up
2
3
//
3
4
// Pins the self-consistency gate on the generated OpenAPI document (#5168).
Copy file name to clipboardExpand all lines: packages/spec/scripts/pure-schema-construction.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 20 — spawns the real producer once per single-cause fixture: each build case runs the real tsup build (in-process, with the package's own main options and its esbuild plugin) once over the entry or fixture it pins, and the ./shared case bundles two consumers of that output; slowest CI reading 11.86 s (run 40214410982, refresh PR #25960's window (hourly runs 40200279306 to 40272142430)) x 1.25, rounded up
2
3
//
3
4
// The `pure-schema-construction` build plugin (`tsup.config.ts`) marks real
4
5
// calls only (#20686). Its earlier line-prefix rule also rewrote a marked name
Copy file name to clipboardExpand all lines: packages/spec/scripts/root-entry-type-nameability.pin.test.ts
+1Lines changed: 1 addition & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
+
// test-budget: 20 — compiles the package type graph once: the consumer program is one tsc run over the BUILT root entry's declarations, and the canary is one more tsc run over a one-line fixture; slowest CI reading 12.92 s (run 40214410982, refresh PR #25960's window (hourly runs 40200279306 to 40272142430)) x 1.25, rounded up
2
3
3
4
/**
4
5
* Root-entry nameability pin (#11350) — the consumer shape, verbatim.
Copy file name to clipboardExpand all lines: packages/spec/scripts/upgrade-guide-docs.test.ts
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2
-
// test-budget: 30 — spawns the real producer once per single-cause fixture: the last block spawns the real build-upgrade-guide.ts once per mode or usage error it pins (twice for the determinism pin), and the rest are in-process; slowest CI reading 19.7 s (hourly run 38798542648) x 1.25, rounded up
2
+
// test-budget: 40 — spawns the real producer once per single-cause fixture: the last block spawns the real build-upgrade-guide.ts once per mode or usage error it pins (twice for the determinism pin), and the rest are in-process; re-measured 2026-10-26 over the 8 executed readings in refresh PR #25960's window (hourly runs 40200279306 to 40272142430), slowest 30.07 s (run 40214410982) x 1.25, rounded up
3
3
//
4
4
// Pins the protocol upgrade guide's public address (#22449 B′, condition 2):
5
5
// one docs-site page per protocol major plus an index, generated at docs build
0 commit comments