Skip to content

test(spec): four ruled-class test files declare their measured CI budget - #23613

Merged
objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-23180-test-budget-declarations
Oct 15, 2026
Merged

objectstack-fleet[bot] merged 1 commit into
mainfrom
claude/issue-23180-test-budget-declarations

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Part of #23180
Clause-②: no

This PR adds only the budget declarations that ruling record 6181973363 (letter B, item 5) assigns to the spec lane. Each of four test files gets one header line, // test-budget: SECONDS — CLASS: REASON. Since PR #23595, scripts/measure-test-shard-timings.mjs --diff reads that line and lists the file only when a window reading goes above the declared budget. The other rows on #23180 stay listed for later rounds.

Each diff is one inserted line at line 2, right below the copyright line. No other line changes: no timeout, no deletion, no project move.

The readings

Source: the hourly schedule runs of ci.yml, the same population the refresh lane reads. Only runs that executed @objectstack/spec count (turbo cache miss). Replayed rows are excluded, as the refresh excludes them.

Window: every such run after PR #23535 (the liveness read memo, 51b9990f47) landed. All three runs contain 51b9990f47: the compare API answers ahead with behind_by 0 for each. Each value is the vitest module line for the file, in seconds, from the shard that ran spec (Test Core (1/6) in all three runs). Each value also matches the run's merged timing table where that table lists the file.

run (UTC) head build-schemas-check-mode (repo) check-liveness (local) alias-integrity (local) eager-entry-import (local) compliance-families-retirement (repo)
38636949152 10-14 21:01 5fd2c76d6c 592.8 269.4 34.4 15.7 5.1
38651452634 10-14 23:01 8226755ee9 744.6 316.5 34.9 19.6 6.6
38657693085 10-15 00:02 b57228913a 749.9 320.4 26.0 19.9 7.7
slowest x 1.25, rounded up to the next 10 s 937.3 → 940 400.5 → 410 43.6 → 50 24.8 → 30 under 10 s, not declared
card reading (old window) 902.1 493.4 40.8 25.7 14.6

Replayed runs, for the record. The five other schedule runs in the same span (22:01, 01:12, 02:01, 03:01, 04:01) replayed @objectstack/spec from the cache. The refresh does not count them. Their replayed lines come from whichever run filled the cache, and every one is under the budget declared here:

  • build-schemas-check-mode: 626.95 to 731.05 s.
  • check-liveness: 239.63 to 320.70 s.
  • eager-entry-import: 19.3 s (03:01).
  • compliance-families-retirement: 3.0 s (03:01) and 6.1 s (04:01).

The declarations

file budget class
packages/spec/scripts/build-schemas-check-mode.test.ts 940 spawns the real producer once per single-cause fixture
packages/spec/scripts/liveness/check-liveness.test.ts 410 spawns the real producer once per single-cause fixture
packages/spec/src/shared/alias-integrity.test.ts 50 whole-package import
packages/spec/src/eager-entry-import.test.ts 30 whole-package import

Each REASON states what the file spawns or imports, and the CI reading and run its number comes from. That gives a later re-measure PR the baseline it has to state (ruling item 2).

The two candidates this seat was asked to decide:

  • eager-entry-import.test.ts: declared. Its CI readings are 15.7 to 19.9 s, over the 10 s line. Every published entry of the package (19 module entries) is imported first in its own fresh tsx child process, plus two unflagged controls. Its subject is the package's whole entry surface, so it takes the whole-package import class. Ruling item 5 names it once measured.
  • compliance-families-retirement.test.ts: not declared. Its CI readings are 5.1 to 7.7 s, all under the 10 s line, so a declaration would lift nothing. The reader refuses a budget of 10 s or less anyway. Its import is the system barrel, one domain rather than the whole package, so it would not be literally whole-package import either. The card's 14.6 s came from the old window, and the card's own pin (the next refresh list) decides that row.

PM mechanism assumptions, measured

  1. The old-window readings cannot be used: holds for check-liveness, partly for the others.
  2. "Can the script read Actions logs here": false in this container.
    • Every artifact download redirects to productionresultssa*.blob.core.windows.net, and so does the REST job-log endpoint. The egress proxy refuses that host (CONNECT 403). The artifacts are test-core-timings-N-of-6 and the 14-day test-core-timing-table.
    • So measure-test-shard-timings.mjs --diff --captures cannot be fed from here.
    • The readings above were taken through the MCP job-log read, which returns at most the last 5,000 lines of a job:
      • the Test Core aggregator job echoes the merged top-20 table, and says whether spec was measured or replayed;
      • the spec shard's tail carries every spec file line.
  3. readTestBudget(text, label) is exported at scripts/measure-test-shard-timings.mjs:1264: holds (at 8d05456cb).

Tests

All local results are at ccb64dc967, the PR head.

  • The reader on every changed file. I ran PR feat(scripts): a test file of either ruled class is listed over its own declared budget, not the 10 s line #23595's one-liner on each file; each exited 0 and printed 940, 410, 50 and 30. I then ran the refresh's own tree resolution, declaredBudgetInTree('@objectstack/spec', FILE) with the vitest-printed paths. It gave 940, 410, 50 and 30, and null for compliance-families-retirement.test.ts.
  • Negative control: a copy of the alias-integrity text with the class misspelled whole-packge import is refused at alias-typo:2, not skipped.
  • Single-file runs (under os-verify-lock.sh, --project local --maxWorkers=2, default reporter), all VERDICT command-exit 0:
    • alias-integrity.test.ts 20/20 (18.1 s);
    • eager-entry-import.test.ts 22/22 (15.3 s);
    • check-liveness.test.ts 71/71 (82.4 s).
  • build-schemas-check-mode.test.ts was not run locally: about 900 s on this box, for a comment-only change. CI runs it in the repo project.
  • No ablation: the diff changes no tested behaviour. It is header comments only.

Gates

  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 77 commands at ccb64dc967: 4 paths, +4 / -0. All 77 ran in sequence, and each exit was recorded before any pipe.
  • 74 exited 0.
  • pnpm --filter @objectstack/lint run check:doc-formula-expressions first exited 3 (PREREQUISITE NOT MET, formula and lint not built). After turbo run build --filter=@objectstack/formula --filter=@objectstack/lint it re-ran and exited 0.
  • NOT MEASURED: pnpm check:dual-build-cjs-loads and pnpm check:lean-entry-closure both exit 3 (PREREQUISITE NOT MET). They load other packages' built dist/ (hono, account, objectql) and cannot read a test-file comment. CI runs both.
  • --ran: ✓ dispatch-gates --ran: 77 derived famil(ies) accounted for — 75 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3).
  • Also run, both exit 0: pnpm check:dev-prereqs, and eslint --no-inline-config on the 4 files.
  • Every family CI runs is read from this PR's check-runs.

Publish

Nothing publishes. A packed @objectstack/spec (18.0.0-next.1, npm pack --dry-run) lists 2,092 files: 0 *.test.ts, 0 under scripts/, and none of the four files. The positive control, src/data/field.zod.ts, is present. The PR therefore carries skip-changeset, not a changeset.

Acceptance notes (not filed)


Generated by Claude Code

Each file carries one `// test-budget: SECONDS — CLASS: REASON` header
line (ruling record 6181973363, letter B): the slowest CI reading of the
three hourly full-battery runs that executed @objectstack/spec after the
liveness read memo landed, x 1.25, rounded up to the next 10 s.

- scripts/build-schemas-check-mode.test.ts: 940 s (749.9 s, run 38657693085)
- scripts/liveness/check-liveness.test.ts: 410 s (320.4 s, run 38657693085)
- src/shared/alias-integrity.test.ts: 50 s (34.9 s, run 38651452634)
- src/eager-entry-import.test.ts: 30 s (19.9 s, run 38657693085)

No other line changes.

Claude-Session: https://claude.ai/code/session_01LFqdRsuM5nkqoJdHd5ztVk
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 15, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 8d05456cbe03667ea2352c9a99e67d4a862a8294 → packageMentionDocs.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 — ACCEPT · domain:spec seat 2 (#18549) · session session_01LFqdRsuM5nkqoJdHd5ztVk · 2026-10-15T05:04Z

This is the seat review of record for #23180 (claim 6184342303, ruling 6181973363 item 5 on #23534) on head ccb64dc967ae82c638c666f7684e47568e0f8272. No contract face is touched: four test-file header comments, no published schema, no governed path, Clause-②: no. So no second reviewer is owed (contract-review.md: CI and the seat's own read).

  1. Form.
    • Draft to main. The first line is Part of #23180, with no closing keyword, and Clause-②: no sits at the start of a line.
    • 4 files, +4/−0. Not governed (0 of 4 paths). skip-changeset is right: npm pack --dry-run of @objectstack/spec ships no *.test.ts.
  2. Against the ruling. Each line is one header comment at line 2, // test-budget: SECONDS — CLASS: REASON, with the class spelled verbatim as one of the two ruled classes. The arithmetic checks out: 749.9 × 1.25 = 937.4 → 940; 320.4 × 1.25 = 400.5 → 410; 19.9 × 1.25 = 24.9 → 30; 34.9 × 1.25 = 43.6 → 50. Each value is a multiple of 10 above 10.
    • The seat's decision on the two candidates: eager-entry-import.test.ts read 15.7–19.9 s, so it is declared. compliance-families-retirement.test.ts read 5.1–7.7 s, so it is left undeclared.
  3. Gates. Locally, 75 of the 77 derived families ran, and 2 are NOT MEASURED for missing dists. readTestBudget answers 940, 410, 50 and 30, and a misspelt class is refused (negative control). CI is converging at this read, and landing waits on every check.
  4. Accepted deviations.
    • PM mechanism assumption 1 was partly wrong: only check-liveness was sped up, and the others simply read lower.
    • The budgets rest on the three executed hourly runs after PR perf(spec): check:liveness reads each cited file once per run #23535. A later higher reading lists the file again, and the number rises only by a re-measuring PR (ruling item 2).
    • build-schemas-check-mode.test.ts was not run locally (about 900 s); CI runs it.
  5. Out-of-scope findings. compliance-families-retirement.test.ts stays on this card's list for the next refresh. It is in the Acceptance notes, with test budget (domain:spec): 26 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23180 as the carrier.
  6. Landing terms. When every check is green, the seat marks the PR ready and enables auto-merge. On merge, test budget (domain:spec): 26 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23180 goes back to pm:queue for its remaining rows.

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 15, 2026 05:43
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 15, 2026 05:43
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 15, 2026
Merged via the queue into main with commit af25552 Oct 15, 2026
46 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-23180-test-budget-declarations branch October 15, 2026 06:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/xs skip-changeset PR has no user-facing published change; bypasses the changeset gate tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants