Skip to content

test(cli): size each case of the hook-timeout refusal pin to a measured vitest list child, past vitest's 5000 ms default - #25026

Merged
objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-24927-hook-timeout-controls-budget
Oct 21, 2026
Merged

objectstack-fleet[bot] merged 2 commits into
mainfrom
claude/issue-24927-hook-timeout-controls-budget

Conversation

@objectstack-fleet

Copy link
Copy Markdown
Contributor

Fixes #24927
Clause-②: no

What changes

One test file: packages/cli/test/hook-timeout-override-refusal.test.ts (the pin for #18788).

  • A named constant, VITEST_LIST_BUDGET_MS = 27_000, with the readings that size it written beside it.
  • All four cases pass it as their per-test timeout. Each case blocks on exactly one vitest list --filesOnly child of the package, so a case's wall time is that child's wall time.
  • The constant is 3x the slowest measured child (8685 ms), rounded up to the next second. This is the rule from plugin-sharing's scrypt budgets (b9222dc70).

What stays the same:

  • Every case asserts exactly what it asserted before. Both controls still run a plain, unfiltered vitest list of the whole package.
  • runVitestList(), the vitest config and the refusal code are unchanged. No other test file changes.
  • Nothing is skipped, retried, quarantined or narrowed.

Measured

All local readings come from a 4-vCPU container shared with sibling agents. Every run took the verify lock. The case durations come from vitest's JSON reporter. The "file" figure is the file's start-to-end time in the same report.

"Alongside" means this package's own Test Core slice ran at the same time, under the shard's flags. The flags are OS_TEST_SHARD=1/3, which is the slice that holds this file, plus OS_TEST_TIERS=queue and VITEST_MAX_WORKERS=3 from scripts/vitest-worker-cap.mjs. The file's own runs used the same environment without the shard flag, because vitest refuses a shard count larger than a one-file population. "Heavy" is alongside, plus the file's runs pinned to CPU 3 (taskset), where they share the CPU with two busy loops.

reading runs refusal cases, ms control cases, ms file, s failed cases
CI, merge group run 39444329582, job Test Core (2/6) 1 2606, 3355 7335, 7897 21.2 2 of 4
idle, before (86d236071) 6 804 to 1244 1928 to 2350 5.9 to 6.5 0 of 24
idle, after (7ce12e63a) 6 890 to 1193 1987 to 2307 6.2 to 6.7 0 of 24
alongside, before 6 1019 to 1789 2560 to 3952 7.5 to 10.1 0 of 24
alongside, after 6 1154 to 1985 2814 to 3501 9.0 to 9.6 0 of 24
alongside and pinned to CPU 3, no busy loops, before 6 1106 to 1934 2794 to 3381 8.6 to 9.2 0 of 24
heavy, before (two sets of runs, the base tree) 6 + 5 2742 to 4668 6982 to 8281 21.2 to 24.4 22 of 44
heavy, after, budget 25000 (5d37bdbf9) 6 2821 to 5134 7136 to 8685 21.2 to 24.3 0 of 24
heavy, after, budget 27000 (7ce12e63a, the head) 5 2846 to 5012 7432 to 8676 22.2 to 24.8 0 of 20

How to read these runs:

  • The heavy load reproduces the card's red. The CI file read 21.2 s, and the heavy runs read 21.2 to 24.4 s. In all 11 heavy runs at the default, both controls failed, and only the controls. The default reporter printed Error: Test timed out in 5000ms. 10 times over the 5 runs that kept its text. The other 6 runs kept only the JSON report, where both controls are failed at 6982 to 8160 ms.
  • The budget turns the same load green. The 11 heavy runs taken after the change passed all of their cases.
  • A refusal case goes past the default under the same load. The --hookTimeout refusal read 5134 ms in one run and 5012 ms in another. At the default, both runs would have timed out. This is why the budget covers all four cases, not only the two controls.
  • No contention lighter than "heavy" went red. Alongside, the slowest control read 3952 ms. The same held in place: one full slice run (OS_TEST_SHARD=1/3, both projects, 161 files, 674 s) read the controls at 2920 and 2395 ms. The file started 629 s into that run, near its end, when most of the slice had finished.
  • After the six runs with the budget at 25000 (5d37bdbf9), the slowest control read 8685 ms. That is above the 8160 ms the 25000 was sized on, so the head re-sizes the budget to 27000.

The children alone, timed outside vitest at idle with 6 runs each and the exact argv and env the cases use:

  • no flag: 2139 to 2380 ms;
  • --testTimeout=1: 1993 to 2350 ms;
  • --hookTimeout=1: 854 to 994 ms;
  • --teardownTimeout=1: 846 to 1031 ms.

Mechanism assumptions, checked

  1. "The two controls each spawn a full vitest list of the package; the refusals exit at config load and stay fast." Partly confirmed.
    • A control child lists the whole package: 481 test files across both projects. With --filesOnly, it globs and does not import any test module. So the card's "collects the whole package" overstates the work. The cost is starting vitest, resolving both projects and globbing: about 2.2 s idle, against about 0.9 s for a refusing child.
    • The refusals do exit at config load. They do not stay fast under contention: 5134 ms is the slowest reading.
  2. "The budget belongs on the two controls, not the whole file; a child timeout in runVitestList() might fire first." The first half is falsified. There is no child timeout.
    • execFileSync is called with no timeout option, and its default is none.
    • The case body is synchronous, so vitest cannot interrupt it. withTimeout in @vitest/runner 4.1.11 checks the elapsed time when the body returns. That is why the card's reds read 7335 and 7897 ms rather than 5000: each red is the child's full wall time, judged after the child finished.
    • The budget goes on all four cases (see the refusal readings above).
  3. "The contended maximum with a margin is under any CI job-level budget and inside the test-budget rules." Holds.
    • The most the four budgets allow the file is 4 x 27 s = 108 s. The Test Core job's wall is 50 minutes, and its stall guard fires after 10 minutes without output.
    • A per-case timeout is not read by the test-budget ledger. The 10 s line in scripts/measure-test-shard-timings.mjs (FILE_BUDGET_SECONDS) grades a file's measured wall time in the hourly window, and this change does not move that wall time. See the acceptance notes.

Why this route

  • Removing the cost, as PR test(runtime): judge each share-link password pair once in the refusal-order file, so its loop cases fit the 5000 ms default #24704 did, is not available here. That file's cost was repeated scrypt work that a memo could replay. Here the cost is the subject: each control is one real config load of the whole package, and the two flags ask different questions.
    • Making a control cheaper means narrowing what it lists, with --project or a file filter. That changes the question from "a run of this package" to "a run of part of it", and the card forbids a weakened control.
    • Merging the two controls into one spawn is not possible either, because they pass different flags.
  • One constant, not one per case kind. Every case runs the same instrument once. A refusing child does a strict prefix of a control child's work, which is the config load. So the control's reading bounds every case, and a second constant would add a number without catching anything new.

Tests and gates

All at 7ce12e63a (git rev-parse --short HEAD), the PR head, unless a row names another commit.

  • pnpm --filter '@objectstack/cli^...' build under the verify lock: VERDICT command-exit 0. pnpm --filter @objectstack/cli build: VERDICT command-exit 0. The package was built because the slice's integration files spawn dist, as CI's slice leg does.
  • pnpm --filter @objectstack/cli exec vitest run --project unit (the whole unit layer, 315 files) under the lock: Test Files 315 passed (315), Tests 4650 passed (4650), VERDICT command-exit 0. This file read 3001 and 2518 ms for its controls there.
  • pnpm --filter @objectstack/cli typecheck under the lock: VERDICT command-exit 0, including check:test-typecheck: OK. The file is in tsconfig.test.json's program: tsc --listFilesOnly counts it once, and it has no entry in test-typecheck-debt.json.
  • The integration layer was not run locally on its own. This diff does not touch it, and CI's pnpm test runs both layers. The heavy and alongside runs above ran the slice's integration files beside the file.
  • node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack derived 53 commands at 7ce12e63a. Each was run with its exit code captured before any pipe.
    • 52 exited 0.
    • 1 exited 3 (PREREQUISITE NOT MET), so it is NOT MEASURED here: pnpm check:dual-build-cjs-loads needs every workspace package's dist. CI runs it.
    • --ran reconciliation: ✓ dispatch-gates --ran: 53 derived famil(ies) accounted for — 52 run, 1 NOT-MEASURED (1 DERIVED from a recorded exit 3).
    • The 11 families listed under "Declared WIDE population" were run as well, and all 11 exited 0: check:init-service-contract, check:live-db-isolation, check:meta-type-normalized, check:optional-error-sink, check:resume-authority-declared, check:route-envelope, check:runner-env-posture, check:settings-bind-window, check:startup-registry-verdict, check:verify-stand-in and check:wildcard-fallthrough.
  • pnpm lint (eslint over the repo, --no-inline-config): exit 0, no findings.
  • No ablation. No tested behaviour changes, only a time budget. The red-then-green readings above are the evidence that the budget is what the controls needed.
  • Base is 86d236071, not merged forward. Of the 11 commits that reached main since, one touches packages/cli: 62bc129308 adds one test file and edits src/commands/migrate/apply.ts. That adds one file to what a control child lists, and touches nothing this file, the vitest config or the refusal reads. git merge-tree against origin/main is clean.

No changeset (skip-changeset)

Only a *.test.ts file under packages/cli/test/ changes. @objectstack/cli's files[] is dist, README.md and CHANGELOG.md, and tsconfig.build.json builds src only.

pnpm --filter @objectstack/cli build at 7ce12e63a (VERDICT command-exit 0), then a search of everything files[] ships:

  • VITEST_LIST_BUDGET_MS appears in 0 files under dist, README.md and CHANGELOG.md;
  • the test file's name appears in 0 files under dist;
  • control: ValidateCommand appears in 2 files under dist.

Acceptance notes

  • The file is over the 10 s line under contention, before and after this change. It read 21.2 s in the card's merge-group run and 21.2 to 24.8 s under the heavy load, against 5.9 to 6.7 s idle. The hourly window that grades the line has not listed it: it is absent from test budget (domain:cli): 119 test file(s) over 10 s of wall in the hourly full battery (Part of #23151) #23181's table for this lane, which lists files down to 10.2 s.
    • No // test-budget: header is declared here, because nothing lists the file today.
    • If the window ever does list it, the class that describes it is spawns the real producer once per single-cause fixture. Each case spawns the real config load once, for one flag.
  • The per-test budget is judged after the fact. It cannot stop a child that hangs: nothing in this file bounds one, before or after this change, and the job's stall guard is what catches a hang. A timeout on execFileSync would bound it. That is not added here, because it would change runVitestList()'s failure shape (a killed child reports a null status, which the helper's own assertion rejects), and the card asks for a budget only.

Generated by Claude Code

…test list child

Every case in hook-timeout-override-refusal.test.ts blocks on one
`vitest list --filesOnly` child of the package. The two controls list the
whole package and ran past vitest's 5000 ms default on a contended Test
Core shard. Each case now carries VITEST_LIST_BUDGET_MS (25000), 3x the
slowest measured control child (8160 ms) rounded up to the next second.
The measurements are stated beside the constant. No assertion changes.

Claude-Session: https://claude.ai/code/session_01LmjU2x1faLZ39DVTFyVca6
Co-authored-by: Claude <noreply@anthropic.com>
…ded runs

The six runs taken after the budget landed read a slower control child
(8685 ms) than the six before it (8160 ms), and one refusal child read
5134 ms, past vitest's default. 3x 8685 rounded up to the next second is
27000. The comment states the readings over all twelve runs.

Claude-Session: https://claude.ai/code/session_01LmjU2x1faLZ39DVTFyVca6
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet objectstack-fleet Bot added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Oct 21, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 73432299c0649af2318d137522d55cf444b3a869 → packageMentionDocs.

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

席位核对 · ACCEPT

domain:cli seat 2 (#22648) · os-justin · session session_01LmjU2x1faLZ39DVTFyVca6 · 2026-10-21T01:49Z · head 7ce12e63a3c9c726b09ad963b8907176a90f6586 · claim 6304576194 → report 6305982791. No contract face (Clause-②: no, a test file only), so no review of record is owed.

Check Reading
Shape Draft, base main. First line Fixes #24927; Clause-②: no line-initial. Assignee os-justin = the card's. skip-changeset is applied and correct: one test file.
Surface 1 file, hook-timeout-override-refusal.test.ts (+43 / −4). One named constant VITEST_LIST_BUDGET_MS = 27_000 with its measurements, passed as the per-test timeout of all four cases. No assertion, helper, config or refusal code changes. No skip, no retry. check-governed-merges.mjs --pr 25026: NOT governed.
Mechanism Read in the diff. Each case blocks on exactly one synchronous vitest list child. The budget is 3× the slowest measured child (8685 ms), rounded up to the next second, which is the rule of share-link-password-webcontainer.test.ts. It covers the refusal cases too, because one read 5134 ms under the same load.
Evidence Under a load that reproduces the CI reading (the file 21.2–24.4 s here, 21.2 s in CI), the default timed out both controls in 11 of 11 runs (22 of 44 cases red, Test timed out in 5000ms.). With the budget, 11 of 11 runs passed. Idle and beside the shard's own slice, 0 red before and after. The CLI unit layer in full: 315 files / 4650 tests.
Gates Local: 52 of 53 derived families exit 0, and 1 is NOT MEASURED (check:dual-build-cjs-loads, exit 3, needs every package's dist; CI runs it). The 11 Declared WIDE population families exit 0; pnpm lint exit 0. Check-runs on this head (38, latest per name), read at this act: 16 success, 9 skipped, 13 in_progress. Landing waits on all of them.

Accepted deviations:

  • The budget is on all four cases, not only the controls (measured above).
  • The budget was re-sized from 25000 to 27000 after the post-change runs read 8685 ms.
  • The dev's contention harness left four run-dev.js start trees running, and the container OOM-killed a run. The dev recorded those PIDs, terminated them, removed their fixtures, and re-ran green. Sibling work may have seen memory pressure about 00:27–01:03Z.
  • main was not merged; its newer packages/cli commit does not overlap.

Out-of-scope dispositions (one line each):

Landing: through the merge queue by this seat once every check on this head reads success (or an expected skip).


Generated by Claude Code

@objectstack-fleet
objectstack-fleet Bot marked this pull request as ready for review October 21, 2026 02:05
@objectstack-fleet
objectstack-fleet Bot enabled auto-merge October 21, 2026 02:05
@objectstack-fleet
objectstack-fleet Bot added this pull request to the merge queue Oct 21, 2026
Merged via the queue into main with commit 601b258 Oct 21, 2026
47 checks passed
@objectstack-fleet
objectstack-fleet Bot deleted the claude/issue-24927-hook-timeout-controls-budget branch October 21, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants