Skip to content

Speed up macOS tests with bounded release-suite concurrency - #37

Closed
dblock wants to merge 1 commit into
masterfrom
dblock/faster-release-tests
Closed

dblock wants to merge 1 commit into
masterfrom
dblock/faster-release-tests

Conversation

@dblock

@dblock dblock commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

Run up to two independent cases concurrently inside the release, recovery, lifecycle, self-release, semantic, and workflow suites. This overlaps real Git/npm work even when small macOS runners have only two test-file workers, without increasing the file-worker count or introducing unbounded case concurrency.

Register fixture-owned hook failures, conflicting tags, registry errors, publication failures, tampering, and invalid manifests as individually named cases instead of serial variants inside one case. Checks and recovery retries sharing a fixture remain sequential.

Tests that mutate process-wide environment variables remain serial. Every case keeps its own fixture, remotes, artifacts, and cleanup. Production code, real npm tests/lint, archive installation, failure/recovery checks, and the fresh-cache self-release build are unchanged.

Measurements

The completed macOS CI run 43363800770 spent 105.3 seconds in the test runner, versus 42.7 seconds for its Linux Node 22 lane. Inspection showed long serial chains of independent release cases within the large test files.

On the same local macOS ARM64 host, Node 26.13.0/npm 12.2.2, with each revision limited to two test-file workers:

Revision Tests Runner Time
Original serial suites 101 passed 54.9s
Initial four-suite concurrency change 101 passed 44.1s
Additional scenario scheduling and six bounded suites 112 passed 44.6s

The initial four-suite change reduced local wall time by 19.6%. The additional scheduling changes have not demonstrated another end-to-end gain in this local comparison: 44.6s versus 44.1s.

The initial four-suite version also completed hosted macOS CI in 84.9s, versus 120.5s on its direct parent, a 29.5% reduction. That hosted measurement predates the additional scheduling changes.

Validation

  • npm test with normal developer file parallelism: 112 passed, six suites, no skips.
  • Controlled two-worker run of the final content: 112 passed, six suites, no skips, 44.6s runner time.
  • npm run lint: runtime/contract types, ESLint, Markdown, spelling, and workflow names passed.
  • AST comparisons confirm the original wrapped cases and split scenario inputs, assertions, and ordered recovery bodies are preserved. The increase from 101 to 112 registered tests comes from naming existing variants separately, not adding coverage.

AI-authored by GitHub Copilot on behalf of @dblock.

Local Check Claims

Approved environment exception for this PR: the maintainer explicitly accepted the macOS ARM64 / Node 26 claim using the actual tested versions below. The observed hosted runner uses macOS 26.7.2, Node 26.13.1, and npm 11.20.0; this claim does not assert that those exact versions were reproduced. Linux, Windows, lint, and package checks remain unclaimed.

This claim covers the complete successful npm test run on the amended commit, with 112 tests passing. The Linux coverage/Coveralls lane, default-branch pushes, and publishing workflows remain independent of claims.

macOS ARM64 / Node 26 - passed (approved environment exception)
  • Ran: npm test
  • Environment: macOS 26.6.2 (25G83), ARM64, Node 26.13.0, npm 12.2.2; maintainer-approved version exception for this PR
  • Workflow: .github/workflows/test.yml
  • Check: test-macos-latest-ARM64-node-26
  • Commit: 0b4e68bab1aec9725227e1d9c3793f95e29fb13d
  • Content SHA-256: 7540d822e026a4e72e7f930262e50f2cdd0bdd02757bbb65fedfe28e59f178e6

@dblock
dblock force-pushed the dblock/faster-release-tests branch from c536218 to 1fba08d Compare November 13, 2026 04:31
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dblock
dblock force-pushed the dblock/faster-release-tests branch from 1fba08d to 0b4e68b Compare November 13, 2026 04:45
@dblock dblock closed this Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant