Skip to content

Run full and integration specs in parallel - #55

Merged
dblock merged 1 commit into
masterfrom
dblock/spec-performance
Nov 11, 2026
Merged

dblock merged 1 commit into
masterfrom
dblock/spec-performance

Conversation

@dblock

@dblock dblock commented Nov 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • Run the full suite and installed-consumer integration task with development-only parallel_tests. Keep spec:unit unchanged and add spec:serial for serial debugging.
  • Separate workflow and composite integration files around the existing shared scenarios so the two interfaces can run concurrently. Preserve all fresh-process release phases, package builds, real Git operations, credential checks, and recovery assertions.
  • Give coverage workers distinct identities, discard stale resultsets before a run, and generate HTML/LCOV reports once after every worker exits, including failed runs. Preserve SPEC, SPEC_OPTS, and configurable worker counts.

Measured performance

Same machine, Ruby 4.0.9, seed 12345, and Open3 instrumentation; the parallel run used 12 workers.

These measurements used shared base 46f4573, before the subsequent rebase onto merged #53. The final rebased branch was validated separately against current master.

Metric Before After
Full-suite wall time 98.12 s 36.37 s
Examples 328 328
Covered lines 1803 / 1821 1803 / 1821

62.9% less wall time, or 2.70x faster. Every covered and uncovered source line is identical, not just the coverage percentage.

Observed Open3 launches inside spec workers: Bash 119 -> 119, Ruby/Rake 78 -> 78, other Ruby 40 -> 40, and Git 1062 -> 1059. These counts exclude descendant processes inside release phases and the parallel runner's own worker launches.

This trades concurrent CPU use for wall time rather than removing scenarios. Set PARALLEL_TEST_PROCESSORS=2 to limit workers, or use bundle exec rake spec:serial.

Validation

  • After rebasing onto Add reusable Markdown section updates #53: full default task passes with 342 examples, 1854 / 1872 covered lines, and 148 RuboCop files without offenses; gem construction also passes.
  • Before that rebase: all 17 integration examples pass against the installed archive with two workers.
  • Deliberately failing worker propagates failure; unmatched file selection is rejected; serial fallback executes the selected example.

Created by an AI assistant on behalf of @dblock.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@dblock
dblock force-pushed the dblock/spec-performance branch from 09570d1 to ff3a616 Compare November 11, 2026 18:44
@dblock
dblock merged commit f370dae into master Nov 11, 2026
7 checks passed
@dblock
dblock deleted the dblock/spec-performance branch November 11, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant