Skip to content

Add Pest mutation testing quality gate - #37

Merged
erseco merged 20 commits into
quality/backward-compatibilityfrom
quality/mutation-testing
Sep 19, 2026
Merged

erseco merged 20 commits into
quality/backward-compatibilityfrom
quality/mutation-testing

Conversation

@erseco

@erseco erseco commented Sep 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add mutation testing with Pest's native mutation engine
  • run mutation testing in a dedicated PHP 8.4 job without changing the parser's PHP 8.0 runtime requirement
  • establish a 50% minimum mutation-score gate from the measured project baseline
  • mutate only changed PHP source files in pull requests
  • skip mutation execution when a PR has no changed PHP source
  • run a full mutation pass across all covered source weekly
  • allow manual full mutation runs with workflow_dispatch
  • keep the existing 90% line-coverage gate as a separate quality signal
  • document the mutation-testing policy, measured baseline and contributor expectations

Stack

This PR is stacked on #36 and targets quality/backward-compatibility.

Pull-request strategy

Changed PHP source paths are calculated from Git and supplied through Pest's supported --path option:

MUTATION_PATHS="$(git diff --name-only --diff-filter=AMR "origin/$BASE_REF...HEAD" -- 'src/*.php' 'src/**/*.php' | paste -sd, -)"

vendor/bin/pest \
  --mutate \
  --parallel \
  --path="$MUTATION_PATHS" \
  --covered-only \
  --ignore-min-score-on-zero-mutations \
  --min=50

Scheduled/manual full runs use --everything --covered-only --min=50.

Measured baseline

The initial full mutation run produced:

  • mutation score: 53.66%
  • tested mutants: 1,586
  • untested mutants: 1,373
  • timed-out mutants: 4

The initial CI floor is therefore set to 50%, slightly below the measured baseline. This starts the gate green while preventing regressions and allows the threshold to be ratcheted upward as mutation gaps are addressed.

Validation

  • PR changed-source mutation check: passing
  • full mutation baseline measured
  • PHP 8.0–8.5 tests: passing
  • coverage gate: passing
  • Composer validation: passing
  • PHPCS: passing
  • PHPStan: passing

@codecov-commenter

codecov-commenter commented Sep 19, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.4%. Comparing base (a7d8e5b) to head (99d4cc4).

Additional details and impacted files

Impacted file tree graph

@@                       Coverage Diff                        @@
##             quality/backward-compatibility     #37   +/-   ##
================================================================
  Coverage                              92.4%   92.4%           
  Complexity                              801     801           
================================================================
  Files                                    30      30           
  Lines                                  2639    2639           
================================================================
  Hits                                   2441    2441           
  Misses                                  198     198           
Flag Coverage Δ
unit 92.4% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@erseco
erseco added this pull request to stack #38 September 19, 2026 21:24
@erseco
erseco merged commit 8026135 into main Sep 19, 2026
20 checks passed
@erseco
erseco deleted the quality/mutation-testing branch September 19, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants