Repository navigation
Guard required CI commands against failure-masking wrappers #951
Description
Found while re-reviewing #946; this gap predates the monitoring change.
The failure-masking check in tests/test_guard_wiring.py::test_no_gauntlet_short_circuits_on_failure checks the three documented local gauntlets. It does not inspect GitHub Actions run blocks. The existing CI wiring assertions accept a required test command followed by || true.
Reproduction: in an isolated archive of ee1770a, append || true to the validation job's Node test command and run tests/test_guard_wiring.py --no-cov -q. All 16 wiring tests still pass. The actual checkout and CI configuration were not mutated. The sanctioned Bandit/pip-audit scanner + severity-helper exception must remain supported.
Acceptance criteria:
- Reject failure-masking wrappers on required pytest, Node, Ruff and guard invocations in GitHub Actions run blocks.
- Cover multiline shell blocks and command chaining without treating comments or quoted prose as executable commands.
- Preserve the documented scanner + severity-helper exception only when the helper actually gates the scanner output.
- Include negative fixtures/mutations proving that a required test command wrapped with
|| truefails the check. - Wire any added guard into CI and the canonical local gauntlets, preserving their inventory checks.
Related: #549 covers scanner report completeness; #920 covers consolidating the local gauntlets. This issue covers failure masking in workflow commands.
Activity
cemililik commentedon Oct 16, 2026
Found while fixing this. Items 1–3 are addressed in #954.
The gauntlet check accepted a wrapped
|| truearound any command.tests/test_guard_wiring.py::test_no_gauntlet_short_circuits_on_failureonly checks that a gauntlet|| truesits inside a{ …; }group. Its failure message told contributors to "wrap it as '{ cmd || true; }'", so{ pytest tests/ || true; }passed. The new guard rejects that unless the wrapped command is bandit or pip-audit followed by its severity helper reading the same report. The wiring test's message now says so.Workflows could mask failures without
|| true. These forms all kept CI green:|| :and|| exit 0;| teewithoutpipefail, andset +e;continue-on-error;cmd && echo okfollowed by another line: bash's errexit ignores a failure inside an&&/||list.The guard also covers EXIT/ERR traps that can exit 0, functions or aliases that take over a required command, repository scripts run with
./x.shorsource, and composite actions.The standards contradicted each other on
continue-on-error.docs/standards/testing.mdtold contributors to replace any other|| truewith a step-levelcontinue-on-error.docs/standards/code-review.mdlistedcontinue-on-erroras an anti-pattern. Both now say: never on a step that runs a required command.Two partial copies of the gauntlet have drifted. They are in
docs/standards/code-review.mdand the review-pr skill. Not changed in Fix the wizard's Compliance step and field labels, guard CI against failure masking, correct monitoring copy #954; details are on Replace the ~31-command self-review chain triplicated in CLAUDE.md, AGENTS.md and CONTRIBUTING.md withtools/gauntlet.sh— deferred until a decision row approves it #920.Known limits are documented in the guard's docstring and in
testing.md. The guard reads shell text only, so it cannot see:PATH;curlorwget.test(ci): check the copied repository scans clean before a mutation
test(ci): match each gauntlet's document and finding on one line
refactor(ci): move the failure-masking guard's shell parser to its ow…
fix(ci): close gaps and false positives in the failure-masking guard
fix(ci): close the gaps the guard's new accepted shapes opened
fix(ci): read substitutions after a quoted apostrophe in array assign…
fix(ci): close the guard's remaining gaps in quoting, traps and exits
fix(ci): reject failure-masking wrappers around required CI commands
fix(ci): close the masking shapes the guard review found
fix(ci): close the masking shapes the second guard review found
test(ci): point the gauntlet `|| true` hint at the scanner exception