Skip to content

fix(s003): the fips coverage step fails on a red suite - #134

Merged
HackTuah merged 2 commits into
mainfrom
fix/fips-coverage-step
Oct 20, 2026
Merged

HackTuah merged 2 commits into
mainfrom
fix/fips-coverage-step

Conversation

@HackTuah

Copy link
Copy Markdown
Member

The fips job's coverage step could pass with a red suite. It ran mix test --cover | grep -E 'Result:|\| *Total' under the runner's default shell, bash -e {0}, which has no pipefail, so the step's exit was grep's. On job 118381934625 (PR #133, head 45930df) the suite inside that step was 1388/1389 passed, and the step and the job both passed, with the failing test's name filtered out of the log.

Nothing red could merge this way, because the mix gate step before it fails on a red suite. What it cost was the evidence the step exists for: the FIPS-leg coverage line, and the name of whatever failed.

What changes

  • The step runs mix test --cover unpiped. The coverage threshold is 0 (mix.exs), so the suite alone decides the exit. The summary's Total line is still in the output, and a failure now prints its name.
  • test/workflows_test.exs gains a fourth property: no step pipes mix test or mix gate into another command unless it runs with pipefail (shell: bash, or set -o pipefail in the step). The population is every step of every workflow, read from the YAML.

Evidence

  • Red first: the new test on the unchanged workflow fails, naming the one offender: .github/workflows/gate.yml: job fips, step The FIPS tests by name, and this leg's coverage.
  • With the fix, test/workflows_test.exs and test/dco_range_test.exs (the other test that reads gate.yml) pass, 9 of 9. Format and Credo (strict) are clean.
  • The measurement is this PR's own fips job: the coverage step's log now ends with the suite's own result line.

…t pipefail passes on a red suite

Signed-off-by: Ayla Croft <aylacroft@proton.me>
…ite fails it

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 56f32c6 into main Oct 20, 2026
15 of 16 checks passed
@HackTuah
HackTuah deleted the fix/fips-coverage-step branch October 20, 2026 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant