Repository navigation
fix(s003): the fips coverage step fails on a red suite - #134
Merged
Merged
Conversation
…t pipefail passes on a red suite Signed-off-by: Ayla Croft <aylacroft@proton.me>
…ite fails it Signed-off-by: Ayla Croft <aylacroft@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
fipsjob's coverage step could pass with a red suite. It ranmix test --cover | grep -E 'Result:|\| *Total'under the runner's default shell,bash -e {0}, which has nopipefail, so the step's exit wasgrep's. On job 118381934625 (PR #133, head45930df) the suite inside that step was1388/1389 passed, and the step and the job both passed, with the failing test's name filtered out of the log.Nothing red could merge this way, because the
mix gatestep before it fails on a red suite. What it cost was the evidence the step exists for: the FIPS-leg coverage line, and the name of whatever failed.What changes
mix test --coverunpiped. The coverage threshold is 0 (mix.exs), so the suite alone decides the exit. The summary'sTotalline is still in the output, and a failure now prints its name.test/workflows_test.exsgains a fourth property: no step pipesmix testormix gateinto another command unless it runs withpipefail(shell: bash, orset -o pipefailin the step). The population is every step of every workflow, read from the YAML.Evidence
.github/workflows/gate.yml: job fips, step The FIPS tests by name, and this leg's coverage.test/workflows_test.exsandtest/dco_range_test.exs(the other test that readsgate.yml) pass, 9 of 9. Format and Credo (strict) are clean.fipsjob: the coverage step's log now ends with the suite's own result line.