ci: lint the scanner and validate the CI configs - #47
Conversation
4fc286e to
086cc7e
Compare
|
Thanks @joeymussalli. Flagging that this is being read rather than sitting unlooked-at. We are reviewing all 52 open PRs together instead of one at a time. 31 of them edit Reading against #22, which also adds linting. The order we are working to: the test harness in #1 first, since twelve PRs depend on it and nothing is verifiable without it, then the fixes that close fail-open paths, then behaviour changes, then docs. A verdict on this one follows once its cluster is read. Apologies for the wait, and thanks for the contribution. |
sairenchristianbuerano
left a comment
There was a problem hiding this comment.
Thanks @joeymussalli — the change looks good, but it needs a rebase.
Several PRs touching scan/trustabl-scan.sh have landed on main since you
opened this, so it no longer merges cleanly. Could you rebase and resolve?
bash test/run-tests.sh will confirm it — 19 passing on main right now.
Ping me when it's up.
This repo has had no continuous integration of any kind. The entire
product is one bash script, scan/trustabl-scan.sh, plus the two config
files that invoke it: the CodeBuild buildspec and the CodeCatalyst
workflow. Nothing checks any of them. A stray typo in the script, or a
YAML edit that drops a key one of the integrations reads, ships
unnoticed and only surfaces in a customer's pipeline.
This adds a single GitHub Actions job that validates everything that can
be validated without running a scan. It downloads nothing, needs no
credentials, and holds only `contents: read`.
- `bash -n` on scan/trustabl-scan.sh, catching syntax errors.
- shellcheck on the same file, gated at `error` severity.
- shellcheck again at `warning` and above, reported but not enforced.
- YAML parse of codepipeline/buildspec.yml and
codecatalyst/workflows/trustabl.yaml, plus assertions on the keys
the integrations actually consume: the buildspec's
phases.build.commands and artifacts.files, the workflow's
Actions.Trustabl_Scan.Configuration.Steps and Outputs.Artifacts, and
that both still invoke scan/trustabl-scan.sh by that path. A file
that parses but has lost artifacts.files is still broken, so parsing
alone would not be worth much.
Every command was run locally against this tree before committing:
bash -n exits 0; shellcheck 0.10.0 reports zero findings at `error` and
exactly two at `warning`, both SC2034 for the unused SCAN_START and
SCAN_END; the structural validator passes all seventeen assertions, and
fails as intended when artifacts.files or the script path is removed
from a scratch copy. The workflow file itself was parsed with PyYAML and
its step structure inspected.
The severity gate is set at `error` deliberately, so this workflow is
green on main the day it lands rather than arriving red with a backlog
attached; the warning-level findings are still printed on every run via
a continue-on-error step, so they stay visible. A sibling branch,
chore/remove-dead-code, removes those two unused variables and takes the
file to zero at `warning` level. Once it lands, the gate can be tightened
to --severity=warning; nothing here depends on that happening.
086cc7e to
63a097e
Compare
This repo has had no continuous integration of any kind. The entire product is one bash script, scan/trustabl-scan.sh, plus the two config files that invoke it: the CodeBuild buildspec and the CodeCatalyst workflow. Nothing checks any of them. A stray typo in the script, or a YAML edit that drops a key one of the integrations reads, ships unnoticed and only surfaces in a customer's pipeline.
This adds a single GitHub Actions job that validates everything that can be validated without running a scan. It downloads nothing, needs no credentials, and holds only
contents: read.bash -non scan/trustabl-scan.sh, catching syntax errors.errorseverity.warningand above, reported but not enforced.Every command was run locally against this tree before committing: bash -n exits 0; shellcheck 0.10.0 reports zero findings at
errorand exactly two atwarning, both SC2034 for the unused SCAN_START and SCAN_END; the structural validator passes all seventeen assertions, and fails as intended when artifacts.files or the script path is removed from a scratch copy. The workflow file itself was parsed with PyYAML and its step structure inspected.The severity gate is set at
errordeliberately, so this workflow is green on main the day it lands rather than arriving red with a backlog attached; the warning-level findings are still printed on every run via a continue-on-error step, so they stay visible. A sibling branch, chore/remove-dead-code, removes those two unused variables and takes the file to zero atwarninglevel. Once it lands, the gate can be tightened to --severity=warning; nothing here depends on that happening.