feat: report-only trials and Security Hub ASFF export - #48
Conversation
Evaluation-guide step 1 told people to omit the severity threshold (or `|| true`), which still fails on medium+ findings and swallows scanner errors. Add REPORT_ONLY so artifacts publish without gating, while exit 2 still fails the build. The CodePipeline README listed Security Hub as not in v0.1.0. Emit trustabl.asff.json and optionally import it when SECURITY_HUB=true.
SECURITY_HUB=true now works on CodeCatalyst as well as CodePipeline, and ASFF types no longer claim these findings are CVEs. Co-authored-by: Cursor <cursoragent@cursor.com>
Match the Security Hub CLI example more closely and run the offline converter tests on every push so reviewers can see a green check. Co-authored-by: Cursor <cursoragent@cursor.com>
The converter looked for file/message/fix. The engine emits file_path, explanation, and suggested_fix. Prefer those so Security Hub import is not an empty title with a generic description on a real scan. Co-authored-by: Cursor <cursoragent@cursor.com>
Leave install fail-closed to trustabl#44 and repo-wide CI to the dedicated lint PRs. This change is REPORT_ONLY plus Security Hub ASFF only. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
@sairenchristianbuerano @trustabl Implements EVALUATION.md step 1 ( Lint is |
|
Thanks @sheel-PBM. Flagging that this is being read rather than sitting unlooked-at. We are reviewing all 52 open PRs together instead of one at a time. 31 of them edit Queued. This is one of the larger changes in the queue and touches several files, so it gets read after the single-purpose fixes land. The order we are working to: the test harness in #1 first, since twelve PRs depend on it and nothing is verifiable without it, then the fixes that close fail-open paths, then behaviour changes, then docs. A verdict on this one follows once its cluster is read. Apologies for the wait, and thanks for the contribution. |
Co-authored-by: Cursor <cursoragent@cursor.com>
2 participants
Summary
Implements two gaps the repo already documents:
docs/EVALUATION.mdstep 1 — “scan without gating first.” OmittingSEVERITY_THRESHOLDdoes not do that (the binary still exits 1 on medium+), and|| truealso swallows exit 2 (dead scanner).REPORT_ONLY=truepublishes artifacts without gating; scanner/I/O errors still fail.codepipeline/README.md— Security Hub import was marked not in v0.1.0. Every scan now writestrustabl.asff.json.SECURITY_HUB=trueimports it withaws securityhub batch-import-findings(fail-closed). ASFF mapping uses the engine’sScanResultfields (rule_id,file_path,explanation,suggested_fix).Review path:
scan/trustabl-scan.sh(REPORT_ONLY / SECURITY_HUB) andscan/to-asff.sh. The rest is wiring and docs.Distinct from the docs-only report-only PRs (#27, #36, #41). Does not take the jq-install change (#44).
Test plan
bash test/test-asff.sh(offline: ASFF shape, empty findings,--importfail-closed)trustabl.asff.jsonin artifactsREPORT_ONLY=true+ medium+ findings: build succeedsREPORT_ONLY=true+ scanner exit 2: build failsSECURITY_HUB=truewithout CLI/IAM: build failsSECURITY_HUB=truewith Hub enabled: findings in Security Hub