Skip to content

feat(cli): warn when --fail-on gate can't see any vulnerability data#54

Open
dmchaledev wants to merge 1 commit into
mainfrom
claude/magical-ptolemy-l394cs
Open

feat(cli): warn when --fail-on gate can't see any vulnerability data#54
dmchaledev wants to merge 1 commit into
mainfrom
claude/magical-ptolemy-l394cs

Conversation

@dmchaledev

Copy link
Copy Markdown
Contributor

Problem

--fail-on is the headline CI/CD gate, but it can only evaluate vulnerabilities that are embedded in the SBOMs being compareddiff() derives newCVEs from each SBOM's vulnerabilities list (src/diff.ts:41-44), and gateFailures() filters that list (src/cli.ts).

Most real-world SBOMs carry no such data:

  • SPDX 2.x has no vulnerability field at all — parseSPDX() hardcodes vulnerabilities: [] (src/parser.ts:79).
  • The default output of common CycloneDX generators omits the vulnerabilities array; vulnerabilities are usually attached later by a separate scan/VEX step.

When a gate is armed against such inputs, newCVEs is always empty, so the gate always passes and the process exits 0. In CI this reads as "no new CVEs" when the truth is "CVEs were never checked" — a silent fail-open in a security tool.

Fix

Add gateWarning(oldSBOM, newSBOM, failOn): when a gate is armed (failOn !== 'none') but neither SBOM carries vulnerability data, main() prints a warning to stderr explaining why the gate can't fire and how to enable it. The report still goes to stdout and the exit code is unchanged, so behavior is fully preserved for callers whose SBOMs do embed vulnerabilities — and for the default --fail-on none.

Example (gate armed, no vuln data — note the report and exit code are untouched; only the stderr warning is new):

$ sbom-diff old.json new.json --fail-on high
SBOM Diff Report
=================
...
Warning: --fail-on "high" is set, but neither SBOM contains vulnerability data,
so the CVE gate has nothing to evaluate and will pass. Most SBOMs do not embed
vulnerabilities (all SPDX 2.x, and the default output of common CycloneDX
generators); attach a scan/VEX step that emits a CycloneDX 1.4+ "vulnerabilities"
list to enable CVE gating.
$ echo $?
0

Changes

  • src/cli.ts — new exported gateWarning(), wired into main() (stderr only).
  • src/__tests__/cli.test.ts — 6 new unit tests covering: gate off, armed with no data (any + severity policies), old-side data present, new-side data present, and a missing vulnerabilities field.
  • README.md — note under the --fail-on section documenting the embedded-vulnerability precondition and the warning.
  • CHANGELOG.md — Unreleased/Added entry.

Verification

npx tsc --noEmit, npm run lint, and npm test all pass (44 tests, +6). Manually verified end-to-end: warning fires on stderr for vuln-less inputs, stays silent when the gate is off or when either SBOM embeds a CVE.

Scope

Deliberately narrow and additive — it does not change any exit code, gate semantics, or the report on stdout. It is independent of the in-flight purl-keying and VEX-suppression PRs; no overlapping files beyond the shared CLI test file.


Generated by Claude Code

The --fail-on gate only evaluates vulnerabilities embedded in the compared
SBOMs (diff() derives newCVEs from each SBOM's `vulnerabilities` list). Most
SBOMs carry none: SPDX 2.x has no vulnerability field, and the default output
of common CycloneDX generators omits one. In that case the gate has nothing to
evaluate and always passes — a silent fail-open that reads as "no new CVEs" in
CI when the truth is "CVEs were never checked".

Add gateWarning(oldSBOM, newSBOM, failOn): when a gate is armed (failOn !==
'none') but neither SBOM carries vulnerability data, main() prints a warning to
stderr explaining why the gate can't fire and how to enable it. The report
still goes to stdout and the exit code is unchanged, so nothing breaks for
callers whose SBOMs do embed vulnerabilities.

Covered by six new unit tests; README --fail-on section and CHANGELOG updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0142u9rtNkQzJmMuxzjuDbZV
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants