From 29e96a94fff2eacc504e68926ba0f1c4c05ae7b6 Mon Sep 17 00:00:00 2001 From: stacknil Date: Sat, 2 May 2026 11:42:45 +0800 Subject: [PATCH] Add summary-json CI cookbook --- tools/sbom-diff-and-risk/README.md | 4 +- .../sbom-diff-and-risk/docs/report-schema.md | 2 +- .../docs/reviewer-evidence-pack.md | 2 + .../docs/summary-json-ci-cookbook.md | 76 +++++++++++++++++++ 4 files changed, 82 insertions(+), 2 deletions(-) create mode 100644 tools/sbom-diff-and-risk/docs/summary-json-ci-cookbook.md diff --git a/tools/sbom-diff-and-risk/README.md b/tools/sbom-diff-and-risk/README.md index e7c2654..dfc3994 100644 --- a/tools/sbom-diff-and-risk/README.md +++ b/tools/sbom-diff-and-risk/README.md @@ -10,7 +10,7 @@ It uses conservative heuristics for change intelligence. By default it does not This project has two different provenance stories: -For a concise reviewer-facing overview, start with [docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review evidence and verification commands, use [docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For machine-readable JSON output shape, see [docs/report-schema.md](docs/report-schema.md). +For a concise reviewer-facing overview, start with [docs/reviewer-brief.md](docs/reviewer-brief.md). For reproducible review evidence and verification commands, use [docs/reviewer-evidence-pack.md](docs/reviewer-evidence-pack.md). For machine-readable JSON output shape, see [docs/report-schema.md](docs/report-schema.md). For CI consumption of summary-only output, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md). 1. If you want to verify `sbom-diff-and-risk` itself, start with [docs/verification.md](docs/verification.md). 2. If you want to use `sbom-diff-and-risk` to analyze third-party dependency provenance, start with [Dependency provenance analysis](#dependency-provenance-analysis-opt-in) and [Dependency provenance reporting](#dependency-provenance-reporting). @@ -179,6 +179,8 @@ Offline mode remains the default. No network access occurs unless `--enrich-pypi The checked-in [examples/sample-summary.json](examples/sample-summary.json) artifact is generated from the bundled CycloneDX example with `--summary-json outputs/summary.json` and matches the `summary` object in [examples/sample-report.json](examples/sample-report.json). +For CI dashboard, job-summary, and local-threshold examples, see [docs/summary-json-ci-cookbook.md](docs/summary-json-ci-cookbook.md). + ## Dependency Provenance Analysis (Opt-in) This section is about analyzing third-party package provenance signals. It is not about verifying the `sbom-diff-and-risk` tool's own release artifacts. diff --git a/tools/sbom-diff-and-risk/docs/report-schema.md b/tools/sbom-diff-and-risk/docs/report-schema.md index 1a7f549..837c11d 100644 --- a/tools/sbom-diff-and-risk/docs/report-schema.md +++ b/tools/sbom-diff-and-risk/docs/report-schema.md @@ -30,7 +30,7 @@ When provenance policy fields are relevant, reports may also include `provenance ## Summary contract -`summary` is the stable, compact entry point for automation that needs counts without walking the full report. The `--summary-json PATH` CLI option writes only this stable `report.json["summary"]` object. The checked-in [../examples/sample-summary.json](../examples/sample-summary.json) artifact is the summary-only output for the default CycloneDX example and matches the `summary` object in [../examples/sample-report.json](../examples/sample-report.json). +`summary` is the stable, compact entry point for automation that needs counts without walking the full report. The `--summary-json PATH` CLI option writes only this stable `report.json["summary"]` object. The checked-in [../examples/sample-summary.json](../examples/sample-summary.json) artifact is the summary-only output for the default CycloneDX example and matches the `summary` object in [../examples/sample-report.json](../examples/sample-report.json). For CI consumption examples, see [summary-json-ci-cookbook.md](summary-json-ci-cookbook.md). Base `summary` fields: diff --git a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md index 721a4e6..3a9297d 100644 --- a/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md +++ b/tools/sbom-diff-and-risk/docs/reviewer-evidence-pack.md @@ -75,6 +75,8 @@ The SARIF sample is intentionally conservative. It covers selected high-signal f For consumers of the JSON output, see [report-schema.md](report-schema.md). It documents the stable `summary` contract, including conditional `summary.policy` and `summary.enrichment` fields. +For CI dashboard, job-summary, and local-threshold examples that consume `outputs/summary.json`, see [summary-json-ci-cookbook.md](summary-json-ci-cookbook.md). + ## Release Verification Path Start with the GitHub Release for the version under review. For `v0.6.0`, inspect the release and assets: diff --git a/tools/sbom-diff-and-risk/docs/summary-json-ci-cookbook.md b/tools/sbom-diff-and-risk/docs/summary-json-ci-cookbook.md new file mode 100644 index 0000000..c143b21 --- /dev/null +++ b/tools/sbom-diff-and-risk/docs/summary-json-ci-cookbook.md @@ -0,0 +1,76 @@ +# Summary JSON CI cookbook + +This page shows how to consume `--summary-json PATH` in CI without changing the `sbom-diff-risk` analysis model. + +`--summary-json` writes a compact machine-readable JSON object. It is the same object as `report.json["summary"]`, and is useful for CI dashboards, job summaries, and small local gates where a repository wants to set its own thresholds. + +## Minimal command + +```bash +sbom-diff-risk compare \ + --before examples/cdx_before.json \ + --after examples/cdx_after.json \ + --out-json outputs/report.json \ + --summary-json outputs/summary.json +``` + +The full report remains available at `outputs/report.json`. The compact summary-only object is written to `outputs/summary.json`. + +## Python consumer + +This example reads the summary and applies an explicit local threshold. The threshold is chosen by the caller; it is not a built-in package safety verdict. + +```python +import json +from pathlib import Path + +summary = json.loads(Path("outputs/summary.json").read_text(encoding="utf-8")) + +added = summary["added"] +removed = summary["removed"] +changed = summary["changed"] +risk_counts = summary["risk_counts"] + +print(f"added={added} removed={removed} changed={changed}") +print(f"risk_counts={risk_counts}") + +max_new_packages = 2 +if risk_counts.get("new_package", 0) > max_new_packages: + raise SystemExit(f"new_package count exceeds local threshold: {max_new_packages}") +``` + +## PowerShell consumer + +This example uses `ConvertFrom-Json` and applies the same kind of explicit local threshold. + +```powershell +$summary = Get-Content outputs/summary.json -Raw | ConvertFrom-Json + +$added = $summary.added +$removed = $summary.removed +$changed = $summary.changed +$newPackageCount = $summary.risk_counts.new_package + +Write-Output "added=$added removed=$removed changed=$changed" +Write-Output "new_package=$newPackageCount" + +$maxNewPackages = 2 +if ($newPackageCount -gt $maxNewPackages) { + throw "new_package count exceeds local threshold: $maxNewPackages" +} +``` + +## Compatibility notes + +- `summary.policy` appears only when policy evaluation is applied. +- `summary.enrichment` appears only when PyPI or Scorecard enrichment is used. +- `unchanged` is absent because unchanged components are not modeled. +- Absence of `summary.policy` or `summary.enrichment` means the feature was not used, not that it failed. +- Consumers should treat new unrecognized fields as additive data. + +## Non-claims + +- `sbom-diff-risk` is not a CVE scanner. +- The summary is not a dependency safety oracle. +- Default runs do not perform hidden network access. +- Production PyPI publishing remains intentionally deferred.