[codex] sbom-diff-and-risk v0.3.0

.github/workflows/sbom-diff-and-risk-ci.yml

@@ -24,10 +24,10 @@ jobs:
         working-directory: tools/sbom-diff-and-risk
     steps:
       - name: Check out repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v4
 
       - name: Set up Python
-        uses: actions/setup-python@v6
+        uses: actions/setup-python@v5
         with:
           python-version: "3.11"
 
@@ -57,7 +57,7 @@ jobs:
 
   build-and-attest:
     # Keep provenance publication on trusted non-PR runs so consumers verify
-    # workflow-produced wheel and sdist artifacts from this repository workflow.
+    # workflow-produced wheel/sdist artifacts from this repository workflow.
     if: github.event_name != 'pull_request'
     needs: test
     runs-on: ubuntu-latest

.github/workflows/sbom-diff-and-risk-code-scanning.yml

@@ -18,7 +18,7 @@ jobs:
         working-directory: tools/sbom-diff-and-risk
     steps:
       - name: Check out repository
-        uses: actions/checkout@v6
+        uses: actions/checkout@v5
 
       - name: Set up Python
         uses: actions/setup-python@v6

tools/sbom-diff-and-risk/README.md

@@ -1,6 +1,6 @@
 # sbom-diff-and-risk
 
-v0.2.0 adds policy-based enforcement, SARIF export, GitHub code scanning integration, and deterministic parser hardening for Python dependency inputs.
+v0.3.0 adds opt-in PyPI provenance enrichment, provenance-aware policy and reporting, optional advisory Scorecard signals, and self-provenance verification guidance for workflow-built artifacts.
 
 `sbom-diff-and-risk` is a local, deterministic CLI for comparing two SBOMs or dependency manifests and producing JSON plus Markdown reports.
 
@@ -156,9 +156,82 @@ sbom-diff-risk compare \
 - `--warn-on rule[,rule...]`
 - `--strict`
 - `--enrich-pypi`
+- `--pypi-timeout seconds`
+- `--enrich-scorecard`
+- `--scorecard-timeout seconds`
 - `--source-allowlist pypi.org,files.pythonhosted.org,github.com`
 
-`--enrich-pypi` is reserved for future work and currently returns a clear error.
+Offline mode remains the default. No network access occurs unless `--enrich-pypi` or `--enrich-scorecard` is set explicitly.
+
+## Opt-in Provenance Enrichment
+
+PyPI provenance and integrity enrichment is explicit and additive in this PR:
+
+- only Python / PyPI packages are queried
+- no hidden network access occurs in default mode
+- enrichment results are captured as evidence and summarized in the reports
+- per-component `evidence.provenance` records stable lookup fields such as `supported`, `lookup_performed`, and per-file attestation totals
+- lack of attestation is treated as unavailable metadata, not as proof of compromise
+- policy evaluation can use these signals explicitly when configured
+- SARIF stays conservative and only emits selected high-signal provenance policy violations
+
+When enabled, the tool queries PyPI-facing release metadata plus file-level provenance data and records stable evidence fields under component `evidence.provenance`, along with run metadata under `metadata.enrichment` and the top-level trust-signal report fields in the JSON report.
+
+```bash
+sbom-diff-risk compare \
+  --before examples/requirements_before.txt \
+  --after examples/requirements_after.txt \
+  --enrich-pypi \
+  --pypi-timeout 3 \
+  --out-json outputs/report-enriched.json
+```
+
+## Provenance-Aware Reporting
+
+When provenance enrichment is enabled, the reports surface trust signals directly instead of burying them in component evidence:
+
+- JSON includes `provenance_summary`, `attestation_summary`, `enrichment_metadata`, `trust_signal_notes`, and `provenance_policy_impact`
+- Markdown includes `Provenance summary`, `Attestation gaps`, `Policy impact for provenance-related rules`, and `Trust signal notes`
+- core diff semantics do not change when enrichment is enabled
+- SARIF maps only selected high-signal provenance decisions such as `provenance_required`, blocking `missing_attestation`, and blocking `unverified_provenance`
+- provenance-related SARIF alerts prefer file-level locations that point to the relevant compared manifest or SBOM input
+
+Routine enrichment outcomes remain JSON and Markdown evidence for review. Non-blocking enrichment facts do not automatically become SARIF alerts.
+
+## Opt-in Scorecard Enrichment
+
+OpenSSF Scorecard enrichment is also explicit and advisory:
+
+- no Scorecard requests are made unless `--enrich-scorecard` is set
+- lookups only occur when a component can be mapped to a repository with high confidence from explicit metadata
+- repository registry pages and ambiguous URLs are treated as unmapped instead of inferred
+- Scorecard results are auxiliary trust signals, not proof of safety
+- Scorecard-only SARIF alerts are emitted only when policy explicitly turns a threshold breach into a violation
+
+```bash
+sbom-diff-risk compare \
+  --before examples/cdx_before.json \
+  --after examples/cdx_after.json \
+  --enrich-scorecard \
+  --scorecard-timeout 3 \
+  --out-json outputs/report-scorecard.json
+```
+
+If you want policy gating, make it explicit with a v3 policy such as [policy-scorecard-minimal.yml](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml), which sets `minimum_scorecard_score` and opts into the `scorecard_below_threshold` rule.
+
+Setting `minimum_scorecard_score` alone is advisory metadata for review. It only affects policy outcomes when `scorecard_below_threshold` is configured explicitly in `block_on`, `warn_on`, or `ignore_rules`.
+
+## Self-provenance
+
+This repository also records provenance for `sbom-diff-and-risk` itself by generating GitHub artifact attestations for the wheel and source distribution produced by the `sbom-diff-and-risk-ci` workflow.
+
+- the attested files are the wheel and source distribution built by `python -m build` from `tools/sbom-diff-and-risk`
+- the build files are uploaded together as the `sbom-diff-and-risk-dist` workflow artifact
+- only trusted non-PR runs publish the attestation
+- consumers can verify provenance with GitHub's attestation tooling after downloading one of those artifacts
+- this complements the tool's analysis of third-party supply-chain inputs, but it does not replace that analysis
+
+See [docs/self-provenance.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/docs/self-provenance.md) for the exact attested filenames, where the evidence appears in GitHub, and a run-by-run verification flow for consumers.
 
 ## Examples
 
@@ -167,11 +240,15 @@ The [examples/](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-an
 - before/after inputs for CycloneDX JSON, SPDX JSON, `requirements.txt`, and `pyproject.toml`
 - dependency-group examples at `examples/pyproject_groups_before.toml` and `examples/pyproject_groups_after.toml`
 - example policies at `examples/policy-minimal.yml` and `examples/policy-strict.yml`
+- provenance-aware policy examples at `examples/policy-provenance-minimal.yml` and `examples/policy-provenance-strict.yml`
+- a Scorecard-aware policy example at `examples/policy-scorecard-minimal.yml`
 - a sample pass JSON report at [sample-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-report.json)
 - a sample pass Markdown report at [sample-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-report.md)
 - sample policy-warn reports at [sample-policy-warn-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.json) and [sample-policy-warn-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-policy-warn-report.md)
 - sample policy-fail reports at [sample-policy-fail-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.json) and [sample-policy-fail-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-policy-fail-report.md)
 - a sample SARIF export at [sample-sarif.sarif](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-sarif.sarif)
+- provenance-aware sample reports at [sample-provenance-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-provenance-report.json), [sample-provenance-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-provenance-report.md), and [sample-provenance-report.sarif](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-provenance-report.sarif)
+- Scorecard-aware sample reports at [sample-scorecard-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-scorecard-report.json), [sample-scorecard-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-scorecard-report.md), and [sample-scorecard-report.sarif](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-scorecard-report.sarif)
 - requirements-based sample reports at [sample-requirements-report.json](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-requirements-report.json) and [sample-requirements-report.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/examples/sample-requirements-report.md)
 
 ## Enforcement Mode
@@ -214,9 +291,10 @@ SARIF export is intentionally conservative. The current renderer emits a GitHub-
 - `suspicious_source`
 - `unknown_license`
 - `major_upgrade`
-- selected blocking policy results such as `max_added_packages` and `allow_sources`
+- selected policy results such as `max_added_packages`, `allow_sources`, `provenance_required`, and blocking provenance violations like `missing_attestation` or `unverified_provenance`
+- explicit Scorecard policy violations such as `scorecard_below_threshold`
 
-It does not turn every diff or informational heuristic into a code scanning alert.
+It does not turn every enrichment fact, diff, or informational heuristic into a code scanning alert.
 
 ```bash
 sbom-diff-risk compare \
@@ -228,17 +306,8 @@ sbom-diff-risk compare \
 
 For GitHub code scanning integration guidance and a minimal upload workflow, see [docs/github-code-scanning.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/docs/github-code-scanning.md).
 
-## Self-provenance
-
-This repository also records provenance for `sbom-diff-and-risk` itself by generating GitHub artifact attestations for the wheel and source distribution produced by the `sbom-diff-and-risk-ci` workflow.
+For details on how this repository attests the tool's own wheel and source distribution artifacts, see [docs/self-provenance.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/docs/self-provenance.md).
 
-- the attested files are the wheel and source distribution built by `python -m build` from `tools/sbom-diff-and-risk`
-- the build files are uploaded together as the `sbom-diff-and-risk-dist` workflow artifact
-- only trusted non-PR runs publish the attestation
-- consumers can verify provenance with GitHub's attestation tooling after downloading one of those artifacts
-- this complements the tool's analysis of third-party supply-chain inputs, but it does not replace that analysis
-
-See [docs/self-provenance.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/docs/self-provenance.md) for the exact attested filenames, where the evidence appears in GitHub, and a run-by-run verification flow for consumers.
 ## Parser Boundaries
 
 Deterministic local mode intentionally supports a conservative subset of packaging syntax. The detailed matrix lives in [docs/parser-boundaries.md](D:/OneDrive/Code/scientific-computing-toolkit/tools/sbom-diff-and-risk/docs/parser-boundaries.md).
@@ -266,9 +335,12 @@ Deterministic local mode intentionally supports a conservative subset of packagi
 ## Limitations
 
 - default mode is local-file based only.
+- PyPI provenance enrichment is opt-in only via `--enrich-pypi`; default runs stay offline.
 - `generated_at` remains `null` to preserve deterministic report output.
 - `stale_package` is not resolved offline. The report emits `not_evaluated` instead.
-- SARIF export intentionally covers only a conservative subset of findings in v0.2.
+- provenance evidence is recorded for supported PyPI packages only; unsupported and failed lookups remain explicit evidence gaps.
+- SARIF export intentionally covers only a conservative subset of findings in v0.2, including only selected high-signal provenance policy violations.
+- Scorecard enrichment is opt-in only via `--enrich-scorecard`, uses only high-confidence repository mappings, and remains advisory unless policy explicitly gates it.
 - No vulnerability database integration, CVE matching, or advisory enrichment.
 - `requirements.txt` support intentionally covers a conservative subset: plain PEP 508 requirement entries, comments, extras, markers, and line continuations.
 - `requirements.txt` intentionally rejects include/constraint directives, editable installs, direct URL/path refs, index/source options, and other pip-only install flags in deterministic mode.

tools/sbom-diff-and-risk/docs/dependency-risk-heuristics.md

@@ -25,6 +25,7 @@ The current rules are intentionally conservative:
 ## Deferred work
 
 - real `stale_package` evaluation behind explicit enrichment
+- provenance-based policy gates over opt-in enrichment evidence
 - ecosystem-specific trust rules
 - advisory and CVE enrichment
 - configurable risk policy profiles

tools/sbom-diff-and-risk/docs/policy-schema.md

@@ -1,15 +1,17 @@
 # Policy schema
 
-`sbom-diff-and-risk` supports a YAML-only policy schema in v1.
+`sbom-diff-and-risk` supports YAML-only policy schemas in versions `1`, `2`, and `3` for the local, provenance-aware, and optional Scorecard-aware policy flows described here.
 
 The schema is intentionally conservative and fail-closed:
 
 - unknown rule ids are rejected
 - unknown top-level keys are rejected
 - invalid types are rejected
-- only schema version `1` is supported
+- version `1` remains the v0.2-compatible schema and existing v0.2 policies continue to work unchanged
+- version `2` adds provenance-aware gating for explicit PyPI enrichment evidence
+- version `3` adds optional Scorecard-aware gating for explicitly requested Scorecard enrichment
 
-## Fields
+## Version 1 fields
 
 - `version: 1`
 - `block_on: [rule_id, ...]`
@@ -18,7 +20,7 @@ The schema is intentionally conservative and fail-closed:
 - `allow_sources: [host, ...]`
 - `ignore_rules: [rule_id, ...]`
 
-## Supported rule ids
+## Version 1 supported rule ids
 
 - `new_package`
 - `major_upgrade`
@@ -29,6 +31,41 @@ The schema is intentionally conservative and fail-closed:
 - `max_added_packages`
 - `allow_sources`
 
+## Version 2 fields
+
+Version `2` supports every version `1` field plus:
+
+- `require_attestations_for_new_packages: bool`
+- `require_provenance_for_suspicious_sources: bool`
+- `allow_unattested_packages: [package_name, ...]`
+- `allow_provenance_publishers: [publisher_kind, ...]`
+- `allow_unattested_publishers: [publisher_kind, ...]` as an accepted compatibility alias for `allow_provenance_publishers`
+
+`allow_provenance_publishers` is the canonical publisher override field. The parser also accepts `allow_unattested_publishers` as an alias when teams want a more explicit override-style name in review. Neither field treats missing attestations as trusted; they only constrain which attested publisher kinds count as verified provenance.
+
+## Version 2 supported rule ids
+
+Version `2` supports every version `1` rule id plus:
+
+- `missing_attestation`
+- `unverified_provenance`
+- `provenance_unavailable`
+- `provenance_required`
+
+## Version 3 fields
+
+Version `3` supports every version `1` and `2` field plus:
+
+- `minimum_scorecard_score: float`
+
+`minimum_scorecard_score` is advisory by itself. It only affects policy outcomes when you also opt into the `scorecard_below_threshold` rule through `block_on`, `warn_on`, or `ignore_rules`.
+
+## Version 3 supported rule ids
+
+Version `3` supports every version `1` and `2` rule id plus:
+
+- `scorecard_below_threshold`
+
 ## Semantics
 
 - `block_on` turns matching rule ids into blocking violations.
@@ -37,8 +74,20 @@ The schema is intentionally conservative and fail-closed:
 - `max_added_packages` enforces a deterministic threshold on the added component count.
 - `allow_sources` enforces exact host matches against `source_url` hosts for added and changed components.
 - `ignore_rules` suppresses matching rule ids entirely.
+- `missing_attestation` means PyPI release metadata was fetched successfully but no attestations were present.
+- `provenance_unavailable` means the run did not have usable provenance evidence for that package, for example because enrichment was disabled, unsupported, or failed.
+- `unverified_provenance` means attestations were present, but the provenance could not be verified against publisher metadata.
+- `provenance_required` is a policy-only rule emitted when an explicit provenance requirement was not satisfied.
+- `require_attestations_for_new_packages` applies only to added PyPI packages.
+- `require_provenance_for_suspicious_sources` applies only when the component also triggered `suspicious_source`.
+- `allow_unattested_packages` is a narrow package-name override for explicit missing-attestation exceptions only.
+- `allow_unattested_packages` does not waive `provenance_unavailable` or `unverified_provenance`; those remain separate, reviewable policy decisions.
+- `allow_provenance_publishers` and `allow_unattested_publishers` apply only when attestations exist and publisher kinds are available to verify.
+- when enrichment is disabled, deterministic local mode is unchanged unless a provenance-aware policy explicitly turns unavailable evidence into a warning or block.
+- `minimum_scorecard_score` does not create alerts or blocks on its own; it only becomes enforceable when `scorecard_below_threshold` is configured explicitly.
+- Scorecard evidence remains an auxiliary trust signal. A high score is not proof of safety, and missing Scorecard data is not proof of risk.
 
-## Example
+## Version 1 example
 
 ```yaml
 version: 1
@@ -54,3 +103,29 @@ allow_sources:
 ignore_rules:
   - major_upgrade
 ```
+
+## Version 2 example
+
+```yaml
+version: 2
+block_on:
+  - provenance_required
+  - provenance_unavailable
+warn_on:
+  - missing_attestation
+require_attestations_for_new_packages: true
+require_provenance_for_suspicious_sources: true
+allow_unattested_packages:
+  - pip
+allow_unattested_publishers:
+  - github actions
+```
+
+## Version 3 example
+
+```yaml
+version: 3
+warn_on:
+  - scorecard_below_threshold
+minimum_scorecard_score: 7.0
+```

tools/sbom-diff-and-risk/docs/self-provenance.md

@@ -11,7 +11,7 @@ The attested subjects are the exact Python distributables built from `tools/sbom
 
 Those two files are uploaded together as the workflow artifact named `sbom-diff-and-risk-dist`. The attestation applies to the built files themselves, not just to the artifact bundle name shown in the Actions UI.
 
-Current attestations cover workflow-built wheel and sdist artifacts, not GitHub Release assets or PyPI-published distributions.
+This repository does not currently publish PyPI Trusted Publishing provenance or immutable GitHub release attestations as part of this workflow. The current self-provenance coverage is limited to the workflow-produced wheel and source distribution files.
 
 ## Workflow and permissions
 

tools/sbom-diff-and-risk/examples/policy-provenance-minimal.yml

@@ -0,0 +1,8 @@
+# Missing attestation remains a review signal, not proof of compromise.
+version: 2
+warn_on:
+  - missing_attestation
+  - provenance_required
+require_attestations_for_new_packages: true
+allow_unattested_packages:
+  - pip

tools/sbom-diff-and-risk/examples/policy-provenance-strict.yml

@@ -0,0 +1,12 @@
+# Explicit provenance requirements for enriched PyPI evidence.
+version: 2
+block_on:
+  - provenance_required
+  - provenance_unavailable
+  - unverified_provenance
+warn_on:
+  - missing_attestation
+require_attestations_for_new_packages: true
+require_provenance_for_suspicious_sources: true
+allow_unattested_publishers:
+  - github actions

tools/sbom-diff-and-risk/examples/policy-scorecard-minimal.yml

@@ -0,0 +1,4 @@
+version: 3
+warn_on:
+  - scorecard_below_threshold
+minimum_scorecard_score: 7.0