Skip to content

PRD-25: release evidence that opens - #25

Merged
DurthVadr merged 1 commit into
mainfrom
prd-25/release-evidence
Sep 15, 2026
Merged

DurthVadr merged 1 commit into
mainfrom
prd-25/release-evidence

Conversation

@DurthVadr

Copy link
Copy Markdown
Member

The release page's SBOM listed nothing and the attestations lived only in GitHub's store. Now the SBOM lists the package and every locked runtime dependency (and the release fails if it does not), the three Sigstore bundles are release assets in SHA256SUMS, and the release job verifies the attestations before it publishes.

Verified locally against real data

  • The PRD's R1 (file: on the wheel) would have shipped the same empty SBOM. syft 1.42.3, the version inside anchore/sbom-action@e22c389, scanning shim-0.3.1-py3-none-any.whl as a file: zero components. Scanning the build job's existing benchmark venv (tagged wheel --no-deps over requirements.lock --require-hashes): 11 library components with versions (shim, annotated-doc, click, markdown-it-py, mdurl, phonenumbers, pygments, rich, shellingham, tomli-w, typer). The step now scans that venv.
  • The inline SBOM assertion, extracted and run: passes on the venv SBOM; fails on the real 0.3.1 release SBOM (KeyError: 'components'), on a wheel-file SBOM, on the venv SBOM with rich removed ([('rich', '15.0.0')]), and on a version mismatch.
  • tomli and colorama are not asserted: the lock pins them only under markers (python_version < "3.11", Windows), so they are not installed on 3.13. The assertion checks every unmarked pin.
  • Action inputs read at the pinned SHAs: sbom-action file → file:, path → dir:; actions/attest output bundle-path, subject-path accepts newline lists and globs, one bundle per step in a fresh temp dir.
  • gh attestation verify against 0.3.1: --repo exits 0; --bundle with the provenance line exits 0; the SBOM bundle needs --predicate-type https://cyclonedx.org/bom, which the record includes.
  • Both new shell steps simulated: 13 files in SHA256SUMS, all OK.

Requirements

  • R1 SBOM lists package + dependencies, release fails otherwise: tests/contracts/test_workflows.py::test_the_sbom_scans_the_installed_wheel_and_is_checked.
  • R2 bundles attached and hashed (shim-<v>.intoto.jsonl, shim-<v>-py3-none-any.whl.sigstore.json, shim.pyz.sigstore.json; shim.pyz a subject of both attest steps): test_the_attestation_bundles_are_hashed_and_attached.
  • R3 verify before publish (wheel, sdist, shim.pyz; attestations: read): test_the_release_verifies_attestations_before_it_publishes.
  • R4 docs/compatibility.md row with the component count and both --bundle commands; README "How this is verified" row; SECURITY.md sentence with link.
  • R5 each test was confirmed to fail on a deliberately broken workflow (missing bundle, old path, missing permission, missing pyz verify).

Deviations

  • R1 scans the installed-wheel venv, not the wheel file (above). R5's test pins that instead of file: *.whl.
  • The flat-download checksum check moved from build to attest, where the bundles exist, and covers all 14 files. release downloads a new attested-dist artifact (an artifact name cannot be re-uploaded); publish still uses release-dist.
  • shim.pyz.sigstore.json is a copy of the provenance bundle (the pyz is a subject of it).
  • No new job; every action still pinned by SHA.

This runs for real on the 0.3.2 tag.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@DurthVadr
DurthVadr merged commit 8a1e4b4 into main Sep 15, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant