Skip to content

Keep conformance issue images as release assets - #614

Open
Velli20 wants to merge 1 commit into
mainfrom
conformance-issue-assets
Open

Velli20 wants to merge 1 commit into
mainfrom
conformance-issue-assets

Conversation

@Velli20

@Velli20 Velli20 commented Oct 27, 2026

Copy link
Copy Markdown
Owner

Why

Conformance issues embedded their example images from the GitHub Pages report. Pages holds only the newest deployment, is capped at 1 GB, and is rebuilt from artifacts with limited retention (run artifacts 14 days, pages-conformance 90 days), so issue images went missing or changed under the issue. GitHub has no API for uploading issue attachments with GITHUB_TOKEN, and data: images are stripped, so the files need another durable home that renders inline.

What

  • New tools/conformance/src/issue_assets.rs: an AssetStore that keeps files as assets of a conformance-assets pre-release (created on first use). Assets are named <issue key>--<content hash>--<file>: unchanged files are not uploaded again, a changed file gets a new name (so GitHub's image cache never shows a stale image), and an issue's older assets are deleted.
  • issues.rs: before writing an issue, uploads the example case's pN-ref/safe/diff.png, pN-content.txt, pN-streams.txt and summary.md. Existing issues whose bodies link no assets yet are rewritten once. Closing an issue (by the workflow or as not planned) deletes its assets. If the release cannot be written, a warning is logged and bodies fall back to Pages links; dry runs never touch the release.
  • issue_body.rs: images use the asset links (clicks still open the Pages viewer), an "Evidence kept with this issue" line, and an example entry with the asset links in the agents' JSON block.
  • repro.rs: with no Pages report, reruns the issue's cases against the PDFium image kept with the issue.
  • conformance.yml: the issues job gets contents: write for release uploads.
  • tools/conformance/README.md documents the above.

Notes

  • Creating the pre-release also creates a conformance-assets tag and an entry on the Releases page.
  • Not compiled locally: the Skia build could not run in the authoring sandbox, so CI is the first compile and test run. The edited files are rustfmt-clean.
  • Inline rendering of release-asset PNGs in issues is expected (gh release upload stores image/png from the extension), but has not yet been seen in a real issue; the first main run, or cargo conformance issues against a scratch fork, will confirm it.

🤖 Generated with Claude Code

Issues embedded their example images from the GitHub Pages report, which
holds only the newest deployment, is capped at 1 GB, and is rebuilt from
artifacts with limited retention, so images went missing or changed.

The issues job now uploads the example case's images and evidence to a
`conformance-assets` pre-release, named by issue key and content hash, and
links them from the body. Unchanged files are not uploaded again, stale
ones are deleted, and closing an issue deletes its assets. Without write
access the bodies fall back to Pages links. `repro` uses the kept PDFium
image when Pages has no report.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Conformance reads

Documents that Safe-PDF reads differently from the base commit, compared with PDFium. A read
regresses when Safe-PDF fails to open a document it opened before, crashes or times out, or
counts a different number of pages than PDFium.

pdf.js corpus

1009 documents compared with e43007c: 0 read worse, 1 read better.

Improvements

Document Base This pull request
issue17190_1 timeout: time limit exceeded (stage safe-read) read, 1 page

840 documents of the PDFium corpus read as on e43007c.

Details are in the job summaries of the workflow run. Rerun a case locally with
cargo conformance run --corpus <corpus> --read-only --case <id>.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant