Skip to content

bench-diff on pull_request - #330

Merged
mallendem merged 3 commits into
mainfrom
fix/bench-diff-pull-request
Oct 30, 2026
Merged

mallendem merged 3 commits into
mainfrom
fix/bench-diff-pull-request

Conversation

@mallendem

Copy link
Copy Markdown
Contributor

Switch bench-diff from pull_request_target to pull_request.

Why

bench-diff.yml checked out the PR head SHA and ran go test -bench on it under pull_request_target, with a checks: write token. The fork gate was the safe-to-test label, but the trigger included synchronize. Once a fork PR was labelled, every later push ran unreviewed code with that token (label TOCTOU). Event data was also interpolated directly in run:. Benchmarks need no secrets, so pull_request is enough.

Changes (.github/workflows/bench-diff.yml)

  • pull_request_target (labeled, synchronize) becomes pull_request. push is kept.
  • Removed the safe-to-test / fork if: gate and the "Get sha commit" step, which had ${{ }} inside run:. The checkout ref is now github.event.pull_request.head.sha || github.sha, the same SHA as before.
  • actions/checkout and actions/setup-go are pinned by SHA with version comments (v7.0.1 and v7.0.0, the latest releases). benchdiff-action was already pinned (v0.3.5, still the latest).
  • report_status is true only on push. The check-run is POSTed by the action with the job token, which is read-only on pull_request from forks and Dependabot.
  • The job runs on push, or on pull_request only for fork PRs: same-repo PRs are covered by the push run on the same SHA, so benchmarks are not run twice.
  • A new step writes the benchstat output to the job summary. It passes the output through an env var.

Behaviour change

  • Fork PRs: no benchdiff-result check-run any more. The results are in the job summary. Previously a labelled fork PR got the check-run.
  • safe-to-test label: no longer used by this workflow (the label is still used elsewhere in the repo's history, and it can be deleted).
  • Same-repo PRs: the check-run is still posted by the push run for the branch, on the head commit, so it still shows on the PR. The pull_request run does not run for same-repo PRs (only for forks), so there are no duplicate benchmark runs.
  • Dependabot PRs: the check-run comes from the push run, as before. The pull_request run is skipped for them (same-repo).
  • push to main and tags: unchanged.
  • Not done (follow-up): a workflow_run workflow could post the check-run for fork PRs without checking out PR code. I left it out because it needs artifact plumbing and benchdiff-action has no "post only" mode.

Required checks

benchdiff-result and bench-diff are not required. Classic protection on main requires lint, CLA and run-tests. The org rulesets (221698, 221255, 1225254, 221699) have no required status checks.

Verification

Done:

  • YAML parses.
  • zizmor 1.30.1 (offline, all audits) on the file reports no findings.
  • Action SHAs resolved with gh api against the release tags.
  • Read the benchdiff-action source (report-status.sh, run.sh) to confirm how the check-run is posted.

Not done:

  • The workflow has not been run on GitHub. I have not tested a fork PR run or the job summary output.
  • Not checked: whether benchdiff output is large enough to hit the job summary size limit (1 MiB).

Part of the pull_request_target audit: https://github.com/elastic/observability-robots/issues/5292

@mallendem
mallendem marked this pull request as ready for review October 29, 2026 12:05
Copilot AI balanced review requested due to automatic review settings October 29, 2026 12:05
@mallendem
mallendem requested a review from a team as a code owner October 29, 2026 12:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@mallendem
mallendem enabled auto-merge (squash) October 30, 2026 12:35
@mallendem
mallendem merged commit 9639936 into main Oct 30, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants