Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 56 additions & 20 deletions .github/workflows/pr-risk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,14 +70,21 @@ name: PR Risk Grade (reusable)
# queue it already has, and how a PR is re-graded after a `.github/risk.json` change or a
# `risk-dispute`. Absent, the workflow reads the event exactly as before. Three things differ on
# the by-number path, all of them deliberate:
# * BOT-AUTHORED PRs ARE GRADED. The `github.actor != 'dependabot[bot]'` clause callers put in
# their `if:` is a TOKEN guard, not a policy one: a bot-triggered `pull_request` run gets a
# read-only GITHUB_TOKEN, so the label write — this workflow's entire product — would 403.
# On a dispatch the actor is the human who pressed the button and the token is writable, so
# the reason evaporates. Excluding them would also bias any backfill corpus badly; bots
# author a substantial minority of merged PRs. The same is true of the fork clause, and for
# the same reason — fork RISK is unaffected either way, because `external` is derived from
# the API's `isCrossRepository`, never from the actor, and forks grade R3 with no exceptions.
# * FORK PRs ARE GRADED. The `head.repo.full_name == github.repository` clause callers put in
# their `if:` is a TOKEN guard, not a policy one: a fork's `pull_request` run gets a
# read-only GITHUB_TOKEN that the caller's `permissions:` block cannot elevate, so the label
# write — this workflow's entire product — would 403. On a dispatch the token is writable, so
# the reason evaporates. Fork RISK is unaffected either way, because `external` is derived
# from the API's `isCrossRepository`, never from the actor, and forks grade R3 with no
# exceptions. DEPENDABOT-TRIGGERED RUNS need no such hatch and are graded on BOTH paths: the
# caller pattern below deliberately carries no `github.actor != 'dependabot[bot]'` clause,
# because the caller's `permissions:` block elevates Dependabot's read-only token and this
# workflow declares no secrets (see that pattern for the citation, for the one thing the
# clause did buy, and for the callers the reasoning does NOT transfer to). Read that clause
# literally: `github.actor` is who TRIGGERED the run, not who authored the PR, so it never
# keyed on bot authorship — a human pushing to a Dependabot branch made the actor human and
# the PR graded even with the clause in place. Skipping those runs would also bias any
# backfill corpus badly; bots author a substantial minority of merged PRs.
# * THE BASE REF IS RE-READ FROM THE API, per target, because there is no event payload to take
# it from. It is load-bearing rather than cosmetic: it selects which branch's
# `.github/risk.json` judges the PR, live PRs are commonly stacked on feature branches rather
Expand Down Expand Up @@ -164,20 +171,49 @@ name: PR Risk Grade (reusable)
# contents: read
# jobs:
# pr-risk:
# # SCOPE EVERY GUARD CLAUSE TO THE EVENT IT DESCRIBES. Both clauses below are TOKEN
# # guards: a fork's and a bot's `pull_request` run gets a read-only GITHUB_TOKEN, so the
# # label write would 403 and the check would go red on every dependency bump. Neither is
# # a risk judgement — forks are graded R3 from the API's own fork flag, never from the
# # actor. On a workflow_dispatch there is no `github.event.pull_request`, so an
# # unscoped `head.repo.full_name == github.repository` is FALSE and the job SKIPS
# # SILENTLY: the dispatch button appears to do nothing, no run, no error, no annotation.
# # Keeping the clauses behind the event test is what makes the button work at all — and
# # on a dispatch the token is writable and the actor is a human, so there is nothing left
# # for either clause to protect.
# # ONE CLAUSE, AND IT STAYS SCOPED TO THE EVENT IT DESCRIBES. The fork clause is a TOKEN
Comment thread
mattmillerai marked this conversation as resolved.
Resolved
# # guard, not a risk judgement: a fork's `pull_request` run gets a read-only GITHUB_TOKEN,
# # the `permissions:` block below CANNOT elevate it (the Dependabot carve-out below does
# # not extend to forks), so the label write would 403 and the check would go red. Fork
# # RISK is untouched either way — forks grade R3 from the API's own fork flag, never from
# # the actor — so a fork PR is simply ungraded-by-absence here; grade it by dispatching on
# # `pr_number`, or enroll with `pull_request_target` instead, which is safe by construction
# # for this workflow (see the fork paragraph above). Keep the clause BEHIND the event test:
# # on a workflow_dispatch there is no `github.event.pull_request`, so an unscoped
# # `head.repo.full_name == github.repository` is FALSE and the job SKIPS SILENTLY — the
# # dispatch button appears to do nothing, no run, no error, no annotation. That event test
# # is what makes the button work at all, and on a dispatch the token is writable anyway, so
# # there is nothing left for the clause to protect.
# #
# # THERE IS DELIBERATELY NO `github.actor != 'dependabot[bot]'` CLAUSE, AND ADDING ONE BACK
# # IS A REGRESSION. Dependabot's `pull_request` runs do start from a read-only
# # GITHUB_TOKEN, but the `permissions:` block below ELEVATES it — GitHub documents exactly
# # that under "Changing GITHUB_TOKEN permissions" in *Troubleshooting Dependabot on GitHub
# # Actions* / *Automating Dependabot with GitHub Actions* — and this workflow declares NO
# # secrets, so the other half of the Dependabot restriction (a Dependabot run sees
# # Dependabot secrets only, never Actions secrets) costs it nothing. Skipping those runs
# # instead leaves most repos' highest-volume automated PR producer ungraded.
# #
# # ONE THING THE CLAUSE DID BUY, and it does not earn the cost: a Dependabot
# # github-actions PR that bumps THIS caller's own `uses:` pin now runs the newly pinned
# # reusable inside that PR's own run (a `pull_request` run evaluates the HEAD's workflow
# # file) under the elevated token, where the clause used to skip it. The exposure is
# # narrow — Dependabot only proposes SHAs behind this repo's own published tags, the tool
# # checkout is separately gated on `workflows_ref` being an ancestor of `main`, and the
# # fleet's own bump PRs always had this property because the actor clause never covered
# # them. Review that PR before merging it for a different reason: Dependabot rewrites
# # `uses:` only and leaves `workflows_ref` behind, and the equality check that would catch
# # that is a deliberate open gap (see "WHAT ANCESTRY STILL DOES NOT PROVE" above), so an
# # unedited Dependabot bump runs a NEW workflow file against an OLD tool checkout. Fix the
# # input in the same PR.
# #
# # THIS REASONING DOES NOT TRANSFER to a caller that needs an ACTIONS SECRET — an app
# # private key, an API token; `cursor-review-auto-label.yml` in this repo is the local
# # example. Those callers still need the dependabot skip, because no `permissions:` block
# # can hand a Dependabot-triggered run an Actions secret.
# if: >-
# github.event_name != 'pull_request' ||
# (github.actor != 'dependabot[bot]' &&
# github.event.pull_request.head.repo.full_name == github.repository)
# github.event.pull_request.head.repo.full_name == github.repository
Comment thread
mattmillerai marked this conversation as resolved.
Resolved
# permissions:
# contents: read
# issues: write # create the risk:* labels repo-side on first use
Expand Down
43 changes: 43 additions & 0 deletions docs/callers/pr-risk.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,49 @@ your own steps (or jobs) that check out or run PR head code under it reopens
the classic pwn-request hole. Same-repo-only callers (private org repos that
take no fork PRs) should stay on plain `pull_request`.

That is why the caller block above carries **no `if:` at all**, and why the two
paragraphs below talk about a clause it does not show. Neither shape this page
recommends needs one: under `pull_request_target` the token is writable for fork
PRs too, and a same-repo-only repo has no fork PRs to guard against. The third
shape — a public repo staying on plain `pull_request` and accepting that its fork
PRs go **ungraded** rather than red — is the one that needs
`github.event.pull_request.head.repo.full_name == github.repository`, scoped
behind an event test so `workflow_dispatch` still works. Copy that variant from
the caller pattern in
[`pr-risk.yml`'s header](../../.github/workflows/pr-risk.yml) rather than writing
it yourself; an unscoped clause makes the dispatch button a silent no-op.

**Bot PRs are graded — do not add a `github.actor != 'dependabot[bot]'` skip.**
Dependabot's `pull_request` runs do start from a read-only `GITHUB_TOKEN`, but
the caller's `permissions:` block elevates it — that is GitHub's documented
behaviour ([Changing `GITHUB_TOKEN`
permissions](https://docs.github.com/en/code-security/dependabot/troubleshooting-dependabot/troubleshooting-dependabot-on-github-actions#changing-github_token-permissions))
— and this workflow declares no secrets, so the label write succeeds. A
dependabot skip therefore buys almost nothing and leaves most repos'
highest-volume automated PR producer ungraded. (Almost: the clause did skip
Dependabot's own github-actions PRs bumping *this* caller's pins, which now run
the newly pinned reusable inside their own run under the elevated token. The
exposure is narrow — Dependabot only proposes SHAs behind this repo's published
tags, and the tool checkout is separately gated on `workflows_ref` being an
ancestor of `main` — and the fleet's own bump PRs always behaved this way, since
the clause keyed on `github.actor`, the run's **trigger**, not the PR's author. Do
review such a PR before merging: Dependabot rewrites `uses:` and leaves
`workflows_ref` stale, so fix the input in the same PR.) The reasoning is
specific to *this* workflow: a caller that needs an **Actions secret** (an app
private key, an API token —
[`cursor-review-auto-label.yml`](cursor-review-auto-label.md) is the org's
example) still needs the skip, because a Dependabot-triggered run sees Dependabot
secrets only and no `permissions:` block can change that. Fork PRs are the
opposite case: the `permissions:` elevation does not extend to them, which is why
the fork gotcha above is still a real token guard.

Comment thread
mattmillerai marked this conversation as resolved.
Resolved
**Already carrying that skip? Removing it is yours to do, not the fleet's** —
[`bump-pr-risk-callers.yml`](../../.github/workflows/bump-pr-risk-callers.yml)
moves SHA pins only and will never edit your `if:`, so drop the clause in your
own repo, and update in the same PR anything repo-side that mirrors the caller's
`if:`/`concurrency` shape (a unit test asserting on the caller's YAML is the
usual one).

**Enroll it as its own workflow, not a job inside an existing CI workflow.**
The grading job excludes its own *run* from the check rollup it reads; a job
sharing a run with the rest of CI excludes its siblings too and lands on the
Expand Down
30 changes: 22 additions & 8 deletions scripts/pr-risk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,14 +123,28 @@ logic is unchanged either way — it was always an API read keyed on a PR number
never a read of the event payload. What the number-supplied path changes is only
which PR is read, plus three consequences worth knowing:

- **Bot-authored and fork PRs are graded here.** The `github.actor != 'dependabot[bot]'`
and `head.repo.full_name == github.repository` clauses in the caller's `if:`
are **token** guards — a bot's or a fork's `pull_request` run gets a read-only
`GITHUB_TOKEN`, so the label write would 403 and the check would go red on
every dependency bump. On a dispatch the token is writable and the actor is a
human, so neither applies. Fork **risk** is untouched: `external` comes from
the API's own fork flag, never from the actor, and still grades R3.
- **Both clauses must be scoped to the event, or the dispatch is a silent
- **Fork PRs are graded here.** The `head.repo.full_name == github.repository`
clause in the caller's `if:` is a **token** guard — a fork's `pull_request` run
gets a read-only `GITHUB_TOKEN` that the caller's `permissions:` block cannot
elevate, so the label write would 403 and the check would go red. On a dispatch
the token is writable, so it does not apply. Fork **risk** is untouched:
`external` comes from the API's own fork flag, never from the actor, and still
grades R3. **Dependabot-triggered runs need no such
hatch — they are graded on the event path too**, so the caller pattern carries
no `github.actor != 'dependabot[bot]'` clause and one must not be added back:
Dependabot's `pull_request` runs start read-only, but the caller's
`permissions:` block elevates the token ([Changing `GITHUB_TOKEN`
permissions](https://docs.github.com/en/code-security/dependabot/troubleshooting-dependabot/troubleshooting-dependabot-on-github-actions#changing-github_token-permissions)),
and this workflow declares no secrets — the other half of the Dependabot
restriction (Dependabot secrets only, never Actions secrets) costs it nothing.
That last clause is why the reasoning does **not** transfer to a caller which
needs an Actions secret, such as `cursor-review-auto-label.yml`. Note the
removed clause keyed on `github.actor` — who **triggered** the run, not who
authored the PR — so it never selected bot authorship: a human pushing to a
Dependabot branch made the actor human and the PR graded regardless. What it
did select is exactly the read-only-token set the `permissions:` block now
covers.
- **The fork clause must be scoped to the event, or the dispatch is a silent
no-op.** There is no `github.event.pull_request` on a `workflow_dispatch`, so
an unscoped fork clause is false and the job skips with no run, no error and no
annotation. The same applies to the concurrency group: an event-only key
Expand Down
Loading