Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflow-pins/tests/test_inputs_docs_drift.py
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,7 @@
sentinel="workflows_ref",
inputs_heading="## Inputs worth setting",
),
Row(workflow="review-readiness-shadow", sentinel="snapshot_and_policy_json", expect_header_with=False),
Row(workflow="pr-risk", sentinel="workflows_ref"),
Row(workflow="pr-size", sentinel="workflows_ref"),
Row(workflow="public-repo-hygiene", sentinel="workflows_ref"),
Expand Down
56 changes: 56 additions & 0 deletions .github/workflows/review-readiness-shadow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: Review readiness shadow (reusable)

on:
workflow_call:
inputs:
snapshot_and_policy_json:
description: Snapshot and advisory policy JSON, supplied by the caller.
required: true
type: string

permissions:
contents: read
pull-requests: read
checks: read
actions: read
statuses: read

jobs:
report:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
# The runner exposes workflow_sha on the job context. Serialize that
# context because actionlint's static schema does not yet include it;
# the SHA guard fails closed on older runners or an absent value.
- name: Validate provider revision
env:
PROVIDER_SHA: ${{ fromJSON(toJSON(job)).workflow_sha }}
run: |
if [[ ! "$PROVIDER_SHA" =~ ^[0-9a-f]{40}$ ]]; then
echo '::error::Cannot resolve the reusable workflow revision.'
exit 1
fi
- name: Load provider tools at the workflow revision
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
repository: Comfy-Org/github-workflows
ref: ${{ fromJSON(toJSON(job)).workflow_sha }}
path: _review_tool
persist-credentials: false
- name: Build advisory report
env:
GH_TOKEN: ${{ github.token }}
GH_HOST: github.com
SHADOW_EXPECTED_REPO: ${{ github.repository }}
SHADOW_INPUT: ${{ inputs.snapshot_and_policy_json }}
run: |
set -euo pipefail
printf '%s' "$SHADOW_INPUT" | python3 _review_tool/scripts/review-readiness/shadow_report.py > readiness-report.json
- name: Retain report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: review-readiness-shadow-${{ github.run_id }}-${{ github.run_attempt }}
path: readiness-report.json
if-no-files-found: error
retention-days: 30
51 changes: 51 additions & 0 deletions .github/workflows/test-review-readiness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,20 @@ on:
paths:
- 'scripts/review-readiness/**'
- '.github/workflows/test-review-readiness.yml'
- '.github/workflows/review-readiness-shadow.yml'
push:
branches: [main]
paths:
- 'scripts/review-readiness/**'
- '.github/workflows/test-review-readiness.yml'
- '.github/workflows/review-readiness-shadow.yml'

permissions:
contents: read
pull-requests: read
checks: read
actions: read
statuses: read

jobs:
test:
Expand All @@ -25,3 +31,48 @@ jobs:
with:
python-version: '3.12'
- run: python3 -m unittest discover -s scripts/review-readiness/tests -v

prepare-shadow:
if: ${{ github.event_name == 'pull_request' }}
runs-on: ubuntu-latest
outputs:
input: ${{ steps.input.outputs.json }}
steps:
- id: input
env:
TARGET_REPO: ${{ github.repository }}
TARGET_PR: ${{ github.event.pull_request.number }}
AUTHOR: ${{ github.event.pull_request.user.login }}
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
payload=$(jq -cn --arg repo "$TARGET_REPO" --argjson pr "$TARGET_PR" \
--arg author "$AUTHOR" --arg base "$BASE_SHA" --arg head "$HEAD_SHA" \
'{snapshot:{repo:$repo,pr:$pr,author:$author,base_sha:$base,head_sha:$head,tested_sha:$head,policy_version:"synthetic-ci"},policy:{version:"synthetic-ci",required_checks:[{name:"unit",revision:"tested"}],required_human_approvals:1}}')
printf 'json=%s\n' "$payload" >> "$GITHUB_OUTPUT"

shadow:
if: ${{ github.event_name == 'pull_request' }}
needs: prepare-shadow
permissions:
contents: read
pull-requests: read
checks: read
actions: read
statuses: read
uses: ./.github/workflows/review-readiness-shadow.yml
with:
snapshot_and_policy_json: ${{ needs.prepare-shadow.outputs.input }}

verify-shadow:
if: ${{ github.event_name == 'pull_request' }}
needs: shadow
runs-on: ubuntu-latest
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8
with:
name: review-readiness-shadow-${{ github.run_id }}-${{ github.run_attempt }}
- name: Verify report
run: |
jq -e '(.state == "WAITING" or .state == "BLOCKED") and .collection.snapshot == "matched" and .risk_collection == "observed"' readiness-report.json
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ complete, copy-pasteable caller.
| [`assign-reviewers.yml`](.github/workflows/assign-reviewers.yml) | Assigns PR owners using configured paths and recent human approval history. Chooses complementary file coverage before workload, with an explained one-owner default fallback and no random substitutions. Optionally generates a shared history artifact on schedule/manual dispatch and reuses fresh snapshots across PRs. Reads configuration from the base SHA, respects manual assignments, and preserves opt-in `REVIEWER_AUTHOR_ALLOWLIST` scoping. Requires `vars.APP_ID` + `CLOUD_CODE_BOT_PRIVATE_KEY`; callers skip forks and Dependabot triggers. | [assign-reviewers.md](docs/callers/assign-reviewers.md) |
| [`assign-prs-to-author.yml`](.github/workflows/assign-prs-to-author.yml) | Housekeeping — assigns every open PR with no assignees to its author (bot-authored PRs skipped by default). Run on a schedule from a thin caller; useful when a team tracks PR ownership via assignees. The calling job needs `pull-requests: write` and `issues: write`. | [assign-prs-to-author.md](docs/callers/assign-prs-to-author.md) |
| [`pr-size.yml`](.github/workflows/pr-size.yml) | PR-size cap — fails (or, in `mode: warn`, only reports) when a PR's net diff exceeds `max_lines` non-generated changed lines, keeping diffs reviewable. Excludes dependency lockfiles, `linguist-generated` files (read from the base ref, so a PR can't exempt itself), Go generated-code markers, and per-repo `extra_lockfiles` / `extra_generated_globs`. Opt in to `exclude_tests` to cap production code rather than test coverage (excluded test lines are always reported, never silently dropped). A `bypass_label` (default `oversized-ok`) waves through a legitimately large change; a sticky bot comment explains overages when `bot_app_id` + `BOT_APP_PRIVATE_KEY` are supplied (degrades to status + step summary without them). Counting logic + tests live in [`scripts/check-pr-size/`](scripts/check-pr-size). | [pr-size.md](docs/callers/pr-size.md) |
| [`review-readiness-shadow.yml`](.github/workflows/review-readiness-shadow.yml) | Read-only advisory risk and review evidence report. Retains one artifact; it cannot report READY or gate a merge. | [review-readiness-shadow.md](docs/callers/review-readiness-shadow.md) |
| [`pr-risk.yml`](.github/workflows/pr-risk.yml) | **Advisory PR risk grading (shadow check)** — **automatic grading off by default** (`enabled: false`; a manual `workflow_dispatch` grades regardless, so a repo can trial it before switching on); switch it on with `enabled: true` or by setting the caller repo's `RISK_CONFIG` variable to `{"enabled": true}`, which outranks the input in both directions so `{"enabled": false}` is a no-PR kill switch. Grades every PR into a tier `R0` (safest) .. `R3` (riskiest) and syncs one label (`risk:R0`..`risk:R3`, or `risk:ungraded` when an input was unreadable). The label is the entire product: nothing is gated, routed, commented, or merged. Deterministic (`gh` + `jq`, no LLM): `grade = worst(path_floor, provenance, reversibility)` — path-glob map, what-process-produced-the-diff (registered runbooks with identity + diff-shape assertions; forks are R3 with no exceptions), and revertability (persistent-state mutation, deletions under sensitive classes, did green checks cover the lines). Grader + generic defaults live in [`scripts/pr-risk/`](scripts/pr-risk); a consumer sharpens them with `.github/risk.json` / `.github/risk-runbooks.json`, read from the PR's **base ref** so a PR can't edit the rules that judge it. The job excludes its own run from the check rollup and waits (`wait_for_checks_minutes`) for the rest to settle before labeling. Labels ride the plain `GITHUB_TOKEN` (cannot fire `labeled` triggers — no cascade risk); disagreement is recorded with a human-owned `risk-dispute` label. **Two further publish surfaces are available and are OFF by default**, so an enrolled caller behaves byte-identically until it opts in: `sticky_comment: true` posts ONE comment (created once, updated in place — N pushes leave one comment) carrying the per-file path-axis breakdown, the risk CONCENTRATION sentence ("94% of this diff is R0/R1; the 6% that puts the path floor at R3 is these two files, 40 lines") and a "this grade is wrong" checkbox whose state round-trips into a `risk-grade-disputed` label (distinct from the human-owned `risk-dispute`, which the grader still never touches); `check_run: true` publishes the tier and reason as a Check Run on the head commit — the immutable, timestamped, commit-attached record a mutable label cannot be — from a SEPARATE job, the only one holding `checks: write` — a grant every caller makes whether or not it switches the surface on, because GitHub validates every nested job's declared `permissions:` at startup and a job `if:` is a runtime condition (the input decides whether a check is PUBLISHED, not whether the grant is CHECKED). Both surfaces are advisory in the same sense as the label: the Check Run's conclusion is hardcoded `neutral`, and every publish failure is an annotation, never a red check. Label text is remappable via `label_map`. `workflows_ref` is **required**, and both its **shape and its ancestry are enforced** — every job that checks it out fails the run *before* the tool checkout unless the value is a full 40-hex lowercase commit SHA **and** that commit is an ancestor of `main` of this repo (fetched from a literal upstream URL; no `github` context can name this repo from inside a reusable workflow, so a variable there would be an alias a fork could point at itself). Shape rejects everything mutable — a branch, a tag, a `refs/pull/N/head`. Ancestry rejects everything unmerged: a fork of this public repo shares its object store, and GitHub serves a fork PR's head objects from this repo's own URL unauthenticated, so a fork-authored SHA is just as well-shaped and would otherwise be checked out into a job holding the caller's write token. There is deliberately **no opt-out input** — an opt-out would be set by the very pin-bump PR the check distrusts. Two consequences by design: **pinning a not-yet-merged SHA is rejected** (merge first, then bump), and the axis **fails closed** if this repo ever goes private or `main` is force-rewritten past a consumer's pin (re-pin to a commit on `main` to recover). **What is still not machine-checked is that the pin is the *current* one**: a stale pin left behind when `uses:` moved is a merged ancestor too. The test that would close that is "equal to the commit `uses:` resolved to". That value IS readable — not as `github.workflow_sha` (the *caller's* top-level file) and not as the `job_workflow_sha` OIDC claim (not a `github` context property; reading it would need `id-token: write` from every caller), but as `job.workflow_sha`, the `job`-context accessor added in runner v2.334.0 that `groom.yml` reads (BE-8077). It is deliberately not wired in here: asserting equality would fail red on every caller whose `uses:` and `workflows_ref` have already drifted apart, which is a caller-contract change tracked separately. The other residual is irreducible: a fork commit that edits `pr-risk.yml` itself is rejected by ancestry, but a caller pinned at one runs the fork's own copy of the guard, and no in-file check can bound a revision of that file an attacker chose. **Reviewing the caller is what bounds both, and it is the only thing that can: require `uses:` at a full commit SHA of this repo and `with: workflows_ref:` set to that same SHA written out literally, character-for-character — never an expression, never a tag.** The guard also runs *before* enablement is resolved (the resolver is itself loaded from `workflows_ref`), so a floating pin fails red even with the `RISK_CONFIG` kill switch set — the switch stops the grading, not a broken enrollment. Call the workflow directly: a nested `workflow_call` chain through an org wrapper is unsupported. Enroll it as its own workflow rather than a job inside an existing CI workflow (the rollup exclusion is per-run). The calling job needs `contents: read` + `issues: write` + `pull-requests: write` + `checks: write` + `actions: read` + `statuses: read`; GitHub rejects a shorter grant at startup (a reusable workflow can only narrow the caller's token, never elevate it), so a caller enrolled from an older copy of this row fails before any step runs. The two label writes are the ONE label: repo-side label creation on first use maps to `issues`, and labeling a PR maps to `pull-requests` (the labels endpoint is dual-mapped by what the "issue" is, so `issues: write` alone 403s on a PR). `checks: write` is the `publish-check` job's declaration, required unconditionally per above — grading itself uses only `checks: read`. `actions: read` is for the rollup's `CheckRun -> checkSuite -> workflowRun` self-exclusion hop. No secrets. | [pr-risk.md](docs/callers/pr-risk.md) |
| [`pr-derisk.yml`](.github/workflows/pr-derisk.yml) | **On-demand de-risk split plan (`/derisk`, beta)** — **off by default** (`enabled: false`; `vars.DERISK_CONFIG` on the caller repo outranks it in both directions, so `{"enabled": false}` is a no-PR kill switch). Someone with write access comments `/derisk` on a pull request; the workflow re-grades that PR with the pr-risk grader, makes **one** model call for a semantic partition of the diff into a chain of 2–5 **sequential** PRs, and posts **one sticky advisory comment**: a verdict line and a chain table above the fold, the plan, the chain-landing rules and the floor math inside collapsed `<details>`. **Every floor shown is computed by `grade-pr-risk.sh --stdin`** over a synthetic record built from each step's files — the model proposes *which files go together* and nothing else, and any tier it writes is discarded before rendering. The partition must cover the changed-file set **exactly**; a missing or duplicated path is rejected, re-prompted **once**, then falls back. **Nothing is gated, routed, merged, labelled or filed** — filing tickets from a plan is a later rung behind its own command. Honest by construction: when no step lands below the PR's **path floor** the verdict reads "N smaller single-concern R3s, same lane", never a fake lane win, because the verdict is arithmetic over grader output rather than the model's prose — and the comparison is against the path floor rather than the headline tier precisely because a split only moves the path axis, so a fork PR (provenance R3, path floor R0) is told the truth instead of a reduction no partition can deliver. An over-budget diff, an unvalidatable partition and an API failure each post an explaining comment — a `/derisk` that quietly does nothing is never an outcome. Safe as a comment command because an `issue_comment` workflow runs from the caller's **default branch** (a PR cannot edit the workflow serving it), the commenter is gated on `author_association` (`allowed_associations`, default `OWNER,MEMBER,COLLABORATOR` — narrow it, never widen), and **no PR code is ever checked out**: the diff is read over the API and handed to a model as text. `workflows_ref` is **required** and its shape + ancestry are enforced by the guard byte-identical to `pr-risk.yml`'s (see that row for the full rationale and its two residuals); a test fails the build if the copies drift. The calling job needs `contents: read` + `pull-requests: write` + `checks: read` + `actions: read` + `statuses: read`, and the `anthropic_api_key` secret. | [pr-derisk.md](docs/callers/pr-derisk.md) |
| [`pr-area-label.yml`](.github/workflows/pr-area-label.yml) | **Agentic PR area labeling** — classifies each PR into exactly one `area:*` label, and (on push to the consumer's default branch) syncs the repo's `area:*` labels to the taxonomy. The taxonomy is the consumer's own `.github/area-labels.yml` (`repo_context` + `labels[]` with `name`/`color`/`description`/optional `guidance`); the shared workflow carries nothing repo-specific. An optional `sub_labels[]` block adds **deterministic path sub-labels** — `area:*` names with `paths:` globs, matched against the PR's changed files with no model involved, applied *alongside* the classified area and exempt from its cleanup (retired automatically once a PR stops matching). That is how a subsystem living inside another area's service stays filterable without splitting the parent area; names must be disjoint from `labels[]`. An LLM makes the domain-vs-path judgement a static `paths:` map can't, but with **no tools and no token**: PR title/body/paths/labels go to the Anthropic Messages API as data inside `<pr_data>` tags (diff excluded), the reply is enum-constrained by a JSON schema to the taxonomy's own names, and a deterministic step applies it with targeted `area:*` add/remove ops (never a full-set PUT, so concurrent non-area edits survive). The taxonomy is read from the PR's **base ref** — a PR can't rewrite the rules that classify it — and validated (unique `area:[a-z0-9-]+` names, every label resolves to a non-blank routing guide) before it drives a write; everything fails soft rather than failing the check. Classifier logic lives in [`scripts/area-label/`](scripts/area-label), loaded from the pinned `workflows_ref` (validated to a full 40-hex SHA before checkout). Fork/Dependabot PRs are skipped by construction (no writable token, no secret). Labels ride the plain `GITHUB_TOKEN` (no `labeled`-trigger cascade). The calling job needs `contents: read` + `issues: write` + `pull-requests: write`; the `ANTHROPIC_API_KEY` secret is provided org-wide (unset ⇒ `label-pr` fails soft, sync still works). | [pr-area-label.md](docs/callers/pr-area-label.md) |
Expand Down
1 change: 1 addition & 0 deletions docs/callers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ per-workflow pages give you a complete, copy-pasteable caller:
| `cursor-review.yml` | [cursor-review.md](cursor-review.md) | `CURSOR_API_KEY` |
| `cursor-review-auto-label.yml` | [cursor-review-auto-label.md](cursor-review-auto-label.md) | `vars.APP_ID` + App key |
| `pr-size.yml` | [pr-size.md](pr-size.md) | nothing (bot comment optional) |
| `review-readiness-shadow.yml` | [review-readiness-shadow.md](review-readiness-shadow.md) | nothing |
| `pr-risk.yml` | [pr-risk.md](pr-risk.md) | nothing |
| `pr-derisk.yml` | [pr-derisk.md](pr-derisk.md) | `ANTHROPIC_API_KEY` |
| `pr-area-label.yml` | [pr-area-label.md](pr-area-label.md) | `.github/area-labels.yml` (+ `ANTHROPIC_API_KEY`, org-wide) |
Expand Down
Loading
Loading