diff --git a/.github/workflows/linear-ticket.yml b/.github/workflows/linear-ticket.yml index a151a3ba..f1e82473 100644 --- a/.github/workflows/linear-ticket.yml +++ b/.github/workflows/linear-ticket.yml @@ -14,8 +14,11 @@ name: Linear Ticket (reusable) # * the signal workflow checks out nothing, runs no PR content, emits no artifact/cache; # * this validator never downloads that run's artifacts/caches and never checks out the PR # branch or merge commit — it reads only GitHub-owned data and Linear's API; -# * every PR-derived value (branch, title, body, labels, URL) is untrusted DATA, passed -# through env / GraphQL variables, never shell-interpolated. +# * every PR-derived value (branch, title, body, labels, URL, changed-file paths) is +# untrusted DATA, passed through env / GraphQL variables, never shell-interpolated. +# The `exempt-paths` changed-file lookup reads GitHub-owned metadata (the pulls/{n}/files +# endpoint), NOT the PR branch, so it stays inside those constraints; the filenames it returns +# are still author-controlled strings and are only ever pattern-MATCHED. # The job runs on the DEFAULT branch, so its job-level check must NOT be the required merge # check. Instead validate.sh publishes the `Linear ticket` commit status on the refetched PR # head SHA; branch protection requires THAT context. See docs/callers/linear-ticket.md. @@ -83,6 +86,22 @@ on: type: string required: false default: '' + exempt-paths: + description: >- + Optional comma-separated path patterns; a PR is exempt when EVERY changed file + matches at least one of them (mirroring `paths-ignore` semantics: one config file + plus one code file is NOT exempt). The hatch for config-only chores — dynamic-config + flips, version syncs — that come from many different authors, so neither + `exempt-label` (manual, per PR) nor `exempt-actors` (waives that account's real work + too) fits. Empty (default) disables it, leaving every existing caller byte-identical. + Only `*` (not across `/`) and `**` (across `/`) are wildcards; everything else is + literal, `!` negation is rejected, and empty/duplicate/absolute/`..` entries fail the + run. This is an IN-VALIDATOR exemption, deliberately not a `paths-ignore:` on the + signal workflow: a filtered-out signal publishes no `Linear ticket` status at all, and + a required context that is never published leaves the PR pending forever. + type: string + required: false + default: '' require-open-issue: description: >- When true (default), reject a linked issue whose Linear state.type is @@ -240,6 +259,7 @@ jobs: TEAM_KEYS: ${{ inputs.team-keys }} EXEMPT_LABEL: ${{ inputs.exempt-label }} EXEMPT_ACTORS: ${{ inputs.exempt-actors }} + EXEMPT_PATHS: ${{ inputs.exempt-paths }} REQUIRE_OPEN_ISSUE: ${{ inputs.require-open-issue }} ENFORCE: ${{ inputs.enforce }} SOFT_FAIL: ${{ inputs.soft-fail }} diff --git a/README.md b/README.md index 062992a1..0010f6cb 100644 --- a/README.md +++ b/README.md @@ -15,7 +15,7 @@ complete, copy-pasteable caller. | Workflow | Purpose | Setup | |---|---|---| | [`detect-unreviewed-merge.yml`](.github/workflows/detect-unreviewed-merge.yml) | SOC 2 compliance — detects PRs merged without prior approval and opens a tracking issue in [`Comfy-Org/unreviewed-merges`](https://github.com/Comfy-Org/unreviewed-merges). | [detect-unreviewed-merge.md](docs/callers/detect-unreviewed-merge.md) | -| [`linear-ticket.yml`](.github/workflows/linear-ticket.yml) | **Requires a Linear issue that Linear has LINKED to the exact PR when the PR targets a protected base branch** — not a `TEAM-123`-shaped string in author text. The validator reads GitHub's branch `protected` property, so any number of branches protected by branch protection or rulesets are covered without a caller-maintained list; unprotected targets publish no `Linear ticket` status and do not query Linear (a success status would be unsafe because GitHub scopes statuses to a SHA, not a PR). For protected targets, the only thing that passes is an attachment Linear returns for the PR's canonical `html_url` (`attachmentsForURL`, PR URL passed as a GraphQL **variable**) whose issue satisfies the configured team (`team-keys`, matched on the resolved API `team.key`, never a prefix) and state (`require-open-issue` rejects `completed`/`canceled` via `state.type`) policy; a PR linked to several tickets passes when at least one satisfies policy. Identifiers mined from branch/title/body are **diagnostics only** (capped at 20, one batched lookup) — they explain a red check, never turn it green. Fork- and Dependabot-safe by a two-stage split: an unprivileged, checks-out-nothing **signal** workflow on `pull_request`, then a privileged validator on `workflow_run` that GitHub grants the secret + write token. It publishes the required **`Linear ticket`** commit status on the refetched PR head SHA (its own default-branch job is *not* the merge check), refetches the head SHA and base branch before every terminal write so a superseded or retargeted run can't overwrite a newer result, maintains one marker diagnostic comment, and **fails closed** on unreadable branch protection or Linear auth/schema/timeout/rate-limit errors (an infrastructure failure, never a green pass) with bounded retry for Linear's async-link race. `enforce: false` is warn-only — a failing verdict never exits the job nonzero (a broken run still does), and by default (`soft-fail: true`) it still publishes the **red** status so the pilot is visible on the PR while blocking nothing, since blocking is branch protection requiring the context, not this workflow's call (`soft-fail: false` restores the silent always-green variant); one `exempt-label` (e.g. `linear-exempt`) waives it per-PR, and an opt-in `exempt-actors` list (empty by default) skips bot PRs (e.g. `dependabot[bot]`) with no built-in bypass. Logic + hermetic tests live in [`scripts/linear-ticket/`](scripts/linear-ticket); `workflows_ref` is **required**, shape- and ancestry-checked before checkout. Requires `LINEAR_API_TOKEN` + the two-workflow caller. | [linear-ticket.md](docs/callers/linear-ticket.md) | +| [`linear-ticket.yml`](.github/workflows/linear-ticket.yml) | **Requires a Linear issue that Linear has LINKED to the exact PR when the PR targets a protected base branch** — not a `TEAM-123`-shaped string in author text. The validator reads GitHub's branch `protected` property, so any number of branches protected by branch protection or rulesets are covered without a caller-maintained list; unprotected targets publish no `Linear ticket` status and do not query Linear (a success status would be unsafe because GitHub scopes statuses to a SHA, not a PR). For protected targets, the only thing that passes is an attachment Linear returns for the PR's canonical `html_url` (`attachmentsForURL`, PR URL passed as a GraphQL **variable**) whose issue satisfies the configured team (`team-keys`, matched on the resolved API `team.key`, never a prefix) and state (`require-open-issue` rejects `completed`/`canceled` via `state.type`) policy; a PR linked to several tickets passes when at least one satisfies policy. Identifiers mined from branch/title/body are **diagnostics only** (capped at 20, one batched lookup) — they explain a red check, never turn it green. Fork- and Dependabot-safe by a two-stage split: an unprivileged, checks-out-nothing **signal** workflow on `pull_request`, then a privileged validator on `workflow_run` that GitHub grants the secret + write token. It publishes the required **`Linear ticket`** commit status on the refetched PR head SHA (its own default-branch job is *not* the merge check), refetches the head SHA and base branch before every terminal write so a superseded or retargeted run can't overwrite a newer result, maintains one marker diagnostic comment, and **fails closed** on unreadable branch protection or Linear auth/schema/timeout/rate-limit errors (an infrastructure failure, never a green pass) with bounded retry for Linear's async-link race. `enforce: false` is warn-only — a failing verdict never exits the job nonzero (a broken run still does), and by default (`soft-fail: true`) it still publishes the **red** status so the pilot is visible on the PR while blocking nothing, since blocking is branch protection requiring the context, not this workflow's call (`soft-fail: false` restores the silent always-green variant); one `exempt-label` (e.g. `linear-exempt`) waives it per-PR, and an opt-in `exempt-actors` list (empty by default) skips bot PRs (e.g. `dependabot[bot]`) with no built-in bypass. An opt-in `exempt-paths` list exempts a PR whose changed files **all** match (config-only chores arriving from many different authors); it is decided IN the validator — reading the GitHub-owned changed-file list, failing closed past that endpoint's silent 3000-file cap — precisely so an exempt PR still publishes the green required status, which a `paths-ignore:` on the signal workflow could not (an unpublished required context leaves the PR pending forever). Logic + hermetic tests live in [`scripts/linear-ticket/`](scripts/linear-ticket); `workflows_ref` is **required**, shape- and ancestry-checked before checkout. Requires `LINEAR_API_TOKEN` + the two-workflow caller. | [linear-ticket.md](docs/callers/linear-ticket.md) | | [`cursor-review.yml`](.github/workflows/cursor-review.yml) | Label-triggered multi-model code review. A 3-lab × 2-review-type cursor-agent panel runs adversarial + edge-case passes, a judge model consolidates them into one PR review with per-finding severity badges, and the triggerer gets Slack start/complete DMs. Advisory by default: it posts a review, it does not gate. Opt-in `blocking: true` adds a fail-closed **Blocking gate** check — red while any finding thread is unresolved and non-outdated, and red whenever the round that should have produced those threads did not land (the trigger job failed, the post exited zero without reaching the PR, every finding was demoted to the review body, the diff was over `diff_size_cap`, or the run was cancelled) — which blocks merge once the caller's ruleset marks it required (two switches; see [the setup guide](docs/callers/cursor-review.md)). Prompts and scripts live in [`.github/cursor-review/`](.github/cursor-review) — the single source of truth, so consumer repos carry only a thin caller. Generated code is excluded from BOTH the diff-size gate and the diff sent to the panel via the shared [`check-pr-size`](scripts/check-pr-size) classifier (base-ref `linguist-generated`, base-blob Go generated markers — a PR cannot self-exempt a file from review, lockfiles, plus `extra_generated_globs` / `extra_lockfiles`) — the same detection as `pr-size.yml`, so codegen never blows the review cap or wastes review signal. Blank and comment-only lines are also discounted from the size count by default (`ignore_comments`) so AI comment volume doesn't trip the cap and skip the review — comments still reach the panel, only the count ignores them. The runner and the panel model list are fixed by the reusable — there are no `runs_on` / `models` inputs (`models` is a job *output*). Least privilege by construction: no job both checks out PR code and holds a write-scoped credential — every job that checks out PR code and runs `cursor-agent` over it (every panel cell and the judge) holds `contents: read` only, and hands its result to a separate, checkout-free `Post review` job via an artifact; that job and the over-the-cap comment job are the only holders of `pull-requests: write`, and neither checks out PR code. Requires `CURSOR_API_KEY` (+ optional `SLACK_BOT_TOKEN`). | [cursor-review.md](docs/callers/cursor-review.md) | | [`cursor-review-auto-label.yml`](.github/workflows/cursor-review-auto-label.yml) | Companion to `cursor-review.yml`. On PR assignment, applies the review label for an opted-in reviewer (via the CLOUD_CODE_BOT app token, so the label actually triggers the review). The opt-in roster lives in the caller's `vars.CURSOR_REVIEW_OPTED_IN_LOGINS` — no roster is baked into the workflow. Requires `vars.APP_ID` + `CLOUD_CODE_BOT_PRIVATE_KEY`, or the optional `bot_app_id` + `BOT_APP_PRIVATE_KEY` pair to mint from a narrower App instead. | [cursor-review-auto-label.md](docs/callers/cursor-review-auto-label.md) | | [`assign-reviewers.yml`](.github/workflows/assign-reviewers.yml) | Assigns PR owners using configured paths and recent human approval history. Chooses complementary file coverage before workload, with an explained one-owner default fallback and no random substitutions. Optionally generates a shared history artifact on schedule/manual dispatch and reuses fresh snapshots across PRs. Reads configuration from the base SHA, respects manual assignments, and preserves opt-in `REVIEWER_AUTHOR_ALLOWLIST` scoping. Requires `vars.APP_ID` + `CLOUD_CODE_BOT_PRIVATE_KEY`; callers skip forks and Dependabot triggers. | [assign-reviewers.md](docs/callers/assign-reviewers.md) | diff --git a/docs/callers/linear-ticket.md b/docs/callers/linear-ticket.md index 6eea91f9..bd32605c 100644 --- a/docs/callers/linear-ticket.md +++ b/docs/callers/linear-ticket.md @@ -117,6 +117,8 @@ jobs: # # team scoping — a restricted list fails cross-team PRs # exempt-label: linear-exempt # optional; empty disables exemption # exempt-actors: dependabot[bot],renovate[bot] # optional; bot PRs skip the check + # exempt-paths: infrastructure/dynamicconfig/** # optional; a PR whose changed files + # # ALL match is exempt. Empty disables it. # require-open-issue: true # default enforce: false # WARN-ONLY during rollout; flip to true when ready # soft-fail: true # default; warn-only still shows a RED (non-blocking) @@ -154,10 +156,81 @@ job-level detail. | `team-keys` | `''` | Comma-separated allow-list matched against the resolved issue's API `team.key`, never an identifier prefix. **Leave empty (recommended)** unless a repo genuinely wants team scoping — a restricted list fails a cross-team PR on first contact. Malformed or duplicate entries fail the run. | | `exempt-label` | `''` | Single label that waives the requirement (recommended `linear-exempt`). Empty disables exemption. | | `exempt-actors` | `''` | Comma-separated PR-author logins whose PRs skip the check without a label (e.g. `dependabot[bot],renovate[bot]`) — the non-manual hatch for bot PRs that never carry a ticket. Opt-in; empty means no bypass. Listing an actor means **all** of its PRs merge without a Linear ticket, so keep it to trusted automation accounts. | +| `exempt-paths` | `''` | Comma-separated path patterns. A PR is exempt when **every** changed file matches at least one pattern — one config file plus one code file is **not** exempt. The hatch for config-only chores (dynamic-config flips, version syncs) that arrive from many different authors, where `exempt-label` is manual per-PR and `exempt-actors` would waive the gate on all of that account's real work too. Empty (the default) disables it. Only `*` (does not cross `/`) and `**` (crosses `/`) are wildcards; everything else is literal. `!` negation and empty, duplicate, absolute or `..` entries fail the run. See [Why `exempt-paths` is an in-validator exemption](#why-exempt-paths-is-an-in-validator-exemption-and-not-a-paths-ignore-filter). | | `require-open-issue` | `true` | Reject a linked issue whose Linear `state.type` is `completed`/`canceled`. `backlog`/`unstarted`/`started`/`triage` pass. | | `enforce` | `true` | `false` is warn-only: a failing **verdict** never exits the job nonzero. `soft-fail` decides how loudly it is reported; the diagnosis is identical either way. Warn-only is not a promise the job always exits `0` — a broken *run* still does (a failed terminal status write, a missing token/repo, malformed `team-keys`, a non-`pull_request` trigger). | | `soft-fail` | `true` | Warn-only only (ignored when `enforce: true`). `true` publishes a **red `failure`** status, so the PR's check list shows the check failing — loud, but non-blocking for as long as `Linear ticket` is not a required status in your ruleset. `false` restores the silent variant (warn-only publishes `success`; only the summary and comment carry the verdict). **Do not** require the `Linear ticket` context while `enforce: false` — required + soft-fail would block merges. | +## Why `exempt-paths` is an in-validator exemption and not a `paths-ignore` filter + +The obvious way to stop a check running on config-only PRs is a `paths-ignore:` on the caller's +**signal** workflow. **Do not do that.** It is the same required-check trap the Gotchas section +below describes from the other direction. + +`Linear ticket` is a commit *status*, and the whole point of the rollout is that your ruleset +eventually **requires** that context. A `paths-ignore:` stops the signal workflow firing, so the +validator never runs, so no `Linear ticket` status is ever published for that PR — and a required +status context that is never published does not "pass by omission". GitHub leaves it **pending +forever**, which blocks the merge exactly as a failure would, for precisely the PRs you were +trying to let through. The same reasoning is why an unprotected base branch is the *only* case +this workflow declines to publish a status at all: those PRs are outside the gate entirely, and +their base branch is not one your ruleset protects. + +So the exemption is decided *inside* the validator, and an exempt PR takes the same +`finish_exempt()` exit the label and actor hatches take: a **green** `Linear ticket` status, a +job summary naming the reason, and no marker comment. The status is published, the required +context is satisfied, and the audit trail says why. + +Three more properties worth knowing before you set it: + +- **It short-circuits before Linear is queried.** An exempt PR spends no Linear API budget and + passes even while Linear is unreachable — same as `exempt-label` and `exempt-actors`. +- **It costs one extra GitHub API lookup**, `GET /repos/{owner}/{repo}/pulls/{number}/files` — + paginated at 100 files per page, so it is *one request per page*, not one request for the + whole list (a 250-file PR costs 3 requests; an accepted PR just under the 3000-file cap costs + 30). That is GitHub-owned metadata, not PR content — nothing is checked out, and the + filenames it returns are treated as untrusted data and only pattern-matched. The + `pull-requests: write` you already grant covers it; no new permission is needed. +- **It fails closed when the file list cannot be trusted.** That endpoint caps at **3000 files** + and truncates *silently*, so a PR at or past the cap — or any failed/malformed read — reports + a red `Could not verify — the PR's changed-file list is unreadable`, rather than declaring a + partial list fully matched. In practice this means a >3000-file PR in a repo that sets + `exempt-paths` fails the check as an infrastructure error even if it carries a perfectly good + linked ticket; that is deliberate (a run must not publish a verdict it did not reach), and it + is one more reason not to set `exempt-paths` in a repo that does not need it. + +### Pattern grammar + +A deliberate **subset** of GitHub's filter-pattern grammar, small enough to read at a glance: + +| | | +|---|---| +| `**` | zero or more of any character, **including** `/` | +| `*` | zero or more characters, **excluding** `/` | +| everything else | literal — `?`, `[]`, `+` and `.` match themselves | + +GitHub's own `paths:` grammar additionally reads `?` and `+` as postfix *quantifiers* on the +preceding character, which is the opposite of what shell-glob habits expect. A half-remembered +quantifier inside a merge-gate waiver is how a pattern quietly matches more than its author +meant, so those are literal here instead. Every limitation of this grammar under-matches, and +under-matching leaves the ticket requirement **in force** — the safe direction. + +Two consequences: + +- `**` is translated positionally, so `a/**/b.yaml` requires both slashes literally and does + **not** match `a/b.yaml`. For a directory exemption write `a/**`, which is what you want. +- `infrastructure/dynamicconfig/**` does not match `infrastructure/dynamicconfig-backup/x.yaml` + (the `/` before `**` is literal) and does not match the bare directory path either. + +A **rename** contributes both its new path and its `previous_filename`, so moving a code file +into an exempted directory does not buy the exemption. A PR with **zero** changed files is never +exempt — "every file matches" must not be vacuously true on an empty or fully-reverted branch. + +> [!WARNING] +> Nothing stops you writing `exempt-paths: **`, and nothing will tell you afterwards: it matches +> every path, so **every** PR reports green and the gate is off while still looking configured. +> Scope patterns to the directory the ticket-less chores actually live in. + ## Upgrading an existing `enforce: false` caller `soft-fail` is new, and it defaults to `true`. **If your caller already runs `enforce: false`, @@ -214,7 +287,11 @@ skewed apart, so an older `linear-ticket.yml` is running newer scripts — is tr `repository_dispatch` driven by a Linear webhook — not a larger retry budget — and is a deliberate v2 follow-up, out of scope here. 4. Decide bot-PR handling: set `exempt-actors` (e.g. `dependabot[bot],renovate[bot]`) so - dependency PRs pass without hand-labelling each one. + dependency PRs pass without hand-labelling each one. If the warn-only week also surfaces a + recurring stream of ticket-less **config-only** chores from *many different authors* — the + case `exempt-actors` cannot express without waiving the gate on all of those accounts' real + work — set `exempt-paths` for the directory those chores live in, and confirm one such PR + shows a green `Linear ticket` naming the exemption before you move on. 5. Flip `enforce: true`. Leave the check non-required for a short window — the PR-visible result is unchanged from the soft-fail pilot; what changes is that the validator's own run now goes red too, so a systemic breakage is visible in the Actions tab. diff --git a/scripts/linear-ticket/README.md b/scripts/linear-ticket/README.md index 553cecc8..c48a62e8 100644 --- a/scripts/linear-ticket/README.md +++ b/scripts/linear-ticket/README.md @@ -30,6 +30,36 @@ author types is not a link. So: - "Any issue remains" is the pass rule: a PR linked to several tickets passes when at least one linked issue satisfies policy. +The three **exemptions** (`exempt-label`, `exempt-actors`, `exempt-paths`, all opt-in and all +empty by default) are the only other way to a green check. They short-circuit *before* the +Linear query — so an exempt PR spends no token budget and does not depend on Linear being +reachable — and every one of them exits through the shared `finish_exempt()`, which PUBLISHES a +green `Linear ticket` status rather than skipping it. That is the whole point of doing the path +exemption here instead of as a `paths-ignore:` on the consumer's signal workflow: a filtered-out +signal publishes no status at all, and a required context that is never published leaves the PR +pending forever. + +`exempt-paths` specifically (`lib.parse_path_list` / `lib.all_paths_exempt`): + +- Exempt only when **every** changed path matches at least one pattern — `paths-ignore` + semantics. One config file plus one Go file is not exempt. +- **Zero** changed paths is NOT exempt: "every path matches" is vacuously true over an empty + list, and that list is reachable (an empty or fully-reverted branch). +- The pattern grammar is a deliberate SUBSET of GitHub's: `**` crosses `/`, `*` does not, + everything else is literal. `!` negation and empty/duplicate/absolute/`..` entries fail the + run. Every limitation under-matches, and under-matching leaves the requirement in force. +- The changed-file list comes from `GET /repos/{owner}/{repo}/pulls/{number}/files` + (GitHub-owned metadata, paginated at 100/page; no PR branch is checked out). Filenames are + author-controlled strings and are only ever pattern-matched. A **rename** contributes both + its new path and its `previous_filename`, so code cannot be moved into an exempt directory to + buy the exemption. +- That endpoint caps at **3000 files and truncates silently**, so a read at or past the cap — + or a failed/malformed one — raises `ChangedFilesUnavailable` and fails the check closed as + `changed_files_unavailable`, never as a partial list declared fully matched. "Malformed" + includes a length that **disagrees with the PR payload's own `changed_files` count** (a list + that is merely short reads exactly like a complete one once the dropped entry is the + non-exempt path) and any entry that is not an object carrying string paths. + ## Security & failure model - Runs in the **privileged `workflow_run` job**; every PR-derived value is untrusted DATA diff --git a/scripts/linear-ticket/lib.py b/scripts/linear-ticket/lib.py index 5eed5d7d..bbe3d3ab 100644 --- a/scripts/linear-ticket/lib.py +++ b/scripts/linear-ticket/lib.py @@ -20,6 +20,7 @@ from __future__ import annotations +import functools import re from typing import NamedTuple @@ -97,6 +98,101 @@ def parse_actor_list(raw: str) -> list[str]: return [part.strip().lower() for part in (raw or "").split(",") if part.strip()] +# ── path exemption (`exempt-paths`) ───────────────────────────────────────────────────── +# A DELIBERATE SUBSET of GitHub's filter-pattern grammar. Only `*` and `**` are wildcards: +# +# ** zero or more of any character, INCLUDING `/` +# * zero or more characters, EXCLUDING `/` +# +# Everything else — `?`, `[]`, `+`, `.` — is matched LITERALLY. GitHub's own `paths` grammar +# additionally reads `?` and `+` as postfix quantifiers on the preceding character, which is +# the opposite of what a reader who knows shell globbing expects; supporting a half-remembered +# quantifier in a MERGE-GATE WAIVER is how a pattern silently matches more than its author +# meant. So the grammar here is small enough to hold in your head, and every limitation in it +# fails toward NOT exempt — an under-matching pattern leaves the ticket requirement in force, +# which is the safe direction. `!` negation is rejected outright rather than matched literally, +# because treating `!foo` as a literal path would invert the author's intent in silence. +# +# Consequence worth knowing: `**` is translated positionally, so `a/**/b.yaml` requires both +# slashes literally and does NOT match `a/b.yaml`. Prefer the `a/**` form, which is what a +# directory exemption actually wants. +_PATH_PATTERN_TOKEN_RE = re.compile(r"\*\*|\*|[^*]+") + + +def parse_path_list(raw: str) -> list[str]: + """Parse the caller's comma-separated ``exempt-paths`` input into path patterns. + + Empty/whitespace-only input -> ``[]``, which DISABLES the path exemption entirely (the + default: every existing caller behaves exactly as it did before this input existed). + + Raises ValueError — rejecting the whole run as a caller misconfiguration — on an empty + entry, a duplicate, an absolute pattern, a `..` segment, or a leading `!`. Note this is + STRICTER than ``normalize_team_keys``, which tolerates a stray ``BE,,ENG`` empty field: + an empty entry there narrows nothing, whereas this input is a merge-gate WAIVER and a + stray comma is the shape of a half-finished edit. A waiver list runs under exactly what + the caller wrote or it does not run at all. + """ + if not (raw or "").strip(): + return [] + patterns: list[str] = [] + for part in raw.split(","): + pattern = part.strip() + if not pattern: + raise ValueError("empty path pattern (a stray or trailing comma)") + if pattern.startswith("!"): + raise ValueError( + f"negated path pattern is not supported: {pattern!r}") + if pattern.startswith("/"): + raise ValueError( + f"path pattern must be repo-relative, not absolute: {pattern!r}") + if ".." in pattern.split("/"): + raise ValueError(f"path pattern must not contain a '..' segment: {pattern!r}") + if pattern in patterns: + raise ValueError(f"duplicate path pattern: {pattern!r}") + patterns.append(pattern) + return patterns + + +@functools.lru_cache(maxsize=256) +def _compile_path_pattern(pattern: str) -> re.Pattern: + """Translate one pattern into an anchored regex. Cached: a PR can carry 1000s of files.""" + parts = [] + for token in _PATH_PATTERN_TOKEN_RE.findall(pattern): + if token == "**": + parts.append(".*") + elif token == "*": + parts.append("[^/]*") + else: + parts.append(re.escape(token)) + return re.compile("".join(parts)) + + +def path_matches_any(path: str, patterns: list[str]) -> bool: + """True iff ``path`` matches at least one pattern, whole-string.""" + if not path: + return False + return any(_compile_path_pattern(p).fullmatch(path) for p in (patterns or [])) + + +def all_paths_exempt(paths: list[str], patterns: list[str]) -> bool: + """THE PATH EXEMPTION. True iff EVERY changed path matches at least one pattern. + + Mirrors `paths-ignore` semantics: a PR touching one config file and one Go file is NOT + exempt, because the Go file alone is the kind of change the ticket gate exists for. + + Two ways this deliberately returns False: + + * no patterns — the exemption is disabled (the empty default), so nothing is exempt. + * no paths — "every path matches" is VACUOUSLY true over an empty list, and an empty + list is reachable (an empty branch, a fully-reverted PR, and, if a caller + ever loses the over-cap guard in validate.py, a truncated read). Waiving a + merge gate on a vacuous truth is not a waiver anyone asked for. + """ + if not patterns or not paths: + return False + return all(path_matches_any(path, patterns) for path in paths) + + def filter_issues(nodes: list[dict], team_keys: list[str], require_open: bool) -> list[str]: """THE GATE. Identifiers of the linked issues that satisfy policy, sorted unique. @@ -166,6 +262,11 @@ def select_failure_category(infra_error: bool, linked: int, referenced: int) -> * exists_not_linked — no link, but an identifier was REFERENCED in branch/title/body. * no_candidate — no link and no identifier referenced anywhere. + NOT every category lives here: ``changed_files_unavailable`` is decided by validate.py + before Linear is queried at all (the path exemption could not be evaluated), so it is + passed straight to ``finish_fail`` and never flows through this selector. It still needs + copy in every table below, which is what the surface-coverage test pins. + The boundary between the last two is whether an identifier was referenced at all; the batched diagnostic lookup (count_resolved_candidates) only enriches the DETAIL line, it does not move the category — so the copy can never say "no identifier detected" while the @@ -200,6 +301,15 @@ def select_failure_category(infra_error: bool, linked: int, referenced: int) -> "team this check does not accept. Link an issue from an accepted team that is not " "closed, or move the existing issue back to an open state." ), + "changed_files_unavailable": ( + "This check could not be completed because the PR's changed-file list could not be " + "read completely, and this repository configures a path-based exemption " + "(`exempt-paths`) that is decided from that list. A partial list could waive the " + "ticket requirement for a PR that actually carries code, so it fails closed instead. " + "This is an infrastructure error, not a verdict on your ticket. Re-run the check; a " + "PR whose changed-file count is past GitHub's 3000-file API ceiling will keep failing " + "until it is split, so contact the repository owners if that is what you are seeing." + ), "infra_error": ( "This check could not be completed because Linear could not be queried " "(authentication, schema, timeout, or rate-limit exhaustion). This is an " @@ -256,6 +366,7 @@ def failure_guidance(category: str) -> str: "exists_not_linked": "Referenced Linear issue is not linked yet", "policy_mismatch": "Linked Linear issue fails this repo's policy", "infra_error": "Could not verify — Linear could not be queried", + "changed_files_unavailable": "Could not verify — the PR's changed-file list is unreadable", } # Appended to the marker comment whenever the check is red but warn-only, so nobody reads the @@ -283,6 +394,14 @@ def failure_guidance(category: str) -> str: " This run could not reach Linear, so there is nothing to fix on the PR itself: re-run " "the check, and if it keeps failing contact the repository owners." ), + # Same shape: the changed-file read is repository infrastructure, and the one case the + # author CAN act on (a PR past the 3000-file ceiling) is not fixed by linking a ticket + # either — it is fixed by splitting the PR. + "changed_files_unavailable": ( + " This run could not read the PR's changed-file list, so there is nothing to fix by " + "linking a ticket: re-run the check, and if it keeps failing contact the repository " + "owners (a PR past GitHub's 3000-file API ceiling has to be split)." + ), } _ADVISORY_TAIL_DEFAULT = ( " Linking the ticket now is what keeps this from blocking you once the pilot starts " diff --git a/scripts/linear-ticket/tests/test_lib.py b/scripts/linear-ticket/tests/test_lib.py index 5b218e9b..4c1d31f2 100644 --- a/scripts/linear-ticket/tests/test_lib.py +++ b/scripts/linear-ticket/tests/test_lib.py @@ -87,6 +87,118 @@ def _issue(identifier, key, state): return {"issue": {"identifier": identifier, "team": {"key": key}, "state": {"type": state}}} +class ParsePathList(unittest.TestCase): + def test_empty_disables_the_exemption(self): + self.assertEqual(lib.parse_path_list(""), []) + self.assertEqual(lib.parse_path_list(" "), []) + self.assertEqual(lib.parse_path_list(None), []) + + def test_trims_and_preserves_order_and_case(self): + # Paths are case-SENSITIVE on GitHub, unlike the logins parse_actor_list lowercases. + self.assertEqual( + lib.parse_path_list(" infrastructure/dynamicConfig/** , docs/*.md "), + ["infrastructure/dynamicConfig/**", "docs/*.md"], + ) + + def test_empty_entry_rejected(self): + # Stricter than normalize_team_keys, which tolerates "BE,,ENG": this input is a + # merge-gate waiver, so a stray/trailing comma fails the run rather than running under + # a list the caller did not quite write. + for raw in ("a/**,,b/**", "a/**,", ",a/**", "a/**, ,b/**"): + with self.assertRaises(ValueError, msg=raw): + lib.parse_path_list(raw) + + def test_duplicate_rejected(self): + with self.assertRaises(ValueError): + lib.parse_path_list("a/**,a/**") + + def test_negation_rejected(self): + # Matching `!foo` literally would invert the author's intent in silence. + with self.assertRaises(ValueError): + lib.parse_path_list("!secret/**") + + def test_absolute_rejected(self): + with self.assertRaises(ValueError): + lib.parse_path_list("/infrastructure/**") + + def test_parent_segment_rejected(self): + with self.assertRaises(ValueError): + lib.parse_path_list("a/../../etc/**") + + def test_dotdot_inside_a_filename_is_fine(self): + # Only a whole `..` SEGMENT is rejected; "a..b" is a legal file name. + self.assertEqual(lib.parse_path_list("a..b/**"), ["a..b/**"]) + + +class PathMatching(unittest.TestCase): + def test_double_star_crosses_slashes(self): + patterns = ["infrastructure/dynamicconfig/**"] + self.assertTrue(lib.path_matches_any("infrastructure/dynamicconfig/prod.yaml", patterns)) + self.assertTrue( + lib.path_matches_any("infrastructure/dynamicconfig/env/prod/a.yaml", patterns)) + self.assertFalse(lib.path_matches_any("infrastructure/other/prod.yaml", patterns)) + + def test_double_star_does_not_match_a_sibling_prefix(self): + # The `/` before `**` is literal, so the pattern cannot leak into `...config-backup/`. + patterns = ["infrastructure/dynamicconfig/**"] + self.assertFalse( + lib.path_matches_any("infrastructure/dynamicconfig-backup/prod.yaml", patterns)) + self.assertFalse(lib.path_matches_any("infrastructure/dynamicconfig", patterns)) + + def test_single_star_does_not_cross_slashes(self): + self.assertTrue(lib.path_matches_any("docs/readme.md", ["docs/*.md"])) + self.assertFalse(lib.path_matches_any("docs/nested/readme.md", ["docs/*.md"])) + + def test_whole_string_match_not_substring(self): + self.assertFalse(lib.path_matches_any("vendor/docs/readme.md", ["docs/*.md"])) + self.assertFalse(lib.path_matches_any("docs/readme.md.bak", ["docs/*.md"])) + + def test_dot_is_literal_not_a_regex_wildcard(self): + self.assertFalse(lib.path_matches_any("docs/readmeXmd", ["docs/*.md"])) + + def test_question_mark_and_brackets_are_literal(self): + # A deliberate subset of GitHub's grammar: unsupported metacharacters match literally, + # which under-matches — and under-matching leaves the ticket requirement in force. + self.assertTrue(lib.path_matches_any("a?b.yaml", ["a?b.yaml"])) + self.assertFalse(lib.path_matches_any("axb.yaml", ["a?b.yaml"])) + self.assertTrue(lib.path_matches_any("a[1].yaml", ["a[1].yaml"])) + self.assertFalse(lib.path_matches_any("a1.yaml", ["a[1].yaml"])) + + def test_any_pattern_may_match(self): + patterns = ["config/**", "docs/*.md"] + self.assertTrue(lib.path_matches_any("config/a/b.yaml", patterns)) + self.assertTrue(lib.path_matches_any("docs/x.md", patterns)) + + def test_no_patterns_and_empty_path_never_match(self): + self.assertFalse(lib.path_matches_any("config/a.yaml", [])) + self.assertFalse(lib.path_matches_any("", ["**"])) + + +class AllPathsExempt(unittest.TestCase): + PATTERNS = ["infrastructure/dynamicconfig/**"] + + def test_all_matching_is_exempt(self): + self.assertTrue(lib.all_paths_exempt( + ["infrastructure/dynamicconfig/prod.yaml", + "infrastructure/dynamicconfig/staging.yaml"], + self.PATTERNS)) + + def test_one_unmatched_file_is_not_exempt(self): + # The headline case: a config flip that also carries a Go file still needs a ticket. + self.assertFalse(lib.all_paths_exempt( + ["infrastructure/dynamicconfig/prod.yaml", "services/api/handler.go"], + self.PATTERNS)) + + def test_zero_files_is_not_exempt(self): + # "Every path matches" is VACUOUSLY true over an empty list; waiving a merge gate on a + # vacuous truth is not a waiver anyone asked for. + self.assertFalse(lib.all_paths_exempt([], self.PATTERNS)) + + def test_no_patterns_is_not_exempt(self): + self.assertFalse(lib.all_paths_exempt(["infrastructure/dynamicconfig/prod.yaml"], [])) + self.assertFalse(lib.all_paths_exempt([], [])) + + class FilterIssues(unittest.TestCase): LINKED_OPEN_BE = [_issue("BE-1", "BE", "started")] LINKED_DONE_BE = [_issue("BE-1", "BE", "completed")] @@ -164,9 +276,13 @@ def test_nothing_is_no_candidate(self): self.assertEqual(lib.select_failure_category(False, 0, 0), "no_candidate") +ALL_CATEGORIES = ("no_candidate", "exists_not_linked", "policy_mismatch", "infra_error", + "changed_files_unavailable") + + class FailureGuidance(unittest.TestCase): def test_every_category_non_empty(self): - for category in ("no_candidate", "exists_not_linked", "policy_mismatch", "infra_error"): + for category in ALL_CATEGORIES: self.assertTrue(lib.failure_guidance(category).strip()) def test_unknown_category_raises(self): @@ -174,9 +290,6 @@ def test_unknown_category_raises(self): lib.failure_guidance("bogus") -ALL_CATEGORIES = ("no_candidate", "exists_not_linked", "policy_mismatch", "infra_error") - - class StatusContext(unittest.TestCase): def test_context_is_the_human_facing_name_branch_protection_requires(self): # Pinned deliberately. GitHub renders this verbatim as the check's name in the PR's @@ -335,6 +448,14 @@ def test_infra_error_does_not_tell_the_author_to_link_a_ticket(self): self.assertNotIn("Linking the ticket", note) self.assertIn("could not reach Linear", note) + def test_changed_files_unavailable_does_not_tell_the_author_to_link_a_ticket(self): + # An unreadable changed-file list is repository infrastructure, and the one case the + # author CAN act on (a PR past the 3000-file ceiling) is fixed by splitting the PR, + # not by linking a ticket. + note = lib.advisory_note("changed_files_unavailable") + self.assertNotIn("Linking the ticket", note) + self.assertIn("changed-file list", note) + def test_ticket_categories_do_tell_the_author_to_link(self): for category in ("no_candidate", "exists_not_linked", "policy_mismatch"): self.assertIn("Linking the ticket", lib.advisory_note(category)) diff --git a/scripts/linear-ticket/tests/test_validate.py b/scripts/linear-ticket/tests/test_validate.py index f4678917..48f49b9d 100644 --- a/scripts/linear-ticket/tests/test_validate.py +++ b/scripts/linear-ticket/tests/test_validate.py @@ -1,8 +1,11 @@ """Hermetic orchestration tests for validate.py.""" +import json import os import sys +import tempfile import unittest +from unittest import mock sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) @@ -12,27 +15,48 @@ class FakeGitHub: repo = "Comfy-Org/example" - def __init__(self, protected): + def __init__(self, protected, changed_files=None, changed_files_error=None, + declared_changed_files=None, head_ref="feature/be-123"): self.protected = protected + self.head_ref = head_ref self.current_base = "release/next" self.statuses = [] self.deleted_comments = [] + self.comments = [] + self._changed_files = changed_files + self._changed_files_error = changed_files_error + self.declared_changed_files = declared_changed_files + self.changed_files_calls = [] def get(self, path, *, paginate=False): if path.endswith("/pulls/17"): - return { + pr = { "number": 17, "state": "open", "html_url": "https://github.com/Comfy-Org/example/pull/17", - "head": {"sha": "abc123", "ref": "feature/be-123"}, + "head": {"sha": "abc123", "ref": self.head_ref}, "base": {"ref": "release/next"}, "title": "Change something", "body": "", "labels": [], "user": {"login": "octocat"}, } + if self.declared_changed_files is not None: + pr["changed_files"] = self.declared_changed_files + return pr raise AssertionError(f"unexpected GitHub request: {path}") + def changed_files(self, pr, declared_count=None): + self.changed_files_calls.append((pr, declared_count)) + if self._changed_files_error is not None: + raise validate.ChangedFilesUnavailable(self._changed_files_error) + if self._changed_files is None: + raise AssertionError("changed_files must not be consulted") + return list(self._changed_files) + + def upsert_marker_comment(self, pr, body): + self.comments.append((pr, body)) + def branch_is_protected(self, branch): self.requested_branch = branch return self.protected @@ -108,6 +132,252 @@ def test_retargeted_pr_does_not_publish_stale_terminal_status(self): self.assertEqual(github.deleted_comments, []) +class PathExemption(unittest.TestCase): + """The `exempt-paths` short-circuit, which runs after the label/actor hatches and BEFORE + the Linear query — so an exempt PR spends no Linear budget and survives a Linear outage.""" + + PATTERNS = ["infrastructure/dynamicconfig/**"] + + def setUp(self): + # The suite is hermetic (test-linear-ticket.yml says so). The diagnostic lookup in + # _diagnose_and_fail is the one path that reaches Linear without going through + # _query_attachments, so pin it shut rather than relying on every case stubbing it. + patcher = mock.patch.object( + validate, "linear_post", + side_effect=AssertionError("no test in this suite may reach the network")) + patcher.start() + self.addCleanup(patcher.stop) + + def validator(self, github, exempt_paths): + return validate.Validator(github, "token", [], True, True, False, "run-url", + exempt_paths) + + def never_query_linear(self, validator): + validator._query_attachments = lambda _url: self.fail("Linear must not be queried") + + def test_all_changed_files_matching_is_exempt_and_green(self): + github = FakeGitHub(protected=True, changed_files=[ + "infrastructure/dynamicconfig/prod.yaml", + "infrastructure/dynamicconfig/staging.yaml", + ]) + validator = self.validator(github, self.PATTERNS) + self.never_query_linear(validator) + + self.assertEqual(validator.run(event()), 0) + + # Green via the shared finish_exempt path: pending, then a SUCCESS status naming the + # reason. It must publish, not skip — a required-but-unpublished context never merges. + self.assertEqual([status[1] for status in github.statuses], ["pending", "success"]) + self.assertIn("exempt-paths", github.statuses[-1][2]) + self.assertTrue(github.statuses[-1][3], "the terminal write must be critical") + self.assertEqual(github.deleted_comments, [17]) + self.assertEqual(github.comments, []) + + def test_one_unmatched_file_is_not_exempt_and_falls_through_to_linear(self): + github = FakeGitHub(protected=True, changed_files=[ + "infrastructure/dynamicconfig/prod.yaml", + "services/api/handler.go", + ]) + validator = self.validator(github, self.PATTERNS) + queried = [] + validator._query_attachments = lambda url: ( + queried.append(url) or + ([{"issue": {"identifier": "BE-1", "team": {"key": "BE"}, + "state": {"type": "started"}}}], False) + ) + + self.assertEqual(validator.run(event()), 0) + + self.assertEqual(len(queried), 1) + self.assertIn("BE-1", github.statuses[-1][2]) + + def test_zero_changed_files_is_not_exempt(self): + # An empty or fully-reverted branch: "all files match" must not be vacuously true. + # A branch with no identifier in it, so the red path needs no diagnostic lookup. + github = FakeGitHub(protected=True, changed_files=[], head_ref="revert-everything") + validator = self.validator(github, self.PATTERNS) + validator._query_attachments = lambda _url: ([], False) + + self.assertEqual(validator.run(event()), 1) # enforce=True, no linked issue + + self.assertEqual(github.statuses[-1][1], "failure") + self.assertNotIn("exempt", github.statuses[-1][2].lower()) + + def test_empty_input_never_looks_up_changed_files(self): + # The default. Every existing caller must behave byte-identically — including making + # no extra GitHub API call. + github = FakeGitHub(protected=True) # changed_files=None -> asserts if consulted + validator = self.validator(github, []) + validator._query_attachments = lambda _url: ( + [{"issue": {"identifier": "BE-1", "team": {"key": "BE"}, + "state": {"type": "started"}}}], False) + + self.assertEqual(validator.run(event()), 0) + self.assertEqual(github.changed_files_calls, []) + + def test_unreadable_changed_file_list_fails_closed_without_querying_linear(self): + github = FakeGitHub(protected=True, + changed_files_error="the PR reports 4000 changed files, at or " + "past GitHub's 3000-file ceiling") + validator = self.validator(github, self.PATTERNS) + self.never_query_linear(validator) + + self.assertEqual(validator.run(event()), 1) + + self.assertEqual(github.statuses[-1][1], "failure") + self.assertIn("changed-file list", github.statuses[-1][2]) + self.assertNotIn("exempt", github.statuses[-1][2].lower()) + # The marker comment must name the category so the red check is triageable. + self.assertEqual(len(github.comments), 1) + self.assertIn("changed_files_unavailable", github.comments[0][1]) + self.assertIn("3000-file ceiling", github.comments[0][1]) + + def test_declared_count_is_handed_to_the_lookup(self): + github = FakeGitHub(protected=True, changed_files=["infrastructure/dynamicconfig/a"], + declared_changed_files=1) + validator = self.validator(github, self.PATTERNS) + self.never_query_linear(validator) + + self.assertEqual(validator.run(event()), 0) + self.assertEqual(github.changed_files_calls, [(17, 1)]) + + +class ChangedFilesLookup(unittest.TestCase): + """validate.GitHub.changed_files — pagination and the 3000-file ceiling.""" + + def github(self, payload): + gh = validate.GitHub("Comfy-Org/example") + gh.requests = [] + gh.get = lambda path, *, paginate=False: ( + gh.requests.append((path, paginate)) or payload) + return gh + + def test_paginates_past_one_page(self): + # gh api --paginate merges every page's top-level array into one list, so the unit + # under test is what changed_files does with a merged list longer than a single page. + payload = [{"filename": f"infrastructure/dynamicconfig/f{i}.yaml"} for i in range(250)] + gh = self.github(payload) + + paths = gh.changed_files(17) + + self.assertEqual(len(paths), 250) + self.assertEqual(paths[-1], "infrastructure/dynamicconfig/f249.yaml") + self.assertEqual(gh.requests, + [("/repos/Comfy-Org/example/pulls/17/files?per_page=100", True)]) + + def test_rename_contributes_both_paths(self): + # Renaming a Go file INTO an exempt directory must not buy the exemption. + gh = self.github([{"filename": "infrastructure/dynamicconfig/handler.go", + "previous_filename": "services/api/handler.go"}]) + + self.assertEqual(gh.changed_files(17), + ["infrastructure/dynamicconfig/handler.go", + "services/api/handler.go"]) + + def test_declared_count_past_the_cap_raises_before_any_request(self): + gh = self.github([]) + with self.assertRaises(validate.ChangedFilesUnavailable) as caught: + gh.changed_files(17, declared_count=validate.CHANGED_FILES_CAP + 1) + self.assertIn("3000", str(caught.exception)) + self.assertEqual(gh.requests, [], "no request should be spent on a known-over-cap PR") + + def test_declared_count_exactly_at_the_cap_raises(self): + # At the cap the API returns everything it will ever return, and cannot say whether + # that is the whole truth — indistinguishable from a truncated read. + gh = self.github([]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17, declared_count=validate.CHANGED_FILES_CAP) + + def test_returned_list_at_the_cap_raises_even_without_a_declared_count(self): + # The belt to the declared count's braces: a PR payload missing `changed_files` must + # not slip a silently truncated list past the guard. + payload = [{"filename": f"c/f{i}.yaml"} for i in range(validate.CHANGED_FILES_CAP)] + gh = self.github(payload) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + def test_malformed_declared_count_degrades_to_the_list_guard(self): + # A non-int `changed_files` must not raise TypeError out of a privileged job; the + # returned-list guard still covers the truncation case. + gh = self.github([{"filename": "c/a.yaml"}]) + self.assertEqual(gh.changed_files(17, declared_count="lots"), ["c/a.yaml"]) + + def test_failed_request_raises_rather_than_returning_empty(self): + # get() returns None on failure; an empty list would read as "zero changed files". + gh = self.github(None) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + def test_entry_without_a_filename_raises(self): + gh = self.github([{"filename": "c/a.yaml"}, {"status": "modified"}]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + def test_short_list_against_the_declared_count_raises(self): + # THE dangerous direction: the dropped entry is exactly the non-exempt path, so every + # survivor matches and a partial read would publish a waiver. + gh = self.github([{"filename": "c/a.yaml"}, {"filename": "c/b.yaml"}]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17, declared_count=3) + + def test_surplus_list_against_the_declared_count_raises(self): + gh = self.github([{"filename": "c/a.yaml"}, {"filename": "c/b.yaml"}]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17, declared_count=1) + + def test_matching_declared_count_is_accepted(self): + # A rename yields two paths from one entry, so the count is compared against ENTRIES, + # never against the expanded path list. + gh = self.github([{"filename": "c/a.yaml", "previous_filename": "old/a.yaml"}]) + self.assertEqual(gh.changed_files(17, declared_count=1), ["c/a.yaml", "old/a.yaml"]) + + def test_non_object_entry_raises_rather_than_attributeerror(self): + gh = self.github([{"filename": "c/a.yaml"}, "c/b.yaml"]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + def test_non_string_filename_raises_before_it_reaches_the_matcher(self): + # lib.path_matches_any calls re.fullmatch(), which raises TypeError on a non-string. + gh = self.github([{"filename": 17}]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + def test_non_string_previous_filename_raises(self): + gh = self.github([{"filename": "c/a.yaml", "previous_filename": ["old/a.yaml"]}]) + with self.assertRaises(validate.ChangedFilesUnavailable): + gh.changed_files(17) + + +class MalformedExemptPathsInput(unittest.TestCase): + """A malformed `exempt-paths` fails the RUN, the way a malformed `team-keys` does.""" + + def test_main_rejects_a_malformed_list_before_touching_the_event(self): + with tempfile.TemporaryDirectory() as tmp: + event_path = os.path.join(tmp, "event.json") + # DELIBERATELY unparseable: this test's claim is that `exempt-paths` is validated + # BEFORE the event is read, and a well-formed fixture cannot tell the two orders + # apart. If main() ever parses the event first, json.load raises here and the test + # fails loudly instead of passing on a coincidence. + with open(event_path, "w", encoding="utf-8") as handle: + handle.write("{ this is not json") + env = { + "GH_REPO": "Comfy-Org/example", + "GITHUB_EVENT_PATH": event_path, + "LINEAR_API_TOKEN": "token", + "EXEMPT_PATHS": "infrastructure/dynamicconfig/**,,docs/**", + } + original = {key: os.environ.get(key) for key in env} + os.environ.update(env) + try: + self.assertEqual(validate.main(), 1) + finally: + for key, value in original.items(): + if value is None: + os.environ.pop(key, None) + else: + os.environ[key] = value + + class BranchProtectionLookup(unittest.TestCase): def test_encodes_branch_names_and_returns_boolean(self): github = validate.GitHub("Comfy-Org/example") diff --git a/scripts/linear-ticket/validate.py b/scripts/linear-ticket/validate.py index 105be819..4d58c8c9 100644 --- a/scripts/linear-ticket/validate.py +++ b/scripts/linear-ticket/validate.py @@ -18,6 +18,9 @@ GITHUB_EVENT_PATH workflow_run event payload TEAM_KEYS raw ``team-keys`` input (comma-separated; empty = any team) EXEMPT_LABEL exemption label name; empty disables exemption + EXEMPT_ACTORS comma-separated exempt PR-author logins; empty disables exemption + EXEMPT_PATHS comma-separated path patterns; a PR whose changed files ALL match is + exempt. Empty (the default) disables the path exemption entirely REQUIRE_OPEN_ISSUE "true"/"false" ENFORCE "true" (fail closed) / "false" (warn-only: a failing VERDICT never exits nonzero; a broken run still can — see finish_fail and run()) @@ -49,6 +52,23 @@ LINEAR_API_URL = os.environ.get("LINEAR_API_URL") or "https://api.linear.app/graphql" BACKOFF_SECONDS = (2, 4, 8, 16) # between five attempts +# GitHub's documented ceiling on GET /repos/{owner}/{repo}/pulls/{number}/files: the endpoint +# returns at most 3000 files and TRUNCATES SILENTLY — no error, no flag on the response. That +# silence is the hazard the path exemption has to defend against, because a truncated list can +# consist entirely of matching config files while the 3001st changed file is Go. +CHANGED_FILES_CAP = 3000 + + +class ChangedFilesUnavailable(Exception): + """The PR's changed-file list could not be read, or could not be trusted to be complete. + + Raised only when a path exemption is configured, since that is the only time the list is + consulted. Always fails the check closed (``changed_files_unavailable``) rather than + falling through to the Linear query: the run cannot tell an exempt PR from a non-exempt + one, and guessing either way publishes a verdict it did not reach. + """ + + ATTACHMENTS_QUERY = """query PullRequestAttachments($url: String!) { attachmentsForURL(url: $url, first: 20) { nodes { id url issue { id identifier team { key } state { type } } } @@ -177,6 +197,81 @@ def delete_marker_comment(self, pr: int) -> None: if result.returncode != 0: warning(f"Failed to delete marker comment {existing}: {result.stderr.strip()}") + def changed_files(self, pr: int, declared_count: int | None = None) -> list[str]: + """Every changed path in the PR, as untrusted DATA to be MATCHED, never interpolated. + + Paths are GitHub-owned metadata (they come from this endpoint, not from the PR branch), + so reading them is compatible with the fork-safety constraints in linear-ticket.yml's + header — but the filenames themselves are still author-controlled strings, so they only + ever reach ``lib.all_paths_exempt`` and a log line. + + Paginated at 100/page. Raises ChangedFilesUnavailable — never returns a partial list — + when the request fails, when the list is at or past ``CHANGED_FILES_CAP``, when its + length disagrees with ``declared_count``, or when any entry is not an object carrying a + non-empty string ``filename`` (and a string ``previous_filename`` if present). + ``declared_count`` is the PR payload's own ``changed_files`` field, checked FIRST + because it is the only authoritative count of a truncated read — and checked AGAIN + against the returned length, because a list that is merely short is indistinguishable + from a complete one once the non-exempt path is the entry that went missing. + + A rename contributes BOTH its new ``filename`` and its ``previous_filename``: renaming + a Go file into an exempt config directory is still a change to that Go file, and the + exemption must not be reachable by moving code across the pattern boundary. + """ + # `>=`, not `>`: GitHub documents the 3000-file maximum but NOT what it does past it, + # so a PR sitting exactly at the ceiling is indistinguishable from a truncated one and + # is refused too. `isinstance` so a malformed payload degrades to the list-length guard + # below instead of raising TypeError out of a privileged job. + if isinstance(declared_count, int) and declared_count >= CHANGED_FILES_CAP: + raise ChangedFilesUnavailable( + f"the PR reports {declared_count} changed files, at or past GitHub's " + f"{CHANGED_FILES_CAP}-file ceiling for the changed-files API, which truncates " + "silently") + data = self.get(f"/repos/{self.repo}/pulls/{pr}/files?per_page=100", paginate=True) + if not isinstance(data, list): + raise ChangedFilesUnavailable( + "the changed-files API request failed or returned a non-list body") + if len(data) >= CHANGED_FILES_CAP: + raise ChangedFilesUnavailable( + f"the changed-files API returned {len(data)} entries, at or past its " + f"{CHANGED_FILES_CAP}-file ceiling, so the list may be truncated") + # The two counts must agree exactly. A SHORT list is the dangerous direction: drop the + # one non-exempt path and every survivor can match, turning a partial read into a + # published waiver. A SURPLUS list means the payload is not the thing we asked for, so + # it is refused too rather than pattern-matched. Disagreement is only expected when the + # head moved between the two reads, and `_guard_supersession` already suppresses a + # superseded run's terminal write — so failing closed here costs a re-run, never a gate. + if isinstance(declared_count, int) and len(data) != declared_count: + raise ChangedFilesUnavailable( + f"the PR reports {declared_count} changed files but the changed-files API " + f"returned {len(data)} entries; the list does not match the PR") + paths: list[str] = [] + for entry in data: + # Not `(entry or {})`: a truthy non-dict entry would raise AttributeError straight + # out of this privileged job, escaping the ChangedFilesUnavailable contract that + # `_check_path_exemption` fails closed on. Same reason the paths are type-checked — + # a non-string reaches `lib.path_matches_any`'s `.fullmatch()` and raises TypeError. + if not isinstance(entry, dict): + raise ChangedFilesUnavailable( + "a changed-file entry was not an object; the response is malformed") + filename = entry.get("filename") + if not filename: + raise ChangedFilesUnavailable( + "a changed-file entry carried no filename; the response is malformed") + if not isinstance(filename, str): + raise ChangedFilesUnavailable( + "a changed-file entry carried a non-string filename; the response is " + "malformed") + paths.append(filename) + previous = entry.get("previous_filename") + if previous and not isinstance(previous, str): + raise ChangedFilesUnavailable( + "a changed-file entry carried a non-string previous_filename; the " + "response is malformed") + if previous: + paths.append(previous) + return paths + def current_pr_target(self, pr: int) -> tuple[str | None, str | None]: data = self.get(f"/repos/{self.repo}/pulls/{pr}") return ( @@ -249,7 +344,7 @@ def _log_rate_limit(headers) -> None: # ── orchestration ─────────────────────────────────────────────────────────────────────── class Validator: def __init__(self, gh: GitHub, token: str, team_keys, require_open, enforce, soft_fail, - run_url): + run_url, exempt_paths=None): self.gh = gh self.token = token self.team_keys = team_keys @@ -259,6 +354,9 @@ def __init__(self, gh: GitHub, token: str, team_keys, require_open, enforce, sof self.run_url = run_url self.exempt_label = os.environ.get("EXEMPT_LABEL", "") self.exempt_actors = lib.parse_actor_list(os.environ.get("EXEMPT_ACTORS", "")) + # Parsed in main(), not here: a malformed value must fail the RUN with a readable + # message (as team-keys does), not raise out of a constructor. + self.exempt_paths = exempt_paths or [] self.pr_number: int | None = None self.validated_sha: str | None = None self.validated_base_branch: str | None = None @@ -421,6 +519,14 @@ def run(self, event: dict) -> int: f"PR author `@{author}` is in `exempt-actors` — the Linear-ticket requirement " "is waived for this bot/automation account.", f"Exempt via exempt-actors (@{author})") + # The path exemption is checked LAST of the three because it is the only one that + # costs GitHub API calls (the label and actor lists are already in the PR payload). + # It still short-circuits before the Linear query, so an exempt config-only PR spends + # no Linear token budget and does not depend on Linear being reachable. + if self.exempt_paths: + exempt_outcome = self._check_path_exemption(pr) + if exempt_outcome is not None: + return exempt_outcome nodes, infra_error = self._query_attachments(html_url) @@ -433,6 +539,36 @@ def run(self, event: dict) -> int: return self._diagnose_and_fail(nodes, infra_error, branch, title, body) + def _check_path_exemption(self, pr: dict) -> int | None: + """Exit code if the run terminates here (exempt, or the list was unreadable); else None. + + None means "not exempt, carry on to Linear" — the ordinary outcome for a PR that + touches anything outside `exempt-paths`. + """ + try: + paths = self.gh.changed_files(self.pr_number, pr.get("changed_files")) + except ChangedFilesUnavailable as exc: + error(f"Could not read the changed-file list for PR #{self.pr_number}: {exc}. " + "`exempt-paths` is configured, so the exemption cannot be decided; failing " + "closed.") + return self.finish_fail("changed_files_unavailable", + f"Changed-file lookup: {exc}.") + if lib.all_paths_exempt(paths, self.exempt_paths): + log(f"All {len(paths)} changed path(s) match exempt-paths — exempt.") + return self.finish_exempt( + f"Every changed path in this PR ({len(paths)}) matches the repository's " + "`exempt-paths` patterns — the Linear-ticket requirement is waived for this " + "PR because it changes nothing outside the exempted paths.", + f"Exempt via exempt-paths (all {len(paths)} changed paths match)") + # Logged, not commented: a non-exempt PR is the normal case, and the reason it is not + # exempt belongs in the run log rather than on every PR in the repo. Only the COUNT is + # logged, never the names: a git path may legally contain a NEWLINE, so echoing an + # author-controlled filename from this privileged job would let a branch forge + # `::error::`/`::set-output::` workflow commands in the run log. + log(f"{len(paths)} changed path(s); at least one is outside exempt-paths — the " + "Linear-ticket requirement applies.") + return None + def _resolve_pr(self, event: dict, head_sha: str) -> int | None: """Exactly one open PR. Same-repo runs carry workflow_run.pull_requests; fork runs do not, so fall back to the commit->PR association (GitHub-owned data either way).""" @@ -559,6 +695,14 @@ def main() -> int: "keys, unique, comma-separated.") return 1 + try: + exempt_paths = lib.parse_path_list(os.environ.get("EXEMPT_PATHS", "")) + except ValueError as exc: + error(f"Invalid exempt-paths input: {exc}. Entries must be non-empty, unique, " + "repo-relative path patterns, comma-separated (e.g. " + "`infrastructure/dynamicconfig/**`).") + return 1 + with open(event_path, encoding="utf-8") as handle: event = json.load(handle) @@ -571,7 +715,7 @@ def main() -> int: run_url = os.environ.get("RUN_URL", "") validator = Validator(GitHub(repo), token, team_keys, require_open, enforce, soft_fail, - run_url) + run_url, exempt_paths) return validator.run(event)