Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 54 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -580,14 +580,67 @@ per-repo `noise.additionalIgnores`.
"noise": { "additionalIgnores": ["packages/*/build/**"] },
"review": { "trigger": "all", // which PRs `ecr ci` reviews: "all"
"label": "ai-review", // (default, except ai-review:skip) or
"skipLabel": "ai-review:skip" }, // "label" (only labeled PRs)
"skipLabel": "ai-review:skip", // "label" (only labeled PRs)
"readyForHumanLabel": "ready for human" }, // optional, see below
"breakGlass": { "marker": "/skip-review" }, // PR body marker that skips the review
"commentTag": "expo-ai-code-reviewer", // hidden tag used to find/update the comment
"auth": { "providers": {
"anthropic": { "tokenEnv": "CLAUDE_CODE_REVIEW_SHARED_API_TOKEN" } } }
}
```

`review.readyForHumanLabel` (off by default) names a label that `ecr ci`,
`ecr post-review` and `ecr review --pr <n> --post` add to the PR when the review
is ready for a human: the decision is "Ready for human review", with or without
comments, and the run reviewed the whole PR. These runs get no label:

- A partial run: `--agents` (so `/review <agents>` too), `--scopes`, or a review
saved with `ecr review --pr <n> --save-review --agents <ids>` and
posted with `ecr post-review`. A bare `/review`, where the router picks the agents, is a
full run.
- A run where a pass did not finish.
- A run where no reviewer applied, because no changed file matches any agent's
`paths`.
- A routed run with a changed file that no scope matches. With a routing
manifest that has no `**/*` catch-all scope, a PR that touches a file outside
every scope never gets the label.

A run that gets no label, a `/skip-review` that `ecr ci` acts on, or a run that
deletes the review comment removes the label, but only when the reviewer added
it last. Nothing else changes the label, so a ready label added before a push
stays until the next review posts.

- `ecr post-review` and `ecr review --pr <n> --post` post nothing on a PR with
`/skip-review`, and `ecr ci` does nothing on a PR its trigger policy skips (such
as one labeled `review.skipLabel`). Both leave the review comment and the label
as they are.
- The reviewer recognises its labels by login, from the label's newest
`labeled` or `unlabeled` event on the PR. It removes the label only when its
own login added it last, and does not add it again when another login removed
it last. Its login is `gh api user`, or, when that fails under GitHub Actions,
`github-actions[bot]`, the login of the default `GITHUB_TOKEN`. So:
- With a personal token, that person's own removals count as the reviewer's,
and the next ready review adds the label again.
- Under the default `GITHUB_TOKEN`, other workflows' label changes are
`github-actions[bot]` too, and count as the reviewer's.
- A local `ecr review --pr <n> --post` and CI use different logins, so neither
removes a label the other added.
- A GitHub App token cannot read `gh api user`, and its login is not
`github-actions[bot]`, so the reviewer never removes the label it added (nor
recognises its own review comment).
- On a PR with 3,000 events or more, the reviewer cannot tell who changed the
label last, and leaves it as it is.
- It works in the single comment mode. Per-scope comments have no one decision
for the PR, so that mode sets no label.
- `/dismiss` does not change the decision, so it does not change the label.
- When the reviewer fails before it can load the config, it leaves the label as
it is.
- The decision comes from a model that reads untrusted PR content, so a prompt
injection can earn the label. Never use it as a merge gate or in branch
protection.
- The label must differ from `review.label`, `review.skipLabel` and every
`<label>:<agent>` label. The workflow token needs `issues: write`.

</details>

<details>
Expand Down
8 changes: 8 additions & 0 deletions llp/0008-sources-and-reporters.explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,14 @@ The reporter maintains exactly one PR comment, plus optional inline threads (see

**Links fail soft.** `linkContextAsync` can be prebuilt and injected once so a CI fan-out builds one diff-lines + base-SHA context for all scopes instead of one pair of `gh` calls per scope — the reporter-side mirror of `memoizeSource` [observed] ([github.ts:33-37, 200-205](../src/reporters/github.ts)). Both underlying fetches fail soft: a missing diff index degrades in-diff findings to plain-text `file:line`, a missing base SHA degrades out-of-diff findings the same way — never a dead link, never a hard failure of the whole report [observed] ([github.ts:194-235](../src/reporters/github.ts); commits f9fecd5, 9446b42). This follows the repo rule that `ecr ci` must never fail a PR's checks on reviewer error [observed] (AGENTS.md: "`ecr ci` must never fail a PR's checks — reviewer errors degrade gracefully").

**The ready label follows the decision, and the bot removes only its own.** With `review.readyForHumanLabel` set, `report` and `reportAggregate` call `syncReadyLabel` after the comment is posted, and `postSkipNote` and `clear` call it with "not ready". `postSkipNote` is `ecr ci`'s break-glass path only: `ecr post-review` and `ecr review --pr --post` post nothing on break-glass, and a PR the trigger policy skips returns before any reporter call, so those leave the comment and the label as they are. Nothing else touches the label, so a label added before a push stays until the next review posts. Label failures only log: the comment is already up [observed] ([github.ts](../src/reporters/github.ts) `syncReadyLabel`, `readyForHuman`, `lastLabelEvent`; tests `src/__tests__/ready-label.test.ts`).

**Ready means the run reviewed the whole PR.** `readyForHuman` is true when every review completed with full coverage (`incomplete` is empty), at least one reviewer applied, and the worst decision is `approve` or `approve_with_comments`; `reportAggregate` also requires no unmatched files. A review where no agent's `paths` matches approves with every `agentSummaries` row `notRun`, so that is what "no reviewer applied" checks; `incomplete` stays empty there, and a noise-only review has no rows at all. A run that reviewed part of the PR sets the reporter's `partialReview`, and then `syncReadyLabel` treats it as not ready: an explicit `--agents` subset in `ecr ci` and `ecr review --pr --post`, a `--scopes` run, whose other scopes come from the comment's stored state without a check against the head, and a deferred review saved with `--agents`, which records `partialReview` in the artifact for `ecr post-review` to pass on. A bare `/review` lets the router pick agents and is a full run. Without `--scopes` every active scope gets a result, failures included, so no other check for missing scopes is needed ([ci.ts](../src/commands/ci.ts) `runLegacyCi`, `runRoutedCi`; [deferred-review.ts](../src/core/deferred-review.ts)).

**Ownership comes from the label's own history.** The newest `labeled` or `unlabeled` event for the label (`lastLabelEvent`; the events API lists the oldest first) decides what is ours: a review that is not ready removes the label only when the newest event is our account's `labeled`, and a ready review does not add it again when the newest event is another account's `unlabeled`. No extra state is stored in the comment. The walk stops at the page cap; when it stops on a full page, a newer event may lie beyond it, so `syncReadyLabel` changes nothing and logs. "Our account" is `resolveOwnLogin`: `gh api user`, or `github-actions[bot]` for the default `GITHUB_TOKEN`. The check is by login, not by person: a personal token treats that person's own changes as ours, under the default token other workflows' label changes are `github-actions[bot]` too and count as ours, and a local `ecr review --pr --post` and CI run as different logins, so neither removes the other's label. A GitHub App token cannot read `/user`, so under Actions `resolveOwnLogin` falls back to `github-actions[bot]`, while the App's actor is expected to be `<app>[bot]` [not verified]: such a setup never removes the label it added, nor recognises its own comment.

**Only the single comment labels, and the label is advisory.** In per-scope mode there is no one decision for the PR, `ecr ci` logs that it sets no label, and the root-tag `clear` of a mode switch removes ours (`runRoutedCi` in [ci.ts](../src/commands/ci.ts)). `/dismiss` re-renders with the stored decision, so it does not change the label. Label names match case-insensitively, as GitHub treats them, and the config check rejects a label that is, in any case, also `review.label`, `review.skipLabel` or a `<label>:<agent>` trigger. The decision comes from model output on untrusted PR content (see [LLP 0001](./0001-trust-model.principles.md)), so the label is advisory, never a merge gate.

## Inline Threads

With `inline.enabled`, `syncInline` converges one inline thread per in-diff finding: `planInlineSync` decides which threads to create, patch, stub or delete, and the executor runs the plan [observed] ([github.ts](../src/reporters/github.ts) `planInlineSync`, `syncInline`).
Expand Down
21 changes: 21 additions & 0 deletions src/__tests__/deferred-review.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ function config(configDir: string, overrides: Partial<LoadedConfig> = {}): Loade
protectedCategories: ["security", "secrets"],
maxAdjudications: 10,
},
review: { trigger: "all", label: "ai-review", skipLabel: "ai-review:skip" },
...overrides,
} as LoadedConfig;
}
Expand Down Expand Up @@ -85,6 +86,21 @@ test("writes and reads an owner-only exact review artifact", async () => {
expect(JSON.parse(await readFile(saved, "utf8"))).toEqual(parsed);
});

test("records a review saved from an agent subset as partial; older artifacts still parse", async () => {
const root = await tempRoot();
const loaded = config(path.join(root, ".expo-code-review"));
const saved = await writeDeferredReviewArtifact(loaded, {
repo: "expo/expo",
pr: 48557,
headSha: "a".repeat(40),
review: artifact().review,
partialReview: true,
});
expect((await readDeferredReviewArtifact(saved)).partialReview).toBe(true);
// artifact() has no partialReview field, as every artifact saved before it existed.
expect(artifact().partialReview).toBeUndefined();
});

test("strictly rejects malformed, oversized, and extended artifacts", async () => {
const root = await tempRoot();
const malformed = path.join(root, "malformed.json");
Expand Down Expand Up @@ -144,6 +160,11 @@ test("posting config fingerprint covers every comment-policy input", () => {
expect(
reviewPostingConfigFingerprint(config(root, { feedback: { ...base.feedback, mode: "off" } })),
).not.toBe(fingerprint);
expect(
reviewPostingConfigFingerprint(
config(root, { review: { ...base.review, readyForHumanLabel: "ready for human" } }),
),
).not.toBe(fingerprint);
});

test("post-review argv requires an explicit artifact, repo, and safe PR", () => {
Expand Down
Loading
Loading