Skip to content

fix(server): skip CodeRabbit review acknowledgement wakes - #32300

Open
voltcrash wants to merge 1 commit into
pingdotgg:mainfrom
voltcrash:t3/ignore-review-progress-ack
Open

voltcrash wants to merge 1 commit into
pingdotgg:mainfrom
voltcrash:t3/ignore-review-progress-ack

Conversation

@voltcrash

Copy link
Copy Markdown
Contributor

Problem

CodeRabbit's reply to a review request says “Review triggered.” before editing the same comment with the result. PR watches currently wake the agent for both, although the acknowledgement has nothing to act on. Fixes #32275.

Change

Skip only coderabbitai[bot] replies whose first 1,000 characters contain CodeRabbit's reply marker and an “Action(s) performed” section starting with “Review triggered.” or “Full review triggered.” Still advance the watermark, so progress is consumed without spending a comment wake. Outcome edits, other replies, and other authors keep their existing behavior.

Scope and approval

This qualifies for CONTRIBUTING.md's very small, focused obvious-bug exception: the existing watch reports a queued-review acknowledgement as actionable feedback. The fix only filters that acknowledgement in the server evaluator; it adds no settings, contracts, client changes, or provider-specific behavior. The linked issue has not been maintainer-triaged. No changes to edit debouncing, wake labels, snippet rendering, or check recovery.

Verification

  • The two focused CodeRabbit regressions fail on upstream main because the acknowledgement appears in remarks. With the fix, all 23 tests in vp test run apps/server/src/orchestration-v2/pullRequestWatch.test.ts pass. Coverage includes both acknowledgement variants, finished/rate-limited outcome edits delivered once, mixed comments, same-time arrivals, other authors, other replies, and the 1,000-character bound.
  • vp run check:changed passed: format, lint, both unused-code scans, generated GraphQL, three affected package typechecks, and 25 tests across the evaluator and agent-context budget files.
  • Replayed the actual CodeRabbit reply's edit history through the evaluator using Node, without starting a server:
                         upstream main      fixed
Review triggered.        remarks, wakes=1   no change, wakes=0
same acknowledgement     no change          no change
Review finished. edit    remarks, wakes=2   remarks, wakes=1
same outcome             no change          no change

Both versions advance the watermark to 00:24:57Z for the acknowledgement and 00:29:47Z for the outcome on 2026-11-07. The reactor already persists watermark-only changes.

Limitations: no live remote watch or paid provider turn was started. The filter intentionally depends on CodeRabbit's observed reply format; unrecognised formats retain the existing wake behavior. Wake wording is unchanged.

Reviewed with review-loop: 1 round with GPT-6.1 Sol on xhigh. Reviewer verdict: approved.

Created with GPT-6.1 Sol (xhigh) in Codex through T3 Code.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Nov 7, 2026
@macroscopeapp

macroscopeapp Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 3c5e9d5

Macroscope's review found this PR approvable — PR watching skips CodeRabbit’s review-trigger acknowledgements while continuing to track comments and report other events. The narrowly targeted filter reduces unnecessary agent wakes without introducing an obvious material risk.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 3c5e9d5 and 6db649a.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review

No actionable comments were generated in the recent review.

Walkthrough

The PR watch evaluator now filters qualifying CodeRabbit review-triggered acknowledgements from reported remarks. Tests cover watermark advancement, wake counts, outcome edits, and replies that remain actionable.

Changes: CodeRabbit review acknowledgements

Layer / File(s) Summary
Detect and filter acknowledgements
apps/server/src/orchestration-v2/pullRequestWatch.ts
The evaluator recognizes matching CodeRabbit bot comments and excludes fresh acknowledgements from reported remarks.
Test acknowledgement and outcome handling
apps/server/src/orchestration-v2/pullRequestWatch.test.ts
Tests verify that queued acknowledgements advance the watermark without waking the agent, outcome edits are reported once, and other replies remain actionable.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Low

Possibly related issues

  • Issue 30342: The PR tests that an acknowledgement outcome edit is reported once. It does not implement the issue’s broader request to coalesce or label in-place status-comment edits.

Suggested reviewers: saphid

Merge Risk: Minimal · previous review 3c5e9

The change stops CodeRabbit's "review triggered" acknowledgements from waking the agent, while outcome edits and other replies keep waking it as before. No concrete merge-blocking risk was found.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title "fix(server): skip CodeRabbit review acknowledgement wakes" matches the main change. The evaluator now skips CodeRabbit bot replies that acknowledge a queued review, so those replies no long…
Description check Passed The description follows the template. It has Problem, Change, Scope and approval, and Verification sections, and each section is filled in. The Problem section states the expected behavior and the obs…
Linked Issues check Passed Directly linked issue #32275 is open and active, so its coding requirements apply. Its requirements are: (1) a coderabbitai[bot] reply whose first 1,000 characters contain the CodeRabbit reply marker …
Out of Scope Changes check Passed The whole-PR diff against merge base 4cb6945 changes two files: apps/server/src/orchestration-v2/pullRequestWatch.ts and apps/server/src/orchestration-v2/pullRequestWa…
Full details: Title check

Explanation

The title "fix(server): skip CodeRabbit review acknowledgement wakes" matches the main change. The evaluator now skips CodeRabbit bot replies that acknowledge a queued review, so those replies no longer wake the agent. It is a conventional commit title in plain language, short, and specific.

Full details: Description check

Explanation

The description follows the template. It has Problem, Change, Scope and approval, and Verification sections, and each section is filled in. The Problem section states the expected behavior and the observed behavior. The Change section explains the filter and what stays unchanged. The Scope section explains why the change qualifies for the obvious-bug exception and links the issue, but it notes the issue is not maintainer-triaged. The Verification section lists focused test runs, the check:changed run, and a replay of real edit history, with limitations stated. Minor gaps: the template asks for the model and harness at the end, which the author provided; the scope section relies on an exception that a reviewer must judge. Overall the description is mostly complete.

Full details: Linked Issues check

Explanation

Directly linked issue #32275 is open and active, so its coding requirements apply. Its requirements are: (1) a coderabbitai[bot] reply whose first 1,000 characters contain the CodeRabbit reply marker and an "Action(s) performed" section with "Review triggered." or "Full review triggered." must not wake the agent; (2) the watermark must still advance past it; (3) its later outcome edit must wake once; (4) replies answering questions, and replies created as "not completed" or "rate limited", must still wake; (5) the two named regression tests must exist.

Evidence in the diff of apps/server/src/orchestration-v2/pullRequestWatch.ts: the new isReviewAcknowledgement() checks author login coderabbitai[bot] and matches the marker, the "Actions? performed" summary, and "(Full review|Review) triggered." against remark.body.slice(0, 1_000). fresh.filter() now removes these acknowledgements before the remarks change is pushed. fresh still feeds latest, remarksThrough, and remarkIds, so the watermark advances. The activeAt() logic uses editedAt, so an outcome edit at a later time is fresh and produces one remarks change. Replies with other summaries (for example "Action not completed") do not match and still wake.

Evidence in the test diff: the test "skips CodeRabbit review acknowledgements and wakes once for their outcome edits" covers both acknowledgement variants, the no-wake and watermark assertions, finished and rate-limited edits delivered once, and same-time mixed arrivals. The test "keeps other replies and outcomes actionable beside CodeRabbit review progress" covers other replies, a missing marker, a marker beyond the 1,000-character bound, other bots, and human reviewers.

Limits: I could not run the test suite. The PASS rests on code and test reading. The CodeRabbit reply format is taken from the issue text and the PR's replay; I did not verify the live format. The reactor's persistence of watermark-only changes is described by the PR but was not inspected; the evaluator output alone satisfies the requirement as written.

Full details: Out of Scope Changes check

Explanation

The whole-PR diff against merge base 4cb6945 changes two files: apps/server/src/orchestration-v2/pullRequestWatch.ts and apps/server/src/orchestration-v2/pullRequestWatch.test.ts. The evaluator change adds one predicate and routes fresh remarks through it. The test change adds a codeRabbitReply fixture and regression tests for the same behavior. The PR description states that edit debouncing, wake labels, snippet rendering, and check recovery are unchanged. The diff matches this: the check, conflict, wake-count, and snippet code paths are unchanged. The fixture and tests support the linked-issue objective, so they are not out of scope. No settings, contracts, client code, or provider-specific behavior were added in the reviewed diff.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 4cb6945 and 3c5e9d5.

Files selected for processing: 2

  • apps/server/src/orchestration-v2/pullRequestWatch.test.ts
  • apps/server/src/orchestration-v2/pullRequestWatch.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 8 remain after this review.

@juliusmarminge juliusmarminge added bugfix Fix PR opened by an issue lead; enters the PR loop pr:gate-failed A land gate failed labels Nov 7, 2026
@voltcrash
voltcrash force-pushed the t3/ignore-review-progress-ack branch from 3c5e9d5 to 6db649a Compare November 7, 2026 12:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix Fix PR opened by an issue lead; enters the PR loop pr:gate-failed A land gate failed size:S 10-29 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: PR watch wakes the agent for CodeRabbit's "Review triggered" acknowledgement, which has nothing to act on

2 participants