Skip to content

Defect 3: synthesiser ignores other reviewers' prior findings (prior-ingestion) #62

Description

@Jodre11

Context

Part of the "reproducible borderline reviews" flow that began with a real-PR verdict-flip incident. Three layered defects were identified; Defect 1 shipped in #60. This issue tracks Defect 3 — the substantive remaining architectural thread, explicitly scoped OUT of #60 as a non-goal.

The defect

Verified prior ignored. Self-re-review mode reacts only to the current user's prior review. When a PR already carries reviews from other reviewers, those findings are fetched solely for deduplication — never fed to the synthesiser to verify-or-refute. Adjudicating a prior claim ("is this earlier finding real?") is exactly what the synthesiser is good at, and the architecture currently declines to use it.

This is the same self-re-review architecture that blocked the #60 incident-replay validation: the identity entanglement (a reviewer's own prior verdict anchoring the synth) is a symptom of how narrowly prior-review context is currently handled.

Why it was deferred from #60

#60's goal was the generic case (a PR reviewed once, no external prior) — multi-sampling is the generic form of "cross-check against a verified prior" (cross-check against your own resamples). Prior-ingestion only adds value on multi-reviewer PRs, and it carries real risk:

  • Anchoring / sycophancy — the synth may defer to a prior verdict rather than judge independently.
  • Prompt injection — other reviewers' review text is untrusted input; feeding it to the synth widens the injection surface. Needs the same trust-boundary handling specialists already apply to diff content.

Scope (to be designed)

  • Feed other reviewers' findings to the synthesiser as claims to verify-or-refute, not just dedup against.
  • Trust-boundary treatment of external review text (treat as data, never instructions).
  • Guard against anchoring — the synth must be able to overturn a prior, not just agree.
  • Decide the UX: does the posted review show "adjudicated N prior findings: X confirmed, Y refuted"?

Status

Deferred design thread — the biggest remaining piece of the #571 flow. Needs a brainstorming/design pass before implementation (the trust-boundary and anchoring risks make this non-trivial). Not started.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions