Skip to content

Defect 2 (residual): pro-cyclical APPROVE filter hides thin output #61

Description

@Jodre11

Context

Part of the "reproducible borderline reviews" flow that began with a real-PR verdict-flip incident (the same pipeline returned APPROVE on one run and CHANGES_REQUESTED on another, on identical input). Post-incident analysis localised three layered defects. Defect 1 (single-draw specialist recall variance) shipped in #60 (variance-resampling on a boundary gate). This issue tracks the residual of Defect 2.

The defect

Pro-cyclical APPROVE filter. Under an APPROVE verdict the orchestrator suppresses sub-75-confidence findings from the posted review (isPosted(), posting policy in verdict-rubric.md). A low-recall run is more likely to land on APPROVE — so the engine hides its own thin output precisely when that output is thinnest. The filter's noise-suppression value is real on genuinely clean PRs; the problem is it can't distinguish "clean" from "thin."

What #60 already did

  • The boundary gate absorbs much of the amplifier: a thin APPROVE with borderline findings now triggers a 2nd stochastic draw instead of shipping silently.
  • The sub-75 disclosure line (N finding(s) below the posting threshold — see synthesiser report.) addresses the honesty half — an APPROVE no longer looks cleaner than the run was.

What's left (this issue)

Decide whether the residual pro-cyclical dynamic needs a structural change beyond the gate + disclosure, or whether it's now adequately mitigated. This is best judged from organic observation of real reviews (see watch items below), not a speculative change now.

Watch items that would reopen active work here:

  • Does the gate stay quiet on genuinely clean-cut PRs (cost containment)?
  • Does an APPROVE with a disclosure line still feel like it's hiding substance, in practice?
  • Does B2's bare contested-presence trigger over-fire on low-value contested findings? (If so, add a confidence floor to the B2 predicate in review-core.mjs and update test_gate_fires_b2_contested.)

Status

Low urgency. Mitigated by #60; open pending organic signal. Do not change speculatively.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions