Repository navigation
probe(P-D4): refit the base rate inside each bootstrap draw; the post-hoc gate does not pass - #1725
Conversation
) pi is estimated from the same blocks being resampled; holding it fixed left its error out of the interval. Refitted: 0.0242 [-0.0062, 0.0552], so the post-hoc gate does not clear the kill condition. The test now pins that, and the board entry is corrected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Warning Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
This PR fixes the codex P2 on #1724, which merged before the fix landed.
P-D4 scores abstentions at the base rate π. π is estimated from the same 40 blocks that the bootstrap resamples, but #1724 held it fixed across draws. That left π's own estimation error out of the interval.
With π refitted inside every draw:
The interval now includes 0, so the kill condition fires: the post-hoc gate's gain is not established. P-D4 as a whole fails. The pre-registered margin was already vacuous, and the post-hoc arm does not rescue it.
Changes
crates/causal-edge/tests/p_d4_decision_gate_probe.rs:base_rate_in, which refits π on the drawn blocks;post_hoc_gate_gain_is_not_established, which pins the failing result;.claude/board/entries/2026-10-22-probes-pd4-pe2-pd3.md: corrects the status, the P-D4 result and the closeout line. The ablation gains also used fixed π and are marked as overstated.The within-block selective-accuracy check does not use π and is unchanged: +0.120 [0.058, 0.185] on real labels, with an interval that includes 0 on shuffled labels. P-E2 and P-D3 are also unchanged.
Checks
Generated by Claude Code