[weston-wang]

Summary

Fixes six statistical defects found in a project audit. In each case the stated error rate did not hold at the package's own default or minimum settings.

Component	Defect	Before (nominal 5% / 95%)	After
MultivariateChangeDetector	Threshold fitted on in-sample Mahalanobis scores	8.9–40% false alarms (n=20, p=2–8)	3.5–5.3%
SequentialTreatmentResponseDetector	Same leak, on per-subject maximum cumulative scores	10.0–14.8% (n=30, p=4–8)	5.7–6.6%
PrecisionWeightedFusion (random effects)	DerSimonian–Laird Wald interval with k = 2–4	76–88% coverage	≥95.8% (modified HKSJ)
Signature contrasts	Bootstrap SE biased by √((n−1)/n); percentile CI at n = 3	79.6% coverage	93–96.5%
RandomizedTreatmentEffectEvaluator feature CIs	Percentile bootstrap at n = 8	90.4% coverage	95.5%
GeneralizedLeastSquaresFusion	Interval never widened when evidence disagreed	83% coverage (heterogeneity ≈ 1 SE)	97–99%

Treatment-effect inference also changes:

• Omnibus covariance: now the arm-centered pooled within-group covariance, recomputed for every permutation. Previously a real effect inflated the covariance and reduced power.
• Stratified permutation: new randomization_strata argument. In a simulated stratified 8-per-arm trial with a 1.5 SD stratum shift, unrestricted permutation had 1% type I error and 4% power; within-stratum permutation has 5.3% and 25%.
• Multiplicity: Holm-adjusted p-values across follow-up visits.
• Estimand: the docs now state it is the completers estimand.

Other changes:

• Detectors: reference scores are leave-one-out (reference_score_method="leave_one_out"). The change detector requires at least feature count + 2 reference subjects.

• Fusion: RandomEffectsInterval defaults to HARTUNG_KNAPP. WALD keeps the old behaviour.

• GLS: EvidenceDispersionPolicy.MULTIPLICATIVE (default). With Q the disagreement statistic and φ = Q/(k−1):

  • reported SE is multiplied by √max(1, φ);
  • interval half-width is max(z, t_{k−1}√φ) × SE;
  • evidence_overdispersed warning when φ > 1.

  FIXED keeps the old behaviour.

• Small-sample helpers: the variance correction and Welch t critical values are shared in rejuvenationkit/_small_sample.py.

• Docs: the GSE131754 benchmark SEs are regenerated from a real rerun (×1.22). Phase 4 coefficients are documented as simple effects. pysam tests skip when the extra is absent.

Research decision and assumptions

• Leave-one-out calibration: chosen over K-fold because it keeps nearly all reference data in each refit. The resulting threshold is slightly conservative.

• Modified HKSJ: the floor at 1 prevents intervals narrower than DerSimonian–Laird when estimates agree by chance.

• GLS interval rule: three rules were compared by simulation:

  Rule	Coverage
  z with floor	83–93%
  unfloored t	≈95%
  t with floor	98–100%

  Unfloored t is exact when Σ is correct up to scale, but it can collapse to near-zero width when estimates agree by chance. Taking max(z, t√φ) keeps it no narrower than the fixed-effect interval, at the cost of being conservative for k ≤ 3.

• Bootstrap SEs: the bootstrap is kept for its joint covariance and variance-corrected. Intervals are Welch t because percentile intervals are degenerate at small n.

• Feature intervals under strata: they ignore strata, so they are conservative under stratified randomisation. Stratifying the estimator would change the estimand.

Validation

• Tests added or updated: LOO score identity, Monte-Carlo false-alarm and coverage regressions, HKSJ critical value, Welch SE and df, GLS dispersion and non-narrowing, Holm adjustment plus forgery rejection, stratified-permutation provenance, arm-centered omnibus covariance
• ruff check .
• ruff format --check .
• mypy
• pytest: 536 passed, 2 skipped, coverage 85.42%
• Documentation updated (mkdocs build --strict passes)

Examples rerun successfully:

• public_gse131754_genomic_fusion.py
• public_dog_phase1_audit.py (11/213 detections, unchanged)
• randomized_rapamycin_effect.py
• synthetic_canine_genomic_fusion.py
• four_phase_workflow.py
• canine_multimodal_fusion.py

Limitations and failure modes

• Serialized detector models and treatment-effect reports from earlier versions fail validation and must be regenerated. This is intentional, since their calibration methods changed.
• Detector refits cost O(n) small matrix inversions each.
• The GLS and fusion intervals are conservative (97–100%) with two or three inputs.
• Welch intervals slightly undercover (~93%) with very unequal group sizes and variances.
• Covariate adjustment and mixed models remain out of scope.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PDyYDahYopgwfwJXNT11Ze