Skip to content

Add public reanalysis, surrogate validation, clock integration, and methods note - #2

Merged
weston-wang merged 4 commits into
mainfrom
claude/report-audit-69nv4o
Sep 27, 2026
Merged

weston-wang merged 4 commits into
mainfrom
claude/report-audit-69nv4o

Conversation

@weston-wang

Copy link
Copy Markdown
Owner

Summary

This PR addresses four gaps that limited the project's value to the field.

1. Public-data reanalysis that changes a conclusion

  • New module: rejuvenationkit.genomics.concordance. It measures how similar two interventions' effect vectors are after removing shared-control noise. Two arms measured against the same control mice correlate at r ≈ 0.5 by construction, even when neither intervention has any effect. For each pair it reports:
    • the corrected correlation;
    • a disjoint-control cross-check that needs no noise model;
    • a within-stratum permutation test; and
    • Holm adjustment across all pairs.
  • Dataset helpers: GSE131754 helpers and examples/gse131754_shared_control_reanalysis.py.
  • Result on 78 public mouse liver transcriptomes covering 8 lifespan interventions:
    • Across the 10 shared-control pairs, mean naive concordance is r = +0.24. Corrected, it is +0.07, and no pair survives Holm adjustment.
    • Rapamycin shows no detectable concordance with any other intervention.
    • The concordance that survives involves CR and acarbose with GHRKO, Snell dwarf and methionine restriction, all measured against independent controls.
    • The two correction routes agree to within 0.035 on every pair.
    • Write-up: docs/gse131754-concordance-reanalysis.md.

2. Surrogate-endpoint validation harness

  • New module: rejuvenationkit.surrogates, implementing the meta-analytic framework of Buyse et al. It reports:
    • trial-level R² from a method-of-moments fit, with a parametric bootstrap. The nonparametric bootstrap covered only 61% with 6 units; the parametric one covers 93–100%.
    • individual-level R²;
    • the surrogate threshold effect;
    • leave-one-unit-out prediction coverage (93–98% in simulation); and
    • a trial-count planner.
  • Simulation findings:
    • It detects the surrogate paradox: individual R² = 0.82 alongside a trial-level interval that includes 0.
    • Validating even an excellent surrogate (true R² = 0.81) needs about 40 independent units to have a 66% chance of showing R² > 0.5.
  • Example: examples/surrogate_validation.py.

3. biolearn / pyaging integration

  • New module: rejuvenationkit.clocks.
    • It adapts biolearn predict and run_predictions output, and pyaging adata.obs, into typed, provenance-carrying observations.
    • These flow into QC, treatment-effect, state-estimation and surrogate analyses.
    • A test runs a real biolearn Horvath clock end to end; it is skipped when torch or biolearn is absent.
  • age_acceleration fits the clock-on-age line on reference samples only and scores reference samples leave-one-out.
    • An all-sample regression understates a true effect by 38% in single-arm pre-post designs and by 50% with age-imbalanced arms.
    • The reference-only fit is unbiased in both designs.
    • In balanced randomized difference-in-differences the choice does not matter; that case is also documented.

4. Methods note

  • Draft: paper/methods-note.md, covering the detector, fusion, small-sample, shared-control, age-acceleration and surrogate-precision findings.
  • Reproduction: paper/reproduce.py regenerates every table. It is seeded, and its CSVs are committed in paper/results/. A --quick mode runs in the test suite.
  • Numbers: every figure in the note, the docs and the README comes from that script's output.

Research decision and assumptions

  • Concordance scope: the reanalysis is a methods demonstration on one dataset with a deliberately simple pipeline. It explicitly makes no claim about the source publication, which used many datasets and a different model.
  • Concordance correlation floor: the corrected correlation is withheld when either effect vector's signal fraction is below 10%, because the ratio is unstable there. The permutation test stays valid.
  • Surrogate fit: trial-level surrogacy uses method-of-moments rather than full likelihood, trading efficiency for transparency. The bootstrap is parametric because the nonparametric one undercovers with few units.
  • Dependencies: clock adapters duck-type library output formats, and neither library is a dependency. biolearn needs torch, seaborn and lifelines at import time, which is documented.

Validation

  • Tests added: test_genomic_concordance.py, test_surrogates.py, test_clocks.py (including a real-biolearn test), test_paper_reproduce.py
  • ruff check .
  • ruff format --check .
  • mypy
  • pytest: 559 passed, 2 skipped; coverage 85.83%
  • Documentation updated (mkdocs build --strict passes); three new pages added to the nav

Limitations and failure modes

  • Concordance correction scope: it removes shared-animal noise only. Batch effects shared by separately controlled arms remain.
  • Concordance permutation test: conservative when real but unrelated effects exist.
  • GSE131754 sample size: three mice per group, so null results mean "not detectable".
  • Surrogate model: assumes linearity. Survival outcomes must be supplied as per-unit log hazard ratios.
  • Clock integration: the pyaging adapter is tested against its documented output shape, not against an installed pyaging (which pulls in torch).
  • Methods note: a draft that needs author review before any submission.

🤖 Generated with Claude Code

https://claude.ai/code/session_01PDyYDahYopgwfwJXNT11Ze


Generated by Claude Code

…eanalysis

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDyYDahYopgwfwJXNT11Ze
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDyYDahYopgwfwJXNT11Ze
- paper/methods-note.md: draft covering detector calibration, random-effects
  fusion, small-sample intervals, shared-control concordance (with the
  GSE131754 reanalysis), leakage-safe age acceleration, and surrogate
  validation precision.
- paper/reproduce.py regenerates every table (seeded; --quick smoke mode used
  by the test suite); CSV outputs committed under paper/results.
- README and changelog summarize the new findings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDyYDahYopgwfwJXNT11Ze
@weston-wang
weston-wang marked this pull request as ready for review September 27, 2026 05:47
@weston-wang
weston-wang merged commit 930f099 into main Sep 27, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants