Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion docs/research/NEXT_100_PR_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -222,4 +222,16 @@ Make the full pipeline auditable and reviewable by external scientists. First ph
| R2 | Add family novelty report schema (FNR-) (complete). — src/openamp_foundry/evidence/family_novelty_report.py: FNR- 13-field schema; VALID_BASELINE_COMPARISON_OUTCOMES (4), VALID_NOVELTY_STRENGTH_GRADES (4), VALID_CHEAP_BASELINES_CHECKED (5); strong grade requires outside_known_space; not_supported incompatible with outside_known_space; cheap_enemy_score/family_score in [0,1]; dry_lab_only=True enforced; 64 tests in tests/evidence/test_family_novelty_report.py. | Bundles CFC- clustering with cheap-baseline comparison; prevents "novel family" overclaim when candidates are merely sequence variants of known AMPs. | B/C |
| R3 | Add full audit trail report schema (ATR-) linking BSP→WHR→PCU→HCR→PQG for a complete dry-lab→wet-lab loop trace (complete). — src/openamp_foundry/evidence/audit_trail_report.py: ATR- 15-field schema; REQUIRED_ATR_STAGES (5: BSP/WHR/PCU/HCR/PQG), VALID_TRAIL_COMPLETENESS_STATUSES (4), VALID_CHAIN_INTEGRITY_GRADES (5); complete status enforces all 5 stages; grade D incompatible with complete; build() auto-computes n_stages_complete; dry_lab_only=True enforced; 70 tests in tests/evidence/test_audit_trail_report.py. | End-to-end chain of custody from nomination to confirmed hit; required for scientific reproducibility. | B/C |
| R4 | Add scientific review readiness gate schema (SRG-) (complete). — src/openamp_foundry/evidence/scientific_review_readiness_gate.py: SRG- 15-field schema; REQUIRED_SRG_ARTIFACT_TYPES (4: CFC/FNR/ATR/PQG), VALID_READINESS_VERDICTS (4), VALID_SAFETY_FLAG_TYPES (6), VALID_REVIEW_SCOPE_TYPES (4); ready_for_external_review blocks safety flags and failed_gates; open_preprint/peer_review blocked when not_ready/review_blocked_by_safety; dry_lab_only=True enforced; 63 tests in tests/evidence/test_scientific_review_readiness_gate.py. | Machine-verifiable checklist for when a candidate family has enough evidence for external scientific review; requires CFC+FNR+ATR+PQG records and no unresolved safety flags. PR #962. | C |
| R5 | Add preprint evidence bundle schema (PEB-) — read-only snapshot of all evidence for a run. | External reviewers can verify every claim from a single self-contained artifact; closes Phase R. | C |
| R5 | Add preprint evidence bundle schema (PEB-) — read-only snapshot of all evidence for a run (complete). — src/openamp_foundry/evidence/preprint_evidence_bundle.py: PEB- 19-field schema; VALID_BUNDLE_STATUSES (5: draft/internal_review/approved/submitted/published), VALID_PREPRINT_SERVERS (7), VALID_CLAIM_STRENGTH_TIERS (4); DRY_LAB_ONLY_DISCLAIMER canonical text enforced; published requires DOI; submitted/published require real server; replicated_wet_lab requires n_confirmed_hits≥2; dry_lab_only=True enforced; 66 tests in tests/evidence/test_preprint_evidence_bundle.py. BASELINE: 9152→9215. PR #964. | External reviewers can verify every claim from a single self-contained artifact; closes Phase R. | C |

## Phase S — Score transparency and external credibility

Make it possible for an external reviewer to understand, in one artifact, why specific candidates were selected and how the pipeline's scores compare to cheap baselines. The bottleneck to external credibility is "Can the system help choose real experiments better than cheap baselines?" — every item here answers part of that question.

| PR | Task | Why it matters | Review class |
|---:|---|---|---|
| S1 | Add score vs cheap-enemy gap report schema (SEG-). | Flags when pipeline score margin over best cheap baseline is below a meaningful threshold; prevents advancing candidates that cheap baselines explain equally well. | C |
| S2 | Add selection transparency ledger schema (STL-). | Per-candidate-family ledger of every evidence artifact that influenced selection; makes selection_basis auditable and traceable. | C |
| S3 | Add evidence completeness checker schema (ECC-). | Lists all expected artifact types (BSP/WHR/PCU/HCR/PQG/CFC/FNR/ATR/SRG/PEB) and which are present/absent for a candidate family; completeness_grade (A–D); closes "what's still needed for review?" question. | C |
| S4 | Add pipeline run stability report schema (RSR-). | Compares outputs across N pipeline runs; flags candidate families whose ranking is unstable across runs; prevents overconfident selection of borderline candidates. | C |
| S5 | Add prediction quality trend report schema (PQT-). | Tracks cumulative prediction quality across all confirmed WHR results; rolling precision/accuracy trend; degradation alerts; closes Phase S. | C |
Loading