Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 13 additions & 12 deletions docs/DISCOVERY_PREDICTION.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,9 +243,9 @@ Fmoc SPPS with acetonitrile/water gradient; verify MALDI-TOF pre-assay.
Updated from ~55–65% due to addition of proven scaffold families.

Basis:
- Pipeline AUROC = 0.8348 (95% CI: TBD; PR #70 windowed mu_h + anionic guard; 43 unique AMPs, n=87 total; previous 0.8047 PR #66, 0.8086 pre-dedup, 0.8164 pre-Trp-bonus)
- Pipeline AUPRC = 0.8443 (+0.2943 above random baseline 0.4943; PR curve emphasises precision at selection operating point)
- Recall@20 = 43% on internal benchmark (positives recovered in top 20 ranked candidates)
- Pipeline AUROC = 0.8420 (bootstrap CI₉₅: 0.76–0.91, n_bootstrap=2000; PR #72 face_segregation_bonus; 43 unique AMPs, n=87 total; previous 0.8348 PR #70, 0.8047 PR #66, 0.8086 pre-dedup, 0.8164 pre-Trp-bonus)
- Pipeline AUPRC = 0.8627 (+0.3684 above random baseline 0.4943; PR curve emphasises precision at selection operating point)
- Recall@20 = 44% on internal benchmark (positives recovered in top 20 ranked candidates)
- SEED-003 family (up to 4/20): RRWQWRMKKLG is curated known AMP → variants ~65–75% hit rate
- SEED-007 family (up to 4/20): Bombolitin II (*Bombus pennsylvanicus* bumblebee venom) with
antimicrobial activity documented in Bozelli et al. (2017) BBA-Biomembranes and subsequent
Expand Down Expand Up @@ -467,14 +467,15 @@ Despite the probability gap, the pipeline has done the following correctly:
2. **Consistent synthesis feasibility:** All pilot candidates are MODERATE SPPS difficulty
except one HIGH (SEED-002 23-mer). Zero synthesis-impossible candidates nominated.

3. **Strong AUROC:** 0.811 (bootstrap CI₉₅: 0.71–0.89) — the ensemble correctly separates
AMP-like from random-sequence background ~80% of the time.
3. **Strong AUROC:** 0.8420 (bootstrap CI₉₅: 0.76–0.91, n=87, n_bootstrap=2000; PR #72 face_segregation_bonus) — the ensemble correctly separates
AMP-like from random-sequence background ~84% of the time.

4. **Dual-scorer consensus:** All phase3 nominees have disagreement ≤ 0.40 (physicochemical and
4. **Dual-scorer consensus:** All phase3 nominees have disagreement ≤ 0.45 (physicochemical and
Boman scorers agree within the validated threshold). Note: SEED-008 (Trp-rich puroindoline-a)
class sequences produce disagreement ~0.37 — this reflects Boman scale's W=-3.398 limitation
class sequences produce disagreement ~0.43 — this reflects Boman scale's W=-3.398 limitation
(protein-binding potential) vs Trp's actual interfacial insertion mechanism. The threshold was
raised from 0.30 to 0.40 to not falsely exclude this validated AMP mechanism class (PR #61).
raised from 0.30→0.40 (PR #61) and 0.40→0.45 (PR #72) to not falsely exclude this validated
AMP mechanism class. Non-Trp-rich candidates in the 709-sequence pool remain below 0.41.

5. **Safety-first selection:** Safety ≥ 0.60 required (max_safety_risk = 0.40). Mean panel
safety = 0.991. SEED-010 (KR-12 human LL-37 fragment) correctly excluded for cytotoxicity
Expand Down Expand Up @@ -604,10 +605,10 @@ Executing all four actions on the best Wave 1 hits would push the combined proba

## Summary Table: Probability by Gate

| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PRs #48–53 (current) | Primary limiting factor |
| Stage | Gate | Original | After PRs #31–38 | After PRs #39–42 | After PRs #43–47 | After PRs #61–72 (current) | Primary limiting factor |
|-------|------|----------|-----------------|-----------------|-----------------|--------------------------|------------------------|
| 0 | Synthesis success | ~90% | ~90% | ~88% | **~89%** ✓ | **~90%** ✓ (agg model + agg-safe gen + pro penalty + pH74 charge) | SEED-008 W-rich; all aggregation/synthesis risks modelled |
| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~62–72%** ✓ (AUROC 0.8348) | AUROC 0.8348 (AUPRC 0.8443); 6 scaffold families confirmed |
| 1 | MIC ≤ 32 μg/mL | ~55–65% | ~55–65% | ~60–70% | **~60–70%** | **~62–72%** ✓ (AUROC 0.8420) | AUROC 0.8420 (AUPRC 0.8627); 6 scaffold families confirmed |
| 2 | TI > 10 (selectivity) | ~35–50% | ~35–50% | ~38–52% | **~40–55%** ✓ | **~42–57%** ✓ (stronger SEED-004 demotion) | sel_proxy doubled penalty for HIGH_CYTOTOX_RISK tier |
| 3 | t½ > 2 h (serum) | ~10–20% | ~25–40% | ~28–42% | **~28–42%** | **~30–46%** ✓ (3-protease model + short/Trp-rich model correction; pilot-panel data) | SEED-003/008 may outperform model score; all seeds borderline — early serum assay recommended |
| 4 | Scaffold novelty | ~10–15% | ~18–25% | ~25–35% | **~26–36%** ✓ | **~26–36%** | Diversity filter removes cross-seed near-dups |
Expand Down Expand Up @@ -683,7 +684,7 @@ The corrected estimate is more conservative because:

### 2. Benchmark Limitations

The AUROC 0.8348 (phase3 synthesis gate: 0.8126) is measured on a small 43+44 demo dataset, not validated against the full
The AUROC 0.8420 (phase3 synthesis gate: 0.8266) is measured on a small 43+44 demo dataset, not validated against the full
APD3 (> 3,000 AMPs), DRAMP v3.0 (> 19,000 entries), or ESCAPE benchmark (> 80,000 peptides
from 27 repositories). This may overestimate discriminative power.

Expand Down Expand Up @@ -739,7 +740,7 @@ than the 10 originally planned. The path to 50%+ requires wet-lab data integrati
## Confidence Calibration

This assessment is based on:
- Internal benchmark AUROC = 0.8348 (n=87, bootstrap n=2000; phase3 synthesis gate = 0.8126; PR #70 windowed mu_h + anionic guard)
- Internal benchmark AUROC = 0.8420 (n=87, bootstrap CI₉₅: 0.76–0.91, n_bootstrap=2000; phase3 synthesis gate = 0.8266; PR #72 face_segregation_bonus; PR #70 windowed mu_h + anionic guard)
- Literature hit rates for physchem AMP prediction (Loose et al. 2006; Tossi et al. 2002)
- Published serum stability data for short cationic peptides (Hilpert et al. 2006)
- D-amino acid t½ extension data (Wade et al. 1990, PNAS)
Expand Down
Loading