Skip to content

feat: safety pH74 charge consistency + stronger cytotoxicity penalty (PR #53) - #53

Merged
cschanhniem merged 2 commits into
mainfrom
feat/safety-selectivity-improvements
Jun 28, 2026
Merged

cschanhniem merged 2 commits into
mainfrom
feat/safety-selectivity-improvements

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

Two targeted improvements to close consistency gaps identified in the post-PR-51 audit:

1. safety_score() — charge_density_ph74 consistency

safety.py was using the count-based charge_density proxy while activity.py uses charge_density_ph74 (pH 7.4 adjusted). For His-containing sequences (pKa 6.0), these differ significantly — up to 89% for histidine-rich sequences.

Fix: charge_density = abs(features.get("charge_density_ph74", features["charge_density"])) — uses pH 7.4 value preferentially, falls back to proxy for backward compatibility. Now consistent with activity scorer.

2. _pilot_priority() — stronger cytotoxicity penalty

The old formula gave cytotoxic-risk candidates (selectivity_proxy < 0.5) a soft reduced bonus but no explicit penalty. SEED-004 temporin (proxy=0.30) was only 0.035 points below a fully selective equal-ensemble candidate.

Fix: Add cytotox_penalty = 0.05 × (0.5 − proxy) / 0.5 for proxy < 0.5. SEED-004 now swings by 0.055 total (was 0.035) — still dominated by ensemble but enough to consistently deprioritize HIGH_CYTOTOX_RISK candidates when ensemble scores are close.

proxy old net new net swing
1.00 +0.050 +0.050 0
0.50 +0.025 +0.025 0
0.30 +0.015 −0.005 −0.020 more
0.00 +0.000 −0.050 −0.050 more

Test plan

  • 1172 tests pass
  • New test: charge_density_ph74 preferred over proxy in safety scorer
  • New tests: cytotox penalty boundary (0.50), max (0.00), intermediate (0.25)
  • Updated tests: max delta 0.05→0.10, SEED-004 demerit 0.035→0.055
  • Pre-push hook green

🤖 Generated with Claude Code

#53)

- safety_score(): use charge_density_ph74 (physiologically accurate pH 7.4)
  preferentially over the proxy, matching activity_likeness_score() behaviour.
  Prevents His-rich sequences from being incorrectly flagged for charge risk
  at the physiological pH where His is uncharged. Backward compat: falls back
  to charge_density when charge_density_ph74 is absent.

- _pilot_priority(): add cytotox_penalty term for HIGH_CYTOTOX_RISK candidates
  (selectivity_proxy < 0.5). Candidates in this tier now receive both reduced
  bonus AND an additional linear penalty, totalling up to −0.05 extra demerit
  vs the old soft-bonus formulation. Max total selectivity swing: ±0.05 → 0.10.
  SEED-004 temporin (proxy=0.30): net selectivity −0.005 (was +0.015 before).
  Literature: Dathe & Wieprecht (1999) BBA 1462:71-87; Carotenuto et al. (2008).

- 4 new tests: charge_density_ph74 preference in safety, cytotox_penalty
  boundary, max value, and intermediate value
- 3 updated tests: max_delta 0.05→0.10, SEED-004 demerit 0.035→0.055
- Makefile: bump test count to 1172
- pilot.py docstring formula: correct − 0.05×cytotox_penalty → − cytotox_penalty
  (the variable already has 0.05 factored in; the summary line was misleading)
- safety.py comment: clarify abs() is deliberate (charge magnitude = hemolysis risk)
- test_pilot_panel.py: add monotonicity assertion at proxy=0.49 (below threshold)
@cschanhniem
cschanhniem merged commit 1074bf2 into main Jun 28, 2026
@cschanhniem
cschanhniem deleted the feat/safety-selectivity-improvements branch June 28, 2026 08:33
cschanhniem added a commit that referenced this pull request Jun 28, 2026
- Add PR #53 to completed improvements header
- Add detailed PR #53 section: safety pH74 consistency + stronger cytotox penalty
- Update Stage 2 (TI gate) ~41-56% → ~42-57% (stronger SEED-004 demotion)
- Update combined breaking news probability ~28-48% → ~29-49%
- Add item 16 to "What the Pipeline Got Right"
- Update summary table, roadmap gap, and footer probability references
cschanhniem added a commit that referenced this pull request Jun 28, 2026
…#55)

- Rename 'Long cysteine repeat run' → 'Long homo-repeat run (any AA)'; fix deduction
  from −0.15 to −0.10 flat (code: synthesis.py repeat_run >= 5 → score -= 0.10)
- Fix cysteine fraction threshold from 0.15 to 0.20; fix deduction from scaled formula
  to −0.15 flat (code: cys > 0.20 → score -= 0.15)
- Fix aggregation propensity trigger from >0.2 to >0.0; fix formula from
  0.20×(prop−0.2)/0.8 to min(agg×0.25, 0.20) — linear ramp from zero, capped −0.20
- Add missing length penalties: length>30 (up to −0.40 ramp), length<8 (−0.30 flat)
  — the length>30 penalty is the largest single deduction possible; its absence from
  docs was a material omission
- Fix 'five risk signals' → 'six' in safety score prose (μH signal added in PR #53)
- Fix version header: 0.1.0 → 0.2.x with PR range note
- Fix activity score ceiling: clarify it is the arithmetic maximum of weight sum (0.97),
  not an explicit min() cap — the code applies clamp01() which caps at 1.0
cschanhniem added a commit that referenced this pull request Jun 28, 2026
* docs: update METHODS.md to reflect all v0.2.x scoring changes

- Section 4.1: expand activity-likeness formula to show all 7 terms (length, charge
  density at pH 7.4, hydrophobic fraction, aromatic bonus, amphipathicity, helix bonus
  weight 0.03, cross-term) with ceiling 0.97; note charge_density_ph74 preference
- Section 4.2: add μH > 0.55 risk signal (Dathe 1999); update charge_density column
  to charge_density_ph74 with fallback note; tabulate all 6 risk signals with increments
- Section 4.3: add proline penalty threshold (>15% → −0.10, DKP/Fmoc deprotection
  mechanism); tabulate aggregation penalty with ramp details
- New Section 4.3b: serum stability score — protease site density model for trypsin
  (×2), chymotrypsin (×1), elastase (×0.5); score formula and literature basis
- New Section 5.2: pilot panel selection — pilot_priority formula, cytotox_penalty
  for HIGH_CYTOTOX_RISK tier (proxy < 0.5), net selectivity swing +0.055
- Section 8: add retrospective AUROC 0.8164; table of v0.2.x PRs #47-54 with impact
- Section 10: expand references from 4 to 17; add all citations used in scoring code

* docs: fix 4 HIGH issues in synthesis feasibility table (code review PR #55)

- Rename 'Long cysteine repeat run' → 'Long homo-repeat run (any AA)'; fix deduction
  from −0.15 to −0.10 flat (code: synthesis.py repeat_run >= 5 → score -= 0.10)
- Fix cysteine fraction threshold from 0.15 to 0.20; fix deduction from scaled formula
  to −0.15 flat (code: cys > 0.20 → score -= 0.15)
- Fix aggregation propensity trigger from >0.2 to >0.0; fix formula from
  0.20×(prop−0.2)/0.8 to min(agg×0.25, 0.20) — linear ramp from zero, capped −0.20
- Add missing length penalties: length>30 (up to −0.40 ramp), length<8 (−0.30 flat)
  — the length>30 penalty is the largest single deduction possible; its absence from
  docs was a material omission
- Fix 'five risk signals' → 'six' in safety score prose (μH signal added in PR #53)
- Fix version header: 0.1.0 → 0.2.x with PR range note
- Fix activity score ceiling: clarify it is the arithmetic maximum of weight sum (0.97),
  not an explicit min() cap — the code applies clamp01() which caps at 1.0
cschanhniem added a commit that referenced this pull request Jun 28, 2026
)

* docs: update ROADMAP, add AUROC reproducibility note, update RISK_REVIEW

ROADMAP.md:
- Mark v0.1-v0.5 as complete (✓) with specifics of what was delivered in each milestone
- Add v0.5.x section documenting the pre-wet-lab scoring enhancements (PRs #31-#54):
  serum stability, selectivity proxy, pilot priority, aggregation, proline penalty,
  helix bonus, amidation/acetylation recommendations, DISCOVERY_PREDICTION model

METHODS.md Section 8:
- Add 'To reproduce: make validate-scoring' note so reviewers can reproduce AUROC 0.8164
  from the repository without external tooling (addresses code review feedback that
  the headline performance number was not reproducible from visible code)

RISK_REVIEW.md Section 2:
- Add selectivity proxy / HIGH_CYTOTOX_RISK as a dedicated filtered risk row; both the
  safety scorer (μH penalty, pH-7.4 charge) and pilot panel (cytotox_penalty) now
  double-mitigate cytotoxicity risk
- Update hydrophobicity row to mention μH > 0.55 signal added in PR #53
- Update charge density row to note charge_density_ph74 for accuracy
- Add proline content as a reported risk row
- Update action-required note to flag HIGH_CYTOTOX_RISK candidates for hemolysis assay

* docs: fix CLI command in METHODS.md reproducibility note (code review PR #56)

validate-scoring is a top-level command, not a bench subcommand.
'cli bench validate-scoring' would raise argparse error.
Corrected to 'cli validate-scoring' to match the actual Makefile target.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant