Repository navigation
feat: novelty-check-broad — compare panel against 72-AMP curated database - #86
Merged
Merged
Conversation
Compares all 20 pilot panel candidates against a curated 72-sequence AMP reference database (broader than the 45-seed novelty score used during ranking). Key finding: 16/20 candidates are NOVEL (<50% similarity to any known AMP), 1 is CLOSE_RELATIVE (SEED-005_VAR_019 at 60%), and 3 are KNOWN_VARIANT (SEED-003_VAR_017/012 ~82%, SEED-001_VAR_064 ~87% — all expected from seed design). SEED-006/007/008/009 families confirmed genuinely novel. - Adds `novelty-check-broad` CLI command with --panel-csv, --references-csv, --out, --threshold-known, --threshold-close flags - Outputs docs/NOVELTY_BROAD_CHECK.md with per-candidate table and KNOWN/CLOSE/NOVEL classification - Addresses ROADMAP.md gap: "True novelty check against APD3/DRAMP/dbAMP" - 4 new tests in TestNoveltyCheckBroad (all passing) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Guard against inverted thresholds (threshold-close >= threshold-known) returning rc=1 with clear error instead of silently wrong classification - Fix float() ValueError on empty/NA optional fields using (val or 0.0) pattern - Skip panel rows with empty sequence with stderr warning instead of classifying blank string as NOVEL - Fix best_sim initialised at -1.0 (consistent with novelty.py) so refs with 0.0 similarity still populate best_ref - Derive NOVEL seed families dynamically from results instead of hardcoding SEED-006/007/008/009 in the Recommendations text - Add 2 new tests: known_variant_classification and inverted_thresholds_returns_error - Regenerate docs/NOVELTY_BROAD_CHECK.md with fixed Recommendations section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 task done
cschanhniem
added a commit
that referenced
this pull request
Jun 28, 2026
…#87) - ROADMAP.md: mark 'True novelty check' as Partial (PR #86 covers 72-AMP curated DB; full APD3 BLASTp still needed before publication) - ROADMAP.md: mark 'Pre-registration of assay protocol' as Done - WET_LAB_PROBABILITY.md: correct SEED-003 mechanism from 'bovine lactoferricin/defensin' to 'RRWQWRMKKLG-class tachyplesin-like' based on broad novelty check (81.8% similarity to REF-RRW-001) - WET_LAB_PROBABILITY.md: raise SEED-003 P(active) from 50-65% to 60-75% (known-active parent sequence elevates base rate) - WET_LAB_PROBABILITY.md: update composite P(zero hits) from 0.000635 (99.94%) to 0.000508 (99.95%) reflecting corrected SEED-003 estimate - WET_LAB_PROBABILITY.md: add cross-reference to NOVELTY_BROAD_CHECK.md in scope header - Makefile: add 'novelty-broad' target (depends on 'pilot') Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
cschanhniem
added a commit
that referenced
this pull request
Jun 28, 2026
The previous version was outdated (pipeline v0.1.0, 5 seeds, stale phase3 candidates, old batch stats). Updated to reflect current state: - Pipeline version 0.5.x, AUROC 0.8420, 7 seed families - Current 20-candidate pilot panel table with novelty categories (16 NOVEL, 3 KNOWN_VARIANT, 1 CLOSE_RELATIVE from PR #86) - SEED-003 annotated as KNOWN_VARIANT (82% similar to RRWQWRMKKLG) - Gold-standard calibration section with Melittin safety blind spot - Per-family summary table with P(active), reviewer focus, serum notes - Updated reviewer questions targeting current 7-seed specific issues - Updated next steps: serum stability first, MRSA USA300, external predictor checklist requirement before synthesis - Updated pipeline limitations table with Melittin blind spot, serum model edge cases, Pro-rich stability caveat - Document cross-reference index to all companion docs Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 task done
cschanhniem
added a commit
that referenced
this pull request
Jun 28, 2026
SEED-003 variants were missing a dedicated special notes section despite being KNOWN_VARIANT (81.8% similar to RRWQWRMKKLG, a known-active AMP per PR #86 broad novelty check). Key additions: - Known-variant classification with implications for P(activity) (60-75%) and publication novelty (limited — value is as assay control) - Met oxidation risk warning for RRWNWRMKKMG at position 10 - Aggregation caution (Trp at >200 µM in aqueous buffer) - Scorer disagreement interpretation (0.23-0.26 = model consensus) - C-terminal amidation recommendation Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
novelty-check-broadCLI command that compares each pilot panel candidate against a curated 72-sequence AMP reference databaseKey Findings
New artifact
docs/NOVELTY_BROAD_CHECK.md— generated fromoutputs/pilot_panel.csv; should be filed withdocs/ASSAY_PREREGISTRATION.mdbefore synthesis orderTest plan
TestNoveltyCheckBroadtests:returns_zero,creates_report(content),stdout_summary(n_novel==2 for both test sequences),missing_panel_returns_error🤖 Generated with Claude Code