Repository navigation
feat: add novelty bonus to pilot priority formula (#37) - #37
Merged
Merged
Conversation
cschanhniem
added a commit
that referenced
this pull request
Jun 28, 2026
Add +0.05×novelty term to _pilot_priority() so that within a seed family, higher-novelty variants are preferred during pilot panel selection. SEED-005 variants (novelty=0.467, the pool maximum) gain up to +0.023 priority — a tiebreaker that pushes the most novel sequence variants into Wave 1 without overriding the ensemble score. Adds three new tests: higher-novelty-higher-priority, missing-novelty- defaults-to-0, and formula contribution verification.
cschanhniem
force-pushed
the
feat/pilot-priority-novelty-bonus
branch
from
June 28, 2026 05:02
82bf09b to
f07ab65
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
+0.05 × noveltyterm to_pilot_priority()inselection/pilot.py0.0when absent (backwards compatible)Motivation
The
DISCOVERY_PREDICTION.mdroot-cause analysis identified "Low scaffold novelty (-15 pp)" as the single largest probability penalty. The pilot panel previously usedensemble − 0.30×disagreement + 0.05×stability. Within the SEED-005 family (41 candidates, all novelty=0.467), selection was purely by ensemble+stability, meaning the max_per_seed=4 cap could select the 4 highest-ensemble variants rather than the 4 highest-novelty. Adding the novelty term corrects this and increases the chance that the 3-4 highest-novelty SEED-005 variants (e.g. SEED-005_VAR_009, _013, _028) enter Wave 1.New formula
Tests
test_higher_novelty_higher_priority— novelty=0.467 beats novelty=0.10 at same ensembletest_missing_novelty_defaults_to_0— absent key gives same result as novelty=0.0test_novelty_formula_contribution— verifies exact arithmeticAll 886 tests pass.
Test plan
pytest tests/test_pilot_panel.py— 35/35 passedpytest(full suite) — 886/886 passedselect_pilot_panel()signature unchanged🤖 Generated with Claude Code