Skip to content

feat: Phase S S1 score vs cheap-enemy gap report (SEG-) -- pipeline score transparency vs cheap baselines - #966

Merged
cschanhniem merged 1 commit into
mainfrom
feat/s1-score-enemy-gap-report
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/s1-score-enemy-gap-report

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Implements S1 from NEXT_100_PR_MAP.

What

  • src/openamp_foundry/evidence/score_enemy_gap_report.py: SEG- schema; ScoreEnemyGapReport dataclass (16 fields) + CheapEnemyResult helper; validate_score_enemy_gap_report(); build_score_enemy_gap_report() (auto-computes verdict, best_enemy, gap, enforces dry_lab_only=True); format_score_enemy_gap_report()
  • VALID_GAP_VERDICTS (4): gap_meaningful, gap_marginal, gap_absent, comparison_not_run
  • VALID_CHEAP_ENEMY_TYPES (8): charge_score, length_score, hydrophobicity_score, isoelectric_point_score, aromaticity_score, combined_charge_length, random_baseline, sequence_similarity_to_known
  • VALID_GAP_ASSESSMENT_METHODS (5): absolute_difference, relative_margin, rank_correlation, precision_at_k_delta, auc_delta
  • MINIMUM_MEANINGFUL_GAP = 0.05 (absolute score units)
  • tests/evidence/test_score_enemy_gap_report.py: 53 tests in 7 classes
  • BASELINE: 9215 → 9278

Disconfirming pass

  • dry_lab_only=False blocked (tested)
  • TOY- candidate_family_id blocked (tested)
  • gap_meaningful requires gap >= MINIMUM_MEANINGFUL_GAP (tested)
  • gap_absent requires gap <= 0 (tested)
  • comparison_not_run requires n_enemies_tested == 0 (tested)
  • gap inconsistent with scores blocked (tested)
  • n_enemies_tested must match len(enemy_results) (tested)
  • invalid enemy_type blocked in results (tested)
  • build() picks highest-scoring enemy as best (tested)

… pipeline score margin over best cheap baseline is below threshold; VALID_GAP_VERDICTS (4), VALID_CHEAP_ENEMY_TYPES (8), VALID_GAP_ASSESSMENT_METHODS (5); gap_meaningful requires gap>=MINIMUM_MEANINGFUL_GAP (0.05); gap_absent enforced; comparison_not_run blocks enemies; dry_lab_only=True enforced; 53 tests (#966)
@cschanhniem
cschanhniem merged commit b769386 into main Jul 10, 2026
1 check failed
cschanhniem added a commit that referenced this pull request Jul 10, 2026
Co-authored-by: OpenCode <opencode@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant