Skip to content

fix+feat: CSV column validation and scorer disagreement gate - #25

Merged
cschanhniem merged 2 commits into
mainfrom
feat/csv-validation-disagreement-gate
Jun 28, 2026
Merged

cschanhniem merged 2 commits into
mainfrom
feat/csv-validation-disagreement-gate

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Summary

1. Helpful CSV column validation (loaders.py)

Previously, a CSV with seq instead of sequence would crash with a bare KeyError deep in the pipeline — no file name, no row number, no hint of what was expected. Now it raises ValueError with the actual columns found:

ValueError: CSV row 1 is missing required 'sequence' column.
Found columns: ['id', 'seq', 'source'].
Check that the CSV header spells it exactly as 'sequence'.

2. Scorer disagreement hard gate (pipeline.py + configs)

The pipeline already computed disagreement = |activity - boman_activity| per candidate, but never used it as a filter. A candidate with activity=0.9 and boman_activity=0.2 could still be nominated — contradictory signals with no resolution.

New eligibility condition:

and item.scores.get("disagreement", 0.0) <= max_disagreement

Config values added:

  • pipeline.yaml: max_disagreement: 0.40 (lenient — allows some model spread)
  • phase3.yaml: max_disagreement: 0.30 (strict — final synthesis batch needs consensus)

Default (when key absent): 1.0 = gate is off, existing callers unaffected.

Science rationale: two independent scoring approaches (physico-chemical features vs Boman interaction potentials) both predicting activity is better evidence than one score alone. High disagreement means we can't tell which model is right — don't spend synthesis money on it.

Test plan

  • test_missing_sequence_column_raises_value_error — correct error type and message
  • test_missing_sequence_column_error_names_found_columns — error lists actual columns found
  • test_disagreement_gate_blocks_high_uncertainty_candidate — max_disagreement=0.0 rejects every real sequence (all have disagreement > 0)
  • test_disagreement_gate_config_values_documented — pins pipeline.yaml=0.40, phase3.yaml=0.30, confirms phase3 is stricter
  • Full suite: 781 passed
  • ruff check src tests — clean

🤖 Generated with Claude Code

)

CSV loader (loaders.py):
  Missing 'sequence' column now raises ValueError with the actual column names
  found, instead of a bare KeyError — prevents cryptic crashes from mis-formatted
  input files before synthesis.

Disagreement gate (pipeline.py + configs):
  Add max_disagreement to eligibility filter:
    eligible = [...and scores.get("disagreement", 0.0) <= max_disagreement]
  pipeline.yaml: max_disagreement=0.40 (lenient)
  phase3.yaml:   max_disagreement=0.30 (strict — final synthesis batch needs
                 consensus between physchem-activity and Boman index scorer)
  Default is 1.0 (off) so existing callers without the key are unaffected.
  Science rationale: a candidate with activity=0.9 but boman_activity=0.2
  has high uncertainty; blocking it prevents wasting synthesis budget on
  contradictory predictions.

Tests (4 new):
  test_missing_sequence_column_raises_value_error
  test_missing_sequence_column_error_names_found_columns
  test_disagreement_gate_blocks_high_uncertainty_candidate
  test_disagreement_gate_config_values_documented
…ositive control

pipeline.py: scores.get('disagreement', 0.0) → scores['disagreement']
  Consistent with bracket access used for novelty/safety; if the key ever
  goes missing the pipeline fails loudly instead of silently passing all
  candidates through the disagreement gate.

loaders.py: validate CSV header before iterating rows
  Check reader.fieldnames before the row loop so header-only files (zero
  data rows) also raise ValueError. Per-row check was redundant once the
  header is validated — removed. Error message now says 'CSV header is
  missing' not 'CSV row 1 is missing' to avoid confusion.

test_data_loaders.py:
  - import pytest at module level (was inside function — ruff PLC0415)
  - add test_missing_sequence_column_fires_on_header_only_csv for zero-row case
  - remove inline imports

test_pipeline_filters.py:
  - add positive control to test_disagreement_gate_blocks_high_uncertainty_candidate:
    same candidate is selected when max_disagreement=1.0 (relaxed gate), proving
    the disagreement gate is the cause of blocking, not some other filter
  - fix ruff E741: rename l → ln in list comprehensions
@cschanhniem
cschanhniem merged commit e39dec9 into main Jun 28, 2026
3 checks passed
@cschanhniem
cschanhniem deleted the feat/csv-validation-disagreement-gate branch June 28, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant