Skip to content

feat: Phase Q Q3 post-experiment calibration update schema -- WHR results inform calibration - #950

Merged
cschanhniem merged 1 commit into
mainfrom
feat/q3-post-experiment-calibration-update
Jul 10, 2026
Merged

cschanhniem merged 1 commit into
mainfrom
feat/q3-post-experiment-calibration-update

Conversation

@cschanhniem

Copy link
Copy Markdown
Collaborator

Implements Q3 from NEXT_100_PR_MAP.

What

  • src/openamp_foundry/evidence/post_experiment_calibration_update.py: PCU- schema; PostExperimentCalibrationUpdate dataclass (16 fields); validate_post_experiment_calibration_update(); build_post_experiment_calibration_update() (enforces dry_lab_only=False, human_reviewed=True); format_post_experiment_calibration_update()
  • VALID_UPDATE_DIRECTIONS (5): increase_threshold, decrease_threshold, no_change, expand_candidate_pool, restrict_candidate_pool
  • VALID_CALIBRATION_TRIGGER_TYPES (7): hit_rate_above_baseline, hit_rate_below_baseline, novel_family_detected, safety_signal_observed, systematic_prediction_error, insufficient_data, scheduled_review
  • tests/evidence/test_post_experiment_calibration_update.py: 63 tests in 5 classes
  • BASELINE: 8711 → 8774

Disconfirming pass

  • dry_lab_only=True blocked (tested) — real experiment-triggered updates only
  • human_reviewed=False blocked (tested) — no auto-approved calibration changes
  • n_whr_records=0 blocked (tested) — evidence required before any calibration change
  • n_whr_records/whr_ids length mismatch blocked (tested)
  • update_magnitude out of [0,1] blocked (tested)
  • empty rationale and limitations blocked (tested)

…dry_lab_only=False enforced, human_reviewed=True required, VALID_UPDATE_DIRECTIONS (5), VALID_CALIBRATION_TRIGGER_TYPES (7), n_whr_records consistency check; 63 tests (#949)
@cschanhniem
cschanhniem merged commit 1196025 into main Jul 10, 2026
1 check failed
cschanhniem added a commit that referenced this pull request Jul 10, 2026
Co-authored-by: OpenCode <opencode@example.com>
cschanhniem added a commit that referenced this pull request Jul 10, 2026
…pre-registration for reproducibility; VALID_CONFIRMATION_VERDICTS (4), VALID_DIVERGENCE_TYPES (7), VALID_PREDICTION_QUALITY_GRADES (5); prediction-vs-actual divergence flag; 63 tests (#950) (#952)

Co-authored-by: OpenCode <opencode@example.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant