Default Risk Lifecycle Lab

A domain-neutral synthetic laboratory for default-risk-like model lifecycle management.

synthetic history -> train -> calibrate -> holdout -> deploy simulation -> monitor -> recalibrate / rollback

The project contains no production model files, real feature names, customer data, business thresholds, or employer-specific logic.

Demonstrated capabilities

• chronological train/calibration/holdout evaluation;
• probability calibration with Brier score;
• synthetic feature, prediction, missingness, and prevalence drift;
• monitoring policy with healthy/warning/critical states;
• shadow model comparison;
• lifecycle recommendation: keep, investigate, recalibrate, or rollback;
• label maturity and delayed-snapshot governance;
• leakage-safe as-of history features;
• calibration comparison, cohort diagnostics, and shadow-model analysis;
• reproducible Markdown/PNG/JSON report;
• FastAPI, Docker, CI, and tests.

Run

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
pytest
python scripts/run_monitoring_report.py --scenario all
python scripts/run_full_report.py --scenario all
uvicorn monitoring.api:app --reload

Open reports/REPORT.md for the lifecycle report and http://127.0.0.1:8000/docs for the API.

The report includes a monitoring dashboard, PSI by feature, prediction shift, Brier score comparison, and lifecycle recommendation.

reports/FULL_REPORT.md is the integrated artifact: label quality, leakage checks, calibration, fairness diagnostics, shadow model comparison, registry state, and monitoring recommendation in one document.

All metrics are synthetic demonstrations, not production evidence.

See lifecycle design and model card for validation boundaries, monitoring ownership, and limitations.

See model development notes for label quality, leakage prevention, calibration, and promotion patterns.