Skip to content

Milestone 1: CreditBoost thin end-to-end slice (Tasks 1-10) - #15

Merged
edward-sf merged 13 commits into
mainfrom
P1-e2e-thin-slice
Sep 1, 2026
Merged

edward-sf merged 13 commits into
mainfrom
P1-e2e-thin-slice

Conversation

@edward-sf

Copy link
Copy Markdown
Owner

Milestone 1: a thin end-to-end slice from Home Credit training data through an
XGBoost model to a containerised FastAPI scoring service published to GHCR.

Implements Tasks 1–10 of docs/superpowers/plans/2026-08-30-creditboost-thin-slice.md.
Task 11 (the credentialed production training run) is not included.

What's here

config.py 21 model features, 19 request fields, thresholds, AUC floor
features.py the single shared transform imported by both training and serving
schema.py / banding.py Pydantic contracts and risk bands
artifact.py save/load plus the feature-order gate
data.py / train.py CSV loading, stratified split, training CLI
serve/ FastAPI app, fail-fast startup
Dockerfile / smoke.sh 702MB runtime image and its smoke test

101 tests, 99% coverage. The committed artifact is fixture-trained
(provenance: "fixture") so everything here is verifiable without Kaggle
credentials.

Deviations from the plan, and why

  • Python 3.12 only (plan said 3.11 + 3.12). numpy and xgboost both now
    require ≥3.12, so a 3.11 job would resolve an older ML stack than the shipped
    image and could fail loading an artifact serialised by xgboost 3.4.1.
  • GHCR image name. The plan's ghcr.io/${{ github.repository }} resolves to
    a name containing capitals, which GHCR rejects — every publish would have
    failed. Now built from the lowercased owner plus creditboost, which is what
    the plan's own Task 11 verification step expects.
  • xgboost-cpu on Linux. Cuts ~650MB of NVIDIA CUDA libraries unused by CPU
    inference. Scoring verified bit-exact against the full build. macOS keeps
    plain xgboost, the only variant shipping a wheel there.
  • CNT_CHILDREN defaults to None, not 0. An unanswered field must not be
    imputed at the API boundary, where the distinction cannot be recovered.
  • Dependency-direction test rewritten. The plan's assert 'sklearn' not in sys.modules was broken — import xgboost pulls sklearn in regardless. Now
    checks creditboost.data / creditboost.train directly, which also catches a
    guarded import the original would have missed.
  • Artifact hardening beyond the plan: the feature-order mismatch error names
    the first disagreeing index, an xgboost major-version check was added, and
    ArtifactError unifies the failure taxonomy.

Not included

Task 11 needs data/application_train.csv from Kaggle (gitignored, never in CI)
and will replace the fixture artifact with a production one.

🤖 Generated with Claude Code

edward-sf and others added 10 commits August 31, 2026 16:06
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Also raises requires-python to >=3.12: numpy and xgboost both now require
3.12, so a 3.11 CI job would resolve an older ML stack than the shipped image.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CNT_CHILDREN defaults to None rather than 0: an unanswered field must not be
imputed at the API boundary, where the distinction cannot be recovered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Beyond the plan: the mismatch error now names the first disagreeing index and
values, an xgboost major-version check raises while tolerating patch drift and
unparseable versions, and ArtifactError unifies the failure taxonomy.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The dependency-direction test checks creditboost.data/creditboost.train in
sys.modules directly. The plan's sklearn proxy check was broken (xgboost pulls
sklearn in regardless) and would have missed a guarded forbidden import.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Scopes xgboost-cpu to Linux so the image drops ~650MB of NVIDIA CUDA
libraries unused by CPU inference; macOS keeps plain xgboost, which is the
only variant shipping a wheel there. Scoring verified bit-exact.

smoke.sh also asserts probability parses as a number in [0,1].

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Matrix is 3.12-only: numpy and xgboost both require >=3.12, so a 3.11 job
would resolve an older ML stack than the shipped image.

Image name is built from the lowercased owner plus a literal 'creditboost'.
The plan's ghcr.io/${{ github.repository }} resolves to a name with capitals,
which GHCR rejects, so every publish would have failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
edward-sf and others added 3 commits August 31, 2026 20:48
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Trained on the full 307,511-row Home Credit dataset. Validation ROC-AUC
0.7533 against a 0.70 floor, PR-AUC 0.2410, Brier 0.0680; 246,008 training
rows, 318 trees.

Calibration confirms the decision to omit scale_pos_weight: mean predicted
probability 0.080394 against an actual positive rate of 0.080728.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Serving emitted no application logs at all: nothing configured logging, so
  under uvicorn every INFO record was dropped and a credit decision left no
  audit trail. Adds structured JSON logging scoped to the creditboost logger.
- The PII guard test inspected only getMessage(), so it could not fail; the
  fields it was meant to guard all travel via extra=. Now inspects real output.
- The skew gate checked only the metadata sidecar, so an artifact with
  transposed feature_names inside model.json booted and then 500d per request.
- The committed-artifact provenance assertion was vacuous against the Literal.
- CLAUDE.md and README.md described a repo with no application code, and the
  macOS libomp prerequisite was undocumented.
- Fairness invariants were asserted on source constants but never the artifact.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@edward-sf

Copy link
Copy Markdown
Owner Author

Review

Task level reviews were done between each commit with minor changes needed after certain tasks.

A final branch-level review caught a number of issues that couldn't be properly observed at the task level reviews we did previously.

CI returns clean for test, lint, and build. Through Milestone 1, we match the project specs.

Merge approved. I'll begin planning Milestone 2.

@edward-sf
edward-sf merged commit 86d2f17 into main Sep 1, 2026
4 checks passed
@edward-sf
edward-sf deleted the P1-e2e-thin-slice branch September 1, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment