[edward-sf]

creditboost-train now measures the adverse impact ratio of every model across protected groups, stamps the result into the artifact, and refuses to write one that falls below the four-fifths threshold of 0.80.

Implements docs/superpowers/specs/2026-09-02-creditboost-disparate-impact-design.md task-by-task, test-first, one commit per task.

Why the conventional four-fifths test would have been theatre

Milestone 3 established that no protected attribute is a model feature. That is a statement about inputs, and it is not fairness — neutral features act as proxies, which is why disparate impact is a distinct doctrine from disparate treatment.

The obvious way to measure it does not work here. The four-fifths rule comes from UGESP, written for employment selection where selection rates are moderate. This model approves 97.3% of applicants, and at that rate the arithmetic compresses every ratio toward 1.0. Measured against the shipped model with adverse defined as the high band:

Attribute	Adverse-rate disparity	AIR	Verdict
Age (62+ vs under 62)	9.19×	0.974	passes
Marital status	4.10×	0.971	passes
Sex	1.97×	0.980	passes

An applicant under 62 is denied at more than nine times the rate of one aged 62 or over, and the test returns 0.974. A gate built that way would pass essentially any model — worse than no gate, because it manufactures documented assurance in exactly the artifact a compliance reader would trust.

Defining the adverse outcome as not banded low — the applicant is not auto-approved — restores sensitivity while keeping both the literal rule and its 0.80 threshold. It is grounded rather than convenient: Regulation B defines adverse action to include credit granted "on substantially the terms requested," so a medium band is adverse action.

Results

model-v0.3.0, measured on the 61,503-row validation split:

Attribute	AIR	Margin over 0.80
CODE_GENDER	0.86841	+0.06841
NAME_FAMILY_STATUS	0.81785	+0.01785
DAYS_BIRTH (62+ vs under 62)	0.80996	+0.00996

roc_auc 0.75307, unchanged. Age clears by one point, which is the design working: a gate that cannot fail proves nothing, and the response to a future failure is investigation, not adjusting the constant.

What changed

• fairness.py — computes the ratio as min/max over favourable rates. Inverted, a failing model reads as passing and the milestone is decorative, so one test exists solely to catch that direction.
• Age buckets at the ECOA line of 62, not a quantile, and an unknown age is excluded rather than filed as young — NaN >= 62 is False, which would otherwise report a rate for applicants whose age is not known.
• ModelMetadata.fairness is required, not optional. Optional would make "every shipped model has measured disparate impact" a hope rather than a guarantee. Enforced at three boundaries: training, artifact.load, and verify_artifact inside the Docker builder.
• "Not measured" never reads as "passed" — a pydantic validator enforces that exactly one of adverse_impact_ratio and unmeasured_reason is set, and failing_attributes never returns an unmeasured one.
• CODE_GENDER is now required in training data and still never accepted from a caller. FAIRNESS_ATTRIBUTES is a requirement, PROTECTED_ATTRIBUTES a prohibition; they overlap and are deliberately not merged. FEATURE_ORDER stays at 20, REQUEST_FIELDS at 19.
• No override flag, for the same reason the AUC floor has none.
• /metadata exposes the report with no endpoint change.

Also fixes a defect predating this milestone: lockfile, artifact_cli and reasons were never registered in the import-linter contract, because earlier string-replace edits silently no-opped and a narrower contract still passes. M2's spec claims that contract is what guarantees artifact_cli imports cleanly in the Docker builder where scikit-learn is absent — that claim was false until this branch. Verified by injecting a guarded forbidden import and confirming the contract now reports BROKEN.

Verification

The final whole-branch review independently reproduced the stamped ratios from the released model.json against the real validation split with its own positional numpy implementation — identical to the last digit. And model.json's sha256 is byte-identical to model-v0.2.0's, so this is provably a metadata refresh rather than a new model.

The load-bearing test was mutation-checked: swapping valid_frame for train_frame in train.py makes test_evaluate_is_called_with_the_aligned_validation_frame_and_predictions fail, and it passes again on restore. That seam matters because the fixture's 40-row validation split leaves every group under the minimum, so nothing else would have caught a misalignment — it would have stamped plausible, wrong ratios with no symptom.

194 tests, ruff, mypy and lint-imports all clean.

Deliberately deferred

• Adding a fairness-floor check to verify_artifact. Real, but the spec scopes enforcement to training time; expanding the ship boundary belongs in its own milestone.
• NAME_INCOME_TYPE carries Maternity leave (a sex proxy) and Pensioner (an age proxy) as levels. Recorded since Milestone 3, now measurable but unresolved.
• Remediation of any disparity found, intersectional analysis, and fairness of the reason codes across groups.

A note on what this does and does not claim

An adverse impact ratio below 0.80 establishes a prima facie case a creditor may rebut with business necessity. It is evidence, not proof, and this gate is a conservative engineering control rather than a legal conclusion. The disparities above are observed outcome differences: some part of the age gap is genuine repayment behaviour, and the model uses no age feature at all — any age effect arrives through proxies, which is exactly what this analysis exists to surface.

🤖 Generated with Claude Code