Skip to content

Milestone 3: adverse action reason codes - #17

Merged
edward-sf merged 8 commits into
mainfrom
P3-Dev
Sep 2, 2026
Merged

edward-sf merged 8 commits into
mainfrom
P3-Dev

Conversation

@edward-sf

Copy link
Copy Markdown
Owner

/predict now returns up to four plain-language reasons on every response. Building that forced a compliance defect out of hiding in the Milestone 1 feature set, which this PR also fixes.

Implements docs/superpowers/specs/2026-09-02-creditboost-adverse-action-reason-codes-design.md task-by-task, test-first, one commit per task.

The defect, and what it cost to fix

ECOA, 15 U.S.C. §1691(a)(1), names the prohibited bases in one sentence: "race, color, religion, national origin, sex or marital status, or age." The codebase was scrupulous about two of them — CODE_GENDER banned outright, age entering only through a ratio — and used NAME_FAMILY_STATUS as model feature 16, with Widow and Separated among its scoring levels.

Reason codes made that untenable rather than merely untidy. A reason code discloses the actual principal reasons: disclose marital status and the service states on the record that a prohibited basis drove the decision; suppress it and the notice is untruthful, because the reasons disclosed are not the principal ones.

Removing it cost 0.00023 AUC — 0.75307 against the 0.75330 baseline, comfortably clear of the 0.70 floor. The feature was carrying essentially no signal.

The field is still accepted, now declared MONITORING_ONLY_FIELDS. Reg B §1002.13 requires collecting protected attributes precisely so fair-lending monitoring is possible, and the asymmetry is decisive: a model feature can always be restored by retraining, but an attribute a service refuses to accept cannot be collected retroactively. Dropping the request field would have been the only irreversible change in this milestone.

What changed

  • reasons.py — pure ranking module. Sums XGBoost pred_contribs contributions into ten concepts, discards concepts that helped the applicant, ranks, takes the top four. Contributions arrive as a plain mapping, so the logic is tested with dictionaries and no model at all.
  • Concept grouping, not per-feature ranking. Three EXT_SOURCE columns are one idea; ranking per feature would let a single cause fill all four slots by saying the same thing four times. It also closes an ECOA trap for free: employed_to_age is the one feature through which age enters the model, and it can never surface alone because it is only ever reported as employment.
  • Wording distinguishes absent from unfavourable. "No external credit score on file" vs "Credit scores from external bureaus are low" — for a thin-file borrower that is the difference between a true statement and a false one. The DAYS_EMPLOYED == 365243 sentinel is already scrubbed to NaN before the transform returns, so an unemployed applicant reads as "No employment history on record" rather than "employment is short."
  • No new runtime dependency. pred_contribs is built into the xgboost already in the image; the shap package would have violated M1's small-image constraint for no gain.
  • PROTECTED_ATTRIBUTES generalises a rule the codebase was following unevenly, with a test asserting disjointness from FEATURE_ORDER.
  • model-v0.2.0 released and pinned — the first real exercise of M2's release pipeline, and the lockfile bump is exactly the reviewable five-line diff it was designed to be.

Latency with the production model: median 6.03 ms, p95 6.42 ms. The second booster call is not a meaningful cost at this tree count.

Load-bearing tests

  • REASON_CONCEPTS partitions FEATURE_ORDER exactly. The invariant that rots silently when the feature list changes — an unmapped feature would simply never be reportable and nothing else would notice.
  • Contributions plus bias reconstruct the predicted probability. pred_contribs returns n_features + 1 values; an off-by-one yields reasons that are entirely plausible and entirely wrong, a failure with no natural symptom. It passed before the wiring existed, which is the point — it confirmed the row layout before any code depended on it.
  • No reason text names or implies age, sex, or marital status, by word-boundary match. A disclosure that reintroduces a protected basis in prose would undo the feature work it accompanies.

168 tests, ruff, mypy and lint-imports all clean.

Deviations from the plan

  • The fixture generator needed fixing — it built columns from CATEGORICAL_LEVELS then selected by REQUEST_FIELDS, the exact coupling Task 1 breaks. It now owns its monitoring-only level map, kept out of config.CATEGORICAL_LEVELS because anything there becomes a feature. The regenerated fixture's values shifted (removing a dict entry removes an rng.choice and moves the stream); column set unchanged, every asserted property still holds.
  • mypy rejected the plan's type-narrowing idiom: is not None does not survive assignment into a separate boolean, so wording selection is an if/else.
  • pandas types index labels as Hashable, so the missing-mask comprehension casts keys with str().

Deliberately deferred

NAME_INCOME_TYPE carries Maternity leave (a sex proxy) and Pensioner (an age proxy) as levels, so an employment_profile reason can implicate a protected characteristic indirectly. Unlike marital status these are levels rather than a whole prohibited-basis feature, and dropping employment type would cost real signal. Recorded in CLAUDE.md as a named open question for disparate-impact measurement rather than resolved by guesswork.

🤖 Generated with Claude Code

edward-sf and others added 8 commits September 2, 2026 14:36
Reason codes from XGBoost pred_contribs, grouped into ten concepts and
rendered as curated plain-language text, returned on every /predict.

Writing it surfaced a defect in the M1 feature set: NAME_FAMILY_STATUS is a
model feature, and marital status is an enumerated ECOA prohibited basis
alongside sex and age. Explainability forces the issue -- disclose it and the
service states a prohibited basis drove the decision; suppress it and the
notice is untruthful. The spec drops it as a feature, keeps it as an accepted
field for later fair-lending measurement, and names the rule the codebase was
already following unevenly as PROTECTED_ATTRIBUTES.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six tasks: quarantine marital status and drop to 20 features, retrain and
release model-v0.2.0, the reason catalog, the pure ranking module, the
/predict wiring, and docs.

Two things the spec left implicit and the plan pins down. REQUEST_FIELDS and
FEATURE_ORDER both derive from CATEGORICAL_FEATURES today, so removing the
feature would silently drop the request field and defeat the quarantine -- a
separate MONITORING_ONLY_FIELDS tuple breaks that coupling. And MODEL_VERSION
must be bumped before training, since train.py stamps it into the metadata that
release-model.sh then checks against the tag.

Task 1 leaves docker build red until Task 2 releases the retrained artifact;
that window is documented up front rather than discovered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Marital status is an enumerated ECOA prohibited basis, 15 U.S.C. 1691(a)(1),
named in the same sentence as sex and age. The codebase excluded those two and
used NAME_FAMILY_STATUS as model feature 16, with Widow and Separated among its
scoring levels. Milestone 3's reason codes forced the issue: disclose it and the
service states a prohibited basis drove the decision; suppress it and the notice
is untruthful.

The field is still accepted and now declared MONITORING_ONLY_FIELDS. Reg B
1002.13 requires collecting protected attributes for monitoring, and an
attribute a service refuses to accept cannot be collected retroactively, whereas
a feature can always be restored by retraining. REQUEST_FIELDS therefore stops
deriving from CATEGORICAL_FEATURES alone.

Two files beyond the plan's list. Decoupling REQUEST_FIELDS from
CATEGORICAL_FEATURES broke the fixture generator, which built columns from
CATEGORICAL_LEVELS and then selected by REQUEST_FIELDS -- the same coupling. It
now draws monitoring-only fields from its own level map, kept in the generator
rather than config because config.CATEGORICAL_LEVELS drives the transform and
anything in it becomes a feature. The regenerated fixture's values shifted:
removing an entry from CATEGORICAL_LEVELS removes an rng.choice call, moving the
whole stream. The column set is unchanged and every asserted property still
holds.

NOTE: docker build fails from this commit until the retrained model-v0.2.0 is
released. That is the M2 skew gate correctly refusing a 21-feature artifact
against 20-feature code.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ten concepts partitioning the 20 features, each with a code, unfavourable
wording, and -- where reachable -- distinct wording for absent data. Grouping is
what stops one idea filling all four reason slots, and it closes an ECOA trap:
employed_to_age can never surface alone because it is only ever reported as
employment.

A word-boundary test asserts no reason text names age, sex or marital status. A
disclosure that reintroduces a protected basis in prose would undo the feature
work it accompanies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sums contributions within a concept, discards concepts that helped the
applicant, ranks by total and takes the top four. Absent wording is used only
when every feature in the concept was missing -- following the single largest
contributor instead would let an applicant with two of three bureau scores on
file be told no score is on file.

Pure by construction: contributions arrive as a mapping, not a numpy array, so
the logic is tested with dictionaries and no model.

The wording selection is an if/else rather than the plan's conditional
expression: the plan asserted that `text.absent is not None` on one side of an
`and` would narrow the type, but narrowing does not survive assignment into a
separate boolean, and mypy rejected it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Up to four plain-language reasons on every response, ranked by concept
contribution. Two booster calls rather than one: deriving the probability from
the contribution sum would make the primary output a byproduct of the
explanation path.

Median 5.67 ms, p95 8.02 ms end to end through TestClient -- measured against a
fixture-trained model, since the committed artifact is still the 21-feature
v0.1.0 and cannot load against this code. Re-measure after the retrain for a
figure representative of the production tree count.

The missing-mask comprehension casts keys with str(): pandas types index labels
as Hashable, which mypy rejects against Mapping[str, bool].

The margin-reconstruction test is the load-bearing one -- an off-by-one in the
pred_contribs row yields reasons that are entirely plausible and entirely wrong,
a failure with no natural symptom. It passed before the wiring existed, which is
the point: it confirmed how pred_contribs lays out its row before any code
depended on that layout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
20 features, validation roc_auc 0.75307 against the 0.75330 baseline of the
21-feature model-v0.1.0 -- a cost of 0.00023, comfortably clear of the 0.70
floor. Removing an enumerated ECOA prohibited basis turned out to cost
essentially no accuracy, which is the empirical answer to whether the
compliance fix was a trade worth making: there was barely a trade.

Trained on the same 307511 rows, 246008 train / 61503 validation.

Closes the build-red window opened by "stop scoring on marital status".
model-v0.1.0 remains released and downloadable, so reverting is a three-line
lockfile diff.

Latency with the production model, end to end through TestClient: median
6.03 ms, p95 6.42 ms, against 5.67/8.02 for the fixture model measured earlier.
The second pred_contribs call is not a meaningful cost at this tree count.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The prohibited-basis rule is now stated once, over PROTECTED_ATTRIBUTES, rather
than as prose naming two fields and silently omitting the third.

Records the NAME_INCOME_TYPE proxy-level question as a named open item for
disparate-impact work rather than leaving it undocumented, and corrects M2's
"nothing else is scheduled" line, which M3 overtook.

The README's /predict example is the verbatim response from the running
container, checked rather than composed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@edward-sf

Copy link
Copy Markdown
Owner Author

✅ Phase 3 changes reviewed and approved.

@edward-sf
edward-sf merged commit 7dddae5 into main Sep 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant