Skip to content

Milestone 2: model artifact storage - #16

Merged
edward-sf merged 14 commits into
mainfrom
P2-Dev
Sep 2, 2026
Merged

edward-sf merged 14 commits into
mainfrom
P2-Dev

Conversation

@edward-sf

Copy link
Copy Markdown
Owner

Moves the trained model's bytes out of git into a GitHub Release, leaving a checksum-pinned lockfile in their place, with a build-time guard that makes shipping a bad artifact structurally impossible.

Implements docs/superpowers/plans/2026-09-01-creditboost-model-artifact-storage.md task-by-task, test-first, one commit per task.

What changed

  • models/model.lock.json (349 bytes) is now the only tracked file in models/. It pins release tag model-v0.1.0 and a sha256 per asset.
  • creditboost-artifact CLI with fetch, verify, and lock. Downloads via urllib, not curl, so the Docker builder needs no apt-get layer. Transient failures back off and retry; a 404 fails immediately, because a missing release is a defect and retrying only delays a truthful failure.
  • The Docker builder fetches and verifies in a single RUN — a cached layer of fetched-but-unverified bytes would be a trap. docker build . still takes zero arguments and no credentials.
  • file_sha256 moved to a dependency-free hashing.py, because data.py imports scikit-learn, which the builder stage does not install.
  • The layering rule became an import-linter contract, replacing the hand-rolled sys.modules subprocess test: it is static analysis of a static rule, covers eight modules rather than one, and reports the offending chain. CI runs lint-imports.
  • scripts/smoke.sh asserts /health reports provenance: production — the last gate before the GHCR push.
  • The model itself is unchanged (roc_auc 0.7533); only where its bytes live is. History is deliberately not rewritten: the problem was future retrains appending copies, which deleting from HEAD solves.

Verification

Nine of the plan's ten success criteria were verified locally:

  1. git ls-files models/ prints exactly models/model.lock.json, 349 bytes.
  2. A clean clone carrying only the lockfile builds and serves — no arguments, no credentials.
  3. That image's /health reports "provenance":"production", "model_version":"0.1.0".
  4. 138 tests pass with no network access; no test references a non-loopback host.
  5. Corrupting the lockfile digest fails the build with a ChecksumMismatchError naming both digests. Verified, not assumed.
  6. ProvenanceError on a fixture artifact — covered in tests/test_artifact_cli.py; the Docker path is the same code.
  7. The release exists with both assets; fetch-then-verify proven against it.
  8. ruff, mypy, and lint-imports all clean.
  9. smoke.sh exits 1 on a provenance mismatch, 0 on a match.

Criterion 7 — CI green on this PR, GHCR publish on merge — is what this PR exercises.

Deviations from the plan

  • lockfile.read parses the JSON before validating it. The plan's model_validate_json with two except branches could not work: pydantic v2 folds a syntax error into ValidationError, which is itself a ValueError, so the second branch was unreachable and a malformed file reported as a schema problem.
  • The model-v0.1.0 release already existed with both assets carrying exactly the digests the plan records, so only the lockfile half of scripts/release-model.sh was run. The script's guard rails were verified separately: it refuses a missing version and a version mismatch before calling gh.
  • Two small lint-driven test fixes: pytest.raises(ValidationError) instead of blind Exception (ruff B017), and the loopback-server imports hoisted to the top of the test module (E402).

🤖 Generated with Claude Code

edward-sf and others added 14 commits September 1, 2026 22:53
Moves the trained model's bytes out of git and into a GitHub Release,
leaving a checksum-pinned lockfile in their place. Supersedes Milestone
1's "artifact committed to repo" decision.

Key decisions: build-time fetch (image stays self-contained, boot gate
untouched), CI always builds against the real release, a committed
models/model.lock.json pinning tag + sha256, and a creditboost-artifact
verify guard that runs inside the Docker build so an image containing a
fixture-provenance or ECOA-violating artifact cannot be built at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ten TDD tasks from the model artifact storage spec. Ordered around the
spec's sequencing constraint: the model-v0.1.0 release is cut from the
already-committed artifact (Task 6) before the Dockerfile flips to
fetching (Task 7) and before models/*.json leaves HEAD (Task 9), so
there is no commit range in which nothing can build.

Task 1 extracts file_sha256 out of data.py first: data.py imports
scikit-learn, which the Docker builder stage does not install, so the
artifact CLI could not have reached it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plan's two dependency-guard tests asserted `'sklearn' not in
sys.modules`. That check fails on correct code: xgboost imports sklearn
itself whenever sklearn is installed, so artifact_cli -> artifact ->
xgboost -> sklearn regardless of layering. Verified empirically. Both
tests now name creditboost.data and creditboost.train instead, which is
the idiom tests/test_api.py:186-200 already established and explained.

The spec claimed offline builds stop working. They never worked: the
builder stage already runs `pip install` against PyPI. The real change
is two external hosts becoming three.

Also records the blast radius of a deleted release: images stay pullable
from GHCR, so the cost is rebuilding an old commit, not deploying it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces M1's hand-rolled sys.modules subprocess test with a declarative
import-linter contract in pyproject.toml, run by lint-imports in CI. The
rule is static, so a static check suits it: verified locally that the
contract catches a *guarded* transitive import (serve.app -> banding ->
data) and exits 1, where the old test covered serve/ alone and embedded
its probe in an unlintable Python string.

Also strengthens scripts/smoke.sh to assert /health reports
provenance: production. It already runs against the live container in
CI's build job, which gates the GHCR push, but never checked provenance
-- the last place a fixture model could have slipped through.

Softens the release-deletion invariant to match a single-maintainer
portfolio project: recoverable via docker cp from a pulled GHCR image,
with the lockfile digest proving the recovered bytes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The artifact CLI runs in the Docker builder stage, which installs the base
package only and therefore has no scikit-learn. data.py imports sklearn at
module scope, so the hash helper had to move somewhere both sides can import.

Replaces the hand-rolled sys.modules test with a declarative import-linter
contract. The rule is static, so a static check suits it: the contract sees
the whole import graph, catches transitive and guarded imports, reports the
offending chain, and needs no stringified Python that ruff and mypy cannot
see. It also covers seven modules where the test covered one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Pins a release tag plus a sha256 per asset. No model_version field: the CLI
cross-checks the downloaded metadata against config.MODEL_VERSION rather than
adding a third place the version could disagree with itself.

read() parses the JSON before validating it, rather than using
model_validate_json as the plan sketched: pydantic v2 folds a syntax error
into ValidationError, which would have reported a malformed lockfile as a
schema problem and left the plan's second except branch unreachable.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Checks the lockfile digests, that metadata.version matches MODEL_VERSION,
feature order, ECOA prohibited features in both the sidecar and the booster's
own feature_names, and provenance, then proves the whole thing by calling
artifact.load(). Replaces the committed-artifact tests with a guard that
rejects a dirty artifact rather than asserting one particular artifact is
clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Downloads via urllib rather than curl, which keeps an apt-get layer out of the
Docker builder stage. Transient failures back off and retry; a 404 fails
immediately, because a missing release is a defect and retrying it only delays
a truthful failure. Tested against a loopback http.server, so the suite stays
hermetic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Shell owns the gh interaction; Python owns the lockfile's JSON shape, so the
pydantic model stays the only definition of the file format. The script
refuses to publish a fixture artifact or a version that disagrees with the
metadata, before it calls gh.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Publishes the already-committed, already-reviewed production artifact as a
GitHub Release. The model itself is unchanged; only where its bytes live is.
The committed copy is removed in a later task, once the build no longer needs
it.

The release tag already existed with both assets carrying exactly the digests
this lockfile pins, so only the lockfile half of scripts/release-model.sh was
run; fetch-then-verify was proven against the live release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The builder stage fetches and verifies the release assets in a single RUN --
a cached layer of fetched-but-unverified bytes would be a trap -- and the
runtime stage copies from the builder rather than from the build context.
docker build still takes zero arguments; the lockfile carries everything.

smoke.sh now asserts /health reports provenance: production, the last gate
before the GHCR push.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
models/model_meta.json stops being committed in the next task. verify already
guarantees the shipped artifact's metadata version equals config.MODEL_VERSION,
so the two sources are interchangeable by construction.

Also runs lint-imports in the lint job, so the layering contract gates CI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bytes now live in the model-v0.1.0 GitHub Release, pinned by
models/model.lock.json. History is deliberately not rewritten: the problem was
that every future retrain appends another copy, which deleting from HEAD
solves completely.

The two committed-artifact tests are removed, not lost. Their assertions live
in creditboost-artifact verify, which runs at every build rather than only
under pytest, and in tests/test_artifact_cli.py, which asserts the guard
rejects a dirty artifact rather than that one artifact happens to be clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CLAUDE.md is the file future work reads first; leaving it describing a
committed artifact would actively mislead.

Also notes the macOS python.org empty-trust-store gotcha next to the existing
libomp one: it makes the newly documented creditboost-artifact fetch fail with
CERTIFICATE_VERIFY_FAILED, which hit this machine during the milestone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@edward-sf

Copy link
Copy Markdown
Owner Author

Phase 2 focused on removing the model artifact from our commits to reduce storage usage. This phase was intentionally narrow in scope to correct the original bad-practice, which was initially approved simply to facilitate a cleaner P1 thin-slice.

✅ Changes have been approved and planning for Phase 3 has started.

@edward-sf
edward-sf merged commit 16cc461 into main Sep 2, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant