feat(rvq): publish encoder/decoder .keras reference weights#59
Conversation
Extends the "ship both .tflite (deployable) and .keras (reference)" packaging convention established for the hybrid denoiser (#47) to RVQ, for consistency across all trained-weight families. ## Changes - export_for_deployment() now also saves encoder.keras (new -- this model had no .keras export path before) alongside the existing decoder.keras step. Both are recorded in codec_spec.json/deploy_manifest.json's encoder/decoder sections and DeploymentArtifacts. - family_registry.py's RVQ CodecFamilySpec.required_artifacts now includes encoder.keras and decoder.keras (previously decoder.keras existed locally but was never part of the tracked/published contract at all). - artifact_contract.py: new ENCODER_KERAS/DECODER_KERAS constants, both added to RVQ_HF_FILE_RENAMES (unrenamed passthrough, matching how hybrid's denoiser_gain_model.keras publishes). - model_card.py's RVQ file table and the generated experiment docs (scripts/render_experiment_docs.py) now list both .keras files. - Updated docs/deployment.md, docs/release-contract.md, and docs/models/{ecg,ppg}.md's deploy/ directory listings to include encoder.keras (decoder.keras was already documented). ## Validation - Repackaged all 11 local RVQ goldens via `compressionkit golden repackage <id>` (re-exports from already-trained weights, no retraining) -- all now have both encoder.keras and decoder.keras. - `golden validate-all --strict-release` passes 33/33. - Verified RVQCodec still loads/round-trips correctly from a repackaged deploy dir (the LiteRT runtime never touches the new .keras files -- they're reference-only). - 452 tests pass (1 updated fixture), ruff clean, zensical build clean. - Did not republish to HuggingFace this session. Closes #56
There was a problem hiding this comment.
Pull request overview
This PR extends the deploy/release contract for the RVQ family to publish both deployable INT8 LiteRT artifacts (.tflite/.h) and reference float32 Keras artifacts (encoder.keras + decoder.keras), matching the packaging convention already used for the hybrid family.
Changes:
- Export pipeline:
export_for_deployment()now savesencoder.kerasalongside the existingdecoder.keras, and records both incodec_spec.jsonanddeploy_manifest.json. - Contract/registry: RVQ
required_artifactsand the artifact-contract rename table are updated to includeencoder.kerasanddecoder.keras; deploy-contract tests are updated accordingly. - Documentation: model card + deploy/release docs are updated to list the new artifact(s), and experiment docs are regenerated to include the
.kerasfiles.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
compressionkit/export/deploy.py |
Adds RVQ encoder.keras export and includes it in manifests/specs/readme. |
compressionkit/export/family_registry.py |
Updates RVQ required_artifacts to include encoder.keras and decoder.keras. |
compressionkit/export/artifact_contract.py |
Adds ENCODER_KERAS/DECODER_KERAS constants and includes them in RVQ HF rename passthroughs. |
compressionkit/export/model_card.py |
RVQ model-card file table now lists encoder.keras and decoder.keras. |
tests/test_deploy_contract.py |
Updates strict RVQ deploy-contract fixture to include the new .keras artifacts. |
scripts/render_experiment_docs.py |
Updates generated RVQ deploy-artifacts list to include .keras artifacts (but currently still references sample_stimulus.npz). |
docs/deployment.md |
Adds encoder.keras to the RVQ deploy artifact table. |
docs/release-contract.md |
Adds encoder.keras to AI-specific file list. |
docs/models/ppg.md |
Adds encoder.keras to RVQ deploy directory listing. |
docs/models/ecg.md |
Adds encoder.keras to RVQ deploy directory listing. |
docs/experiments/ppg-rvq-2x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-4x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-4x-prior.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-8x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-8x-prior.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-16x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ppg-rvq-32x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-2x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-4x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-4x-prior.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-8x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-8x-prior.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-16x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-32x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
docs/experiments/ecg-rvq-64x.md |
Regenerated deploy-artifacts list to mention .keras artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "- `decoder.tflite` / `decoder.h` — decoder (float32 + optional INT8).", | ||
| "- `decoder.keras` — float32 Python reference decoder.", | ||
| "- `codebook.npz` / `codebook.h` — RVQ codebook tables.", | ||
| "- `sample_stimulus.npz` — license-safe input/output reference frames.", |
|
Good catch, but this is a pre-existing ambiguity, not something this PR introduced (I didn't touch that `sample_stimulus.npz` line -- only added the two `.keras` bullets around it). Turns out both names are "correct" depending on which code path built the local deploy dir: `export_for_deployment()` (called by the training recipe) only writes `sample_data.npz` locally, while `compressionkit/experiments/repackage.py`'s `repackage_rvq_golden()` (used by `compressionkit golden repackage`, which I used to add the new `.keras` files to all 11 goldens in this PR) also writes `sample_stimulus.npz` via a separate `export_stimulus_npz()` call -- and `sample_data.npz -> sample_stimulus.npz` is the actual HF publish-time rename regardless. So the doc section's naming is ambiguous about local-vs-published conventions across two different build paths, not simply wrong. Filed as #60 to investigate properly rather than guessing at a fix here. |
Summary
Extends the "ship both
.tflite(deployable) and.keras(reference)" packaging convention established for the hybrid denoiser (#47) to RVQ, for consistency across all trained-weight families.Changes
export_for_deployment()now also savesencoder.keras(new — this model had no.kerasexport path before) alongside the existingdecoder.kerasstep. Both are recorded incodec_spec.json/deploy_manifest.json's encoder/decoder sections andDeploymentArtifacts.family_registry.py's RVQCodecFamilySpec.required_artifactsnow includesencoder.kerasanddecoder.keras(previouslydecoder.kerasexisted locally but was never part of the tracked/published contract at all).artifact_contract.py: newENCODER_KERAS/DECODER_KERASconstants, both added toRVQ_HF_FILE_RENAMES(unrenamed passthrough, matching how hybrid'sdenoiser_gain_model.keraspublishes).model_card.py's RVQ file table and the generated experiment docs (scripts/render_experiment_docs.py) now list both.kerasfiles.docs/deployment.md,docs/release-contract.md, anddocs/models/{ecg,ppg}.md'sdeploy/directory listings to includeencoder.keras(decoder.keraswas already documented).Testing
compressionkit golden repackage <id>(re-exports from already-trained weights, no retraining) — all now have bothencoder.kerasanddecoder.keras.golden validate-all --strict-release→ 33/33.RVQCodecstill loads/round-trips correctly from a repackaged deploy dir (the LiteRT runtime never touches the new.kerasfiles — they're reference-only).uv run pytest -q— 452 passed (1 updated fixture).uv run ruff check ./uv run ruff format --check .— clean.uv run zensical build— clean.Closes #56