Skip to content

feat(rvq): publish encoder/decoder .keras reference weights#59

Merged
apage224 merged 1 commit into
mainfrom
issue-56-rvq-publish-keras-reference
Jul 3, 2026
Merged

feat(rvq): publish encoder/decoder .keras reference weights#59
apage224 merged 1 commit into
mainfrom
issue-56-rvq-publish-keras-reference

Conversation

@apage224

@apage224 apage224 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Extends the "ship both .tflite (deployable) and .keras (reference)" packaging convention established for the hybrid denoiser (#47) to RVQ, for consistency across all trained-weight families.

Changes

  • export_for_deployment() now also saves encoder.keras (new — this model had no .keras export path before) alongside the existing decoder.keras step. Both are recorded in codec_spec.json/deploy_manifest.json's encoder/decoder sections and DeploymentArtifacts.
  • family_registry.py's RVQ CodecFamilySpec.required_artifacts now includes encoder.keras and decoder.keras (previously decoder.keras existed locally but was never part of the tracked/published contract at all).
  • artifact_contract.py: new ENCODER_KERAS/DECODER_KERAS constants, both added to RVQ_HF_FILE_RENAMES (unrenamed passthrough, matching how hybrid's denoiser_gain_model.keras publishes).
  • model_card.py's RVQ file table and the generated experiment docs (scripts/render_experiment_docs.py) now list both .keras files.
  • Updated docs/deployment.md, docs/release-contract.md, and docs/models/{ecg,ppg}.md's deploy/ directory listings to include encoder.keras (decoder.keras was already documented).

Testing

  • Repackaged all 11 local RVQ goldens via compressionkit golden repackage <id> (re-exports from already-trained weights, no retraining) — all now have both encoder.keras and decoder.keras.
  • golden validate-all --strict-release → 33/33.
  • Verified RVQCodec still loads/round-trips correctly from a repackaged deploy dir (the LiteRT runtime never touches the new .keras files — they're reference-only).
  • uv run pytest -q — 452 passed (1 updated fixture).
  • uv run ruff check . / uv run ruff format --check . — clean.
  • uv run zensical build — clean.
  • Did not republish to HuggingFace this session.

Closes #56

Extends the "ship both .tflite (deployable) and .keras (reference)"
packaging convention established for the hybrid denoiser (#47) to RVQ,
for consistency across all trained-weight families.

## Changes

- export_for_deployment() now also saves encoder.keras (new -- this
  model had no .keras export path before) alongside the existing
  decoder.keras step. Both are recorded in codec_spec.json/deploy_manifest.json's
  encoder/decoder sections and DeploymentArtifacts.
- family_registry.py's RVQ CodecFamilySpec.required_artifacts now
  includes encoder.keras and decoder.keras (previously decoder.keras
  existed locally but was never part of the tracked/published contract
  at all).
- artifact_contract.py: new ENCODER_KERAS/DECODER_KERAS constants, both
  added to RVQ_HF_FILE_RENAMES (unrenamed passthrough, matching how
  hybrid's denoiser_gain_model.keras publishes).
- model_card.py's RVQ file table and the generated experiment docs
  (scripts/render_experiment_docs.py) now list both .keras files.
- Updated docs/deployment.md, docs/release-contract.md, and
  docs/models/{ecg,ppg}.md's deploy/ directory listings to include
  encoder.keras (decoder.keras was already documented).

## Validation

- Repackaged all 11 local RVQ goldens via `compressionkit golden
  repackage <id>` (re-exports from already-trained weights, no
  retraining) -- all now have both encoder.keras and decoder.keras.
- `golden validate-all --strict-release` passes 33/33.
- Verified RVQCodec still loads/round-trips correctly from a
  repackaged deploy dir (the LiteRT runtime never touches the new
  .keras files -- they're reference-only).
- 452 tests pass (1 updated fixture), ruff clean, zensical build clean.
- Did not republish to HuggingFace this session.

Closes #56

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the deploy/release contract for the RVQ family to publish both deployable INT8 LiteRT artifacts (.tflite/.h) and reference float32 Keras artifacts (encoder.keras + decoder.keras), matching the packaging convention already used for the hybrid family.

Changes:

  • Export pipeline: export_for_deployment() now saves encoder.keras alongside the existing decoder.keras, and records both in codec_spec.json and deploy_manifest.json.
  • Contract/registry: RVQ required_artifacts and the artifact-contract rename table are updated to include encoder.keras and decoder.keras; deploy-contract tests are updated accordingly.
  • Documentation: model card + deploy/release docs are updated to list the new artifact(s), and experiment docs are regenerated to include the .keras files.

Reviewed changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
compressionkit/export/deploy.py Adds RVQ encoder.keras export and includes it in manifests/specs/readme.
compressionkit/export/family_registry.py Updates RVQ required_artifacts to include encoder.keras and decoder.keras.
compressionkit/export/artifact_contract.py Adds ENCODER_KERAS/DECODER_KERAS constants and includes them in RVQ HF rename passthroughs.
compressionkit/export/model_card.py RVQ model-card file table now lists encoder.keras and decoder.keras.
tests/test_deploy_contract.py Updates strict RVQ deploy-contract fixture to include the new .keras artifacts.
scripts/render_experiment_docs.py Updates generated RVQ deploy-artifacts list to include .keras artifacts (but currently still references sample_stimulus.npz).
docs/deployment.md Adds encoder.keras to the RVQ deploy artifact table.
docs/release-contract.md Adds encoder.keras to AI-specific file list.
docs/models/ppg.md Adds encoder.keras to RVQ deploy directory listing.
docs/models/ecg.md Adds encoder.keras to RVQ deploy directory listing.
docs/experiments/ppg-rvq-2x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-4x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-4x-prior.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-8x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-8x-prior.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-16x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ppg-rvq-32x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-2x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-4x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-4x-prior.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-8x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-8x-prior.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-16x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-32x.md Regenerated deploy-artifacts list to mention .keras artifacts.
docs/experiments/ecg-rvq-64x.md Regenerated deploy-artifacts list to mention .keras artifacts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

"- `decoder.tflite` / `decoder.h` — decoder (float32 + optional INT8).",
"- `decoder.keras` — float32 Python reference decoder.",
"- `codebook.npz` / `codebook.h` — RVQ codebook tables.",
"- `sample_stimulus.npz` — license-safe input/output reference frames.",
@apage224

apage224 commented Jul 3, 2026

Copy link
Copy Markdown
Contributor Author

Good catch, but this is a pre-existing ambiguity, not something this PR introduced (I didn't touch that `sample_stimulus.npz` line -- only added the two `.keras` bullets around it).

Turns out both names are "correct" depending on which code path built the local deploy dir: `export_for_deployment()` (called by the training recipe) only writes `sample_data.npz` locally, while `compressionkit/experiments/repackage.py`'s `repackage_rvq_golden()` (used by `compressionkit golden repackage`, which I used to add the new `.keras` files to all 11 goldens in this PR) also writes `sample_stimulus.npz` via a separate `export_stimulus_npz()` call -- and `sample_data.npz -> sample_stimulus.npz` is the actual HF publish-time rename regardless. So the doc section's naming is ambiguous about local-vs-published conventions across two different build paths, not simply wrong.

Filed as #60 to investigate properly rather than guessing at a fix here.

@apage224
apage224 merged commit 93dfe6a into main Jul 3, 2026
2 checks passed
@apage224
apage224 deleted the issue-56-rvq-publish-keras-reference branch July 3, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish RVQ decoder/encoder .keras reference weights to HuggingFace for consistency with hybrid

2 participants