Skip to content

minimax-h3 0.16.0rc1: fused NDHWC fp16 VAE encoder, vae_round_trip (cr-306) - #956

Merged
PaulFidika merged 8 commits into
rc/0.26.0from
claude/cr-306-h3-vae
Nov 17, 2026
Merged

PaulFidika merged 8 commits into
rc/0.26.0from
claude/cr-306-h3-vae

Conversation

@PaulFidika

@PaulFidika PaulFidika commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

cr-306. Paul, 11-16, on ComfyUI's "MiniMax H3's video VAE is ~2x faster": "if it's worthwhile, please incorporate it", "we should be using the --fast fp16_accumulation".

  • VAE_ENCODER declares the video VAE's causal-conv encoder to the Runtime's fusion.CausalConvEncoder (cozy-runtime #2223, merged 5d3ec26e): NDHWC fp16, each per-frame GroupNorm, SiLU and pad one pass, each conv accumulating in fp16 with its resnet block's residual in its epilogue (comfy-kitchen Release corrected 15-second H3 package as 1.10.1 #167's kernels). Installed by H3Model and H3VideoVAE.
  • vae_round_trip: the video VAE alone, encode then decode, stages vae_encode/vae_decode, on H3VideoVAE (the release's vae only: a function of H3Model must load its placeable text encoder). Its benchmark (release-gate/sessions/h3-vae.sh) and release-gate cell (suites/runtime/h3-vae.sh).
  • official.encode_clip/decode_chunks: the VAE math OfficialH3Pipeline ran, now shared with OfficialH3VideoVAE; PIXEL_STATS is ImageNet's, the pipeline's held to it.
  • minimax-h3 0.16.0rc1. Its Runtime floor stays the RC's minor (>=0.47.0.dev0, RELEASING.md); the 0.27 promotion's floor must be the first release with fusion.CausalConvEncoder (0.48.0).

Proof

pod-session 44069065274, one RTX 5090, int8 lane, 1344x768, warm passes, same pod: RC Runtime with packages 973cd7d (this branch before the encoder) against dev-build 44063767252 (#2223) with e9538d1 (this branch; 0966958 adds only the version):

before after
encode, 129 frames 16.0 / 16.1 s 7.3 / 7.3 s
decode, 129 frames 4.7 / 4.7 s 4.5 / 4.5 s
encode, 362 frames 44.4 / 44.3 s 20.3 / 20.2 s
decode, 362 frames 14.3 / 14.2 s 13.6 / 13.5 s

Plan report: the encoder fused (33 convs, 12 resnet blocks, 4 downsamplers), the decoder's 36 blocks as before. Latents 65.3 / 65.9 dB against before, the same latents' decode bit-identical, the round trip 57.2 / 58.3 dB (the VAE's own loss ~38 dB); frame 64 side by side shows no seam or border.

…cr-306)

A private entrypoint that reads a video at its H3 canvas, encodes it as upscale does (vae_encode) and decodes
those latents, or a given run's, as generate does, streamed to its MP4 (vae_decode). It needs the VAE's
weights alone, so the VAE's speed against ComfyUI and its release-gate cell cost a few minutes of a pod.
It returns the latents in float32 and every 32nd decoded frame, for the PSNR of one build against another.
H3Model places its text encoder (placeable), so a function of it must load that encoder too: pod-session 44045202003's
run PRO-2 failed model_preparation_failed. H3VideoVAE is the same pipeline with only its vae loaded, the same fused
decoder blocks and 8-bit decoder attention.
…oVAE)

A Model's construction may build only the components its methods declare: pod-session 44048259303's run PRO-2 failed
undeclared_construction with the whole pipeline. The VAE's encode and decode math moves to official.encode_clip and
decode_chunks, which OfficialH3Pipeline and OfficialH3VideoVAE both run; PIXEL_STATS is ImageNet's, the pipeline's
held to it.
…cumulation (cr-306)

VAE_ENCODER declares the encoder to Runtime's fusion.CausalConvEncoder: its causal conv and the padding it
reads, the per-frame GroupNorm, the resnet blocks (norm1/conv1, norm2/conv2, conv_shortcut) and the
downsampler's reflected right column and bottom row. Each GroupNorm, SiLU and pad runs as one pass, each conv
accumulates in fp16 with its bias and the block's residual in its epilogue (comfy-kitchen #167), as ComfyUI's
encoder runs under --fast fp16_accumulation (Paul, 11-16).
@PaulFidika PaulFidika changed the title minimax-h3: fused NDHWC fp16 VAE encoder, vae_round_trip (cr-306) minimax-h3 0.16.0rc1: fused NDHWC fp16 VAE encoder, vae_round_trip (cr-306) Nov 17, 2026
@PaulFidika
PaulFidika marked this pull request as ready for review November 17, 2026 11:56
@PaulFidika
PaulFidika added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 17, 2026
…(0.47.0.dev2307, cozy-runtime #2223); the pipeline's VAE typed as the TileBatchedVideoVAE it is
@PaulFidika
PaulFidika added this pull request to the merge queue Nov 17, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Nov 17, 2026
@PaulFidika
PaulFidika enabled auto-merge November 17, 2026 12:03
@PaulFidika
PaulFidika added this pull request to the merge queue Nov 17, 2026
Merged via the queue into rc/0.26.0 with commit d1a09cb Nov 17, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant