minimax_music3: performance pack — native rope/SwiGLU lowering + opt-in CFG reuse, chunk hop, batched ensemble takes, q4_k depth

[iamwavecut]

What

A performance pack for MiniMax Music 3 (improving the existing family per CONTRIBUTING, building on #241/#243/#271 by @JoeMattie). 13 focused commits; out of the box only the flow lowering changes — everything else is opt-in and the defaults reproduce the current upstream trajectory.

Default-on (changes flow internals, spectrally equivalent):

• Native ggml_rope_ext NEOX lowering instead of the slice/concat rope chains, both flow batches. Also works around a gallocr view-aliasing hazard we bisected: a freshly allocated batch-1 flow graph corrupts its first ~4 invocations, bitwise-reproducibly (batch 2 is saved by an explicit ggml_repeat). Repro/diagnostics ship as MM3_DC_B1_* envs; MM3_LEGACY_ROPE=1 restores the old lowering.
• Fused ggml_swiglu_swapped for the flow FFN (our layout is [states|gate] — plain ggml_swiglu silences the wrong half, mel 25 dB off; the swapped variant is exact). MM3_LEGACY_GLU=1 restores.
• Compact semantic LM head (exact quantized-row repack, byte-identical sampling — gate below), CUDA graph-cache eviction on release, pipeline buffer reuse.

Opt-in (upstream defaults preserved):

• flow_uncond_interval=N — reuse the CFG guidance delta between steps. N=2: flow −25-30%, mel-L1 ~0.3 dB vs exact; N=1 (default) = exact trajectory.
• flow_chunk_hop_frames — the stock chunking (200-frame chunks, hop 100) denoises every second twice. hop=150: flow −35%; crops and the carry window are rederived consistently (hop=100 reproduces the stock geometry byte-for-byte).
• ensemble_takes=K (1..16) — decode K takes of one prompt in a single batched AR pass. The LM+depth stages are weight-bandwidth-bound, so K takes cost far less than K runs; flow/vocoder run per take, outputs land as take_NN via --out-dir. Adds an additive export_batched_decode_state() to the Qwen decode runtime (no behavior change for existing callers).
• ensemble_prefix_frames=N — intro-lock: the first N frames decode once as a shared master trajectory, then takes fork (take_01 continues the master exactly). "One intro, K continuations."
• Depth decoder q4_k (upstream file): −24% depth-stage time, panel-clean; option only, default stays q8_0.

Numbers (fresh run on a clean A40 SECURE pod, driver 580.159.04, this branch, CUDA)

60 s track, 30 flow steps, one seed, single run each (fresh clone of this branch, canonical build script):

run	wall	vs defaults
def60 — upstream defaults	95.9 s	baseline
slim60 — compact LM head only	88.8 s	−7.4% (output byte-identical, see gate)
dc2_60 — flow_uncond_interval=2	87.1 s	−9.2%
hop150_60 — flow_chunk_hop_frames=150	79.4 s	−17.2%
combo60 — interval=3 + hop=150 + q4_k depth + slim head	68.3 s	−28.8%
ens6 — ensemble_takes=6, combo options	223.4 s = 37.2 s/song	−61% per song

20 s sanity pair: native lowering 33.9 s vs MM3_LEGACY_ROPE/GLU 36.2 s (−6% from the lowering itself).

From the longer campaign behind this: the ensemble K-curve on a 60 s track runs 89.3/72.4/67.2/64.4/63.7 s per AR pass for K=1/3/6/10/16 (LM+depth are weight-bandwidth-bound, so extra takes are nearly free until flow dominates). Every recipe step was gated on a spectral panel (chroma/harmonicity/dynamics/onsets staying in the reference band) plus human listening.

Build / run

scripts/build_linux.sh --build-dir build/rel --build-type Release --backend cuda --cuda-arch native \
  --model-set custom --models minimax_music3 --with-tests --target audiocpp_cli --jobs $(nproc)

Baseline run (upstream defaults, only the flow lowering differs):

build/rel/audiocpp_cli --task gen --family minimax_music3 --model <models> --backend cuda \
  --text "<caption>" --request-option "lyrics=<lyrics>" --request-option duration_sec=60 \
  --request-option num_inference_steps=30 --request-option seed=20260827 --metrics --out out.wav

Full opt-in recipe (what combo60 in the table runs):

... same as above, plus:
  --request-option flow_uncond_interval=3 --request-option flow_chunk_hop_frames=150 \
  --session-option minimax_music3.rvq_depth_decoder_gguf=rvq_depth_decoder_q4_k.gguf \
  --session-option minimax_music3.language_model_gguf=language_model_q4_0_slim.gguf

Ensemble: add --request-option ensemble_takes=6 --out-dir out/ (takes land as take_NN.wav).

Validation

• minimax_music3_graph_release_policy_test, minimax_music3_lm_head_test, minimax_music3_pipeline_buffers_test: pass.
• Slim head: output byte-identical to the full head (sha256 gate in the fresh run).
• hop=100 path: byte-identical geometry to stock.

Spectrograms from the fresh run (branch evidence/mm3-perf-20260827 holds these + the raw table):

defaults (95.9 s) vs full opt-in recipe (68.3 s), same seed:

legacy lowering vs native lowering (20 s, same seed — diverged takes, same character):

Known limitations

• Native rope/SwiGLU are not byte-identical to the previous lowering. On matched token streams the flow output sits at mel-L1 ~0.15 dB; on a full same-seed run the AR sampler eventually picks different tokens (any numeric shift in the loop does this), so tracks diverge into equally-valid takes (~0.8 dB log-spec apart, same spectral character — spectrograms above). MM3_LEGACY_ROPE/GLU keep the exact old path if you need it.
• Validated on CUDA only; we saw minimax_music3: degenerate (unconditioned, drone-like) output on Metal — runs at full speed, no errors #303 (Metal) and did not touch Metal-specific paths, but flow lowering changes should be re-checked there.
• ensemble_takes peak VRAM grows ~0.7 GB per take at 60 s (K=6 ≈ 12.4 GB total on A40).
• The MMQ fixup-buffer memset also ships as a standalone micro-PR; whichever lands first, the other merges clean (identical hunk).

[0xShug0]

@iamwavecut Thanks for the PR. One issue I recently found is that some optimizations were rejected by me because, on my setup, they only showed noise-level performance changes, while on different hardware the gains could be significant. So I may miss many useful optimizations.

RTX 5090:

Case	Wall ms	RTF	Idle VRAM	Peak VRAM
main default	12802.5	0.7652	2232 MB	12576 MB
PR default	12522.9	0.7501	2224 MB	12564 MB
main mem_saver=false	10693.3	0.6392	15146 MB	16714 MB
PR mem_saver=false	10289.0	0.6163	15142 MB	16712 MB
main mem_saver=true	12874.5	0.7695	2232 MB	12576 MB
PR mem_saver=true	12499.0	0.7487	2224 MB	12564 MB

I’m not a CI expert, so I’m not sure whether the new cuBLASLt dependency could break Docker images or prebuilts. I need a bit more time to investigate.

[0xShug0]

Also I think the changes to external/ggml/src/ggml-cuda/mmq.cuh was merged already.