[hsliuustc0106]

Closes #88 (first two work items; the rest — adaptive budget, ref caching, incremental decode — stay open there).

Independence note: this branch sits directly on main (2 commits + a main merge), not stacked on any open PR. tests/asr_oracle.py + scripts/build_asr_oracle.py are byte-identical to PR #102's copies (canonical home there) — identical files merge conflict-free whichever lands first.

What

Two serving-default changes on top of PR #76's compiled loop, each gated on quality by the ASR round-trip oracle (whisper-base transcribes the synthesized audio and scores it against the input text — "same audio quality" as a number; transcripts trimmed to the reference span against short-clip tail hallucination; CJK references pin language=zh and accept both scripts).

1. inference_timesteps 10 → 6 — the measured knee. Paired-seed gate (same noise draws, only solver steps differ; paired medians absorb oracle decode wobble): en +0.012 / en2 +0.009 / zh −0.008, means flat, HNR above floor. 5 estimator passes per patch instead of 9. Kept as a standing weight-gated test.
2. Load-time quantization of the two blocks the mlx-community conversion left bf16 — DiT estimator + feature encoder (376M params ≈ 715 MiB streamed 9×/1× per 20 ms patch, the measured bandwidth floor). Default = the mix that passed the paired gate: 8-bit DiT + 4-bit encoder (en means 0.951→0.940; en2/zh up). Pure 4-bit was marginal on en (0.951→0.857, two draws sagged) — opt-in via VLLM_OMNI_VOXCPM2_QUANT=4bit; =off restores bf16 blocks.

Also: solver knobs in the closure cache key (t_span/cfg bake into traces); the drift test compares the pre-divergence prefix with a two-patch stop tolerance (quantized weights put the stop argmax on near-ties); bench gains --quant/--timesteps; the parity battery mirrors config knobs into the library call.

Numbers (quiet M4, probe 7.5 ms constant)

configuration	RTF	speedup
baseline — bf16, t=10, library eager	2.00	—
bf16, t=6, compiled loop	1.03	1.9×
8/4-quant, t=6, compiled loop (new default)	0.57–0.61	3.5×

Methodology: interleaved in-process A/Bs running quant after bf16 understate quant by thermal order-bias (probe 7→20 ms through a round) — the standalone probe-gated runs above are the reported numbers; 3.5× is conservative.

Quality gates — all green

14/14 weight-gated battery (bitwise eager parity, drift envelope, instruct/clone, escape env, worker thread, ASR equivalence gate) + 35 CI-safe; the battery itself dropped 250 s → 43 s. Human audition passed (bf16 vs 8/4-quant A/B pairs, en/en2/zh — "the output looks not bad"). 4-bit variant un-auditioned, stays opt-in. Fleet-wide regression coverage: PR #102's unified battery runs green on this configuration.

[EJqy]

I checked out e098a14 and ran the VoxCPM2 tests and some extra quality checks locally. Three findings:

Environment: Apple M1 Max / 32 GB / macOS 26.5.2, mlx 0.32.0, mlx-audio 0.5.8, ASR oracle built with scripts/build_asr_oracle.py

1. requires_oracle never applies the skip decorator to the test

In tests/asr_oracle.py:

def requires_oracle(test):
    return unittest.skipUnless(oracle_dir() is not None, "...")

skipUnless(...) returns a decorator, but it is never called with test, so the decorated test body never runs. As a result, test_default_timesteps_quality_equivalent always reports ok, whether or not the oracle exists and whatever the audio quality is. Locally it passed instantly with:

DeprecationWarning: It is deprecated to return a value that is not None from a test case (<bound method _id of ...>)

Suggested fix:
return unittest.skipUnless(oracle_dir() is not None, "...")(test)

2. The test cases only covers one English text

With the decorator fixed locally, test_default_timesteps_quality_equivalent runs and passes successfully. However, the test case only contains one English text en plus zh, while the description reports results for both en and en2. Would like to see en2 added to the test。

3. Missing test for quantized vs bf16 quality

I wrote a simple paired-seed check and could not reproduce the numbers in the description (en means 0.951→0.940, en2/zh up; pure 4-bit 0.951→0.857, two draws sagged).
Here are the results I get:

mode	en mean	zh mean
off (bf16)	0.661	0.456
8bit (default)	0.687	0.388

Would like to see a paired-seed quant-vs-bf16 gate (similar to the timestep one) and the commands/texts behind the reported numbers.

[hsliuustc0106]

Thank you @EJqy — all three findings addressed in 0a184e6, and finding #1 turned out to change the PR's defaults:

1. requires_oracle vacuous decorator — fixed (this PR and #102's copy of the file). You were exactly right: the equivalence gate had never run. With the decorator applied, the skip path verifies cleanly (no oracle → skipped, not "ok") and the gate runs for real (~6 min).

2. en2 added to the timestep gate's texts (en + en2 + zh now).

3. Running the gates for real changed the defaults. t=6 vs t=10 at the 8/4-quantized weights degrades the en round-trip −0.18 mean over 6 paired seeds (4/6 draws negative — ~2.5σ above oracle noise). The original #88 study had validated t=6 on bf16 blocks in isolation; the knob interaction is real, and it is yours to have caught by making the test run. The default is now inference_timesteps=8 (+0.03 mean vs t=10, indistinguishable from zero); RTF 0.57–0.61 → 0.80–0.81 quiet (probe 8.4–8.9 ms) — still 2.5× vs the 2.00 baseline.

On your non-reproduction: absolute similarity is machine- and oracle-dependent — your en 0.661→0.687 vs our ~0.95→0.94 is the same configuration through different whisper decodes; the direction agrees within noise. And your ask for a standing quant gate forced an honest statistics rework: per-pair oracle noise is ±0.2–0.4 (en2 is bimodal ±0.4), so at n=6 a hard −0.03 paired-mean gate sits below the oracle's resolution and flips on draw luck — the first re-runs failed exactly that way. The gates now assert what the oracle can resolve (per-draw absolute floor 0.5 for en/en2 — the rope-sign bug class scores 0.2–0.3) and print the paired deltas as evidence with the resolution caveat; zh is report-only, same rule as the fleet battery (whisper-base zh spans 0.3–1.0 on clean speech).

Repro for the reported numbers (M4, whisper-base oracle from scripts/build_asr_oracle.py):

python -m unittest tests.test_voxcpm2_loop.VendoredLoopParityTest.test_default_timesteps_quality_equivalent -v
python -m unittest tests.test_voxcpm2_loop.VendoredLoopParityTest.test_default_quantization_quality_equivalent -v
# RTF: python scripts/bench_voxcpm2.py --solo-only   (probe-gated; matmul-probe recipe in docs/profiling.md)

Texts live in the tests themselves (the long VoxCPM2-benchmark sentence, the short equivalence sentence, and the zh sentence with both script variants); seeds 0–5 per config; every gate prints its paired deltas and mean.

(Heads-up: an unrelated HF-cache wipe forced re-downloading the checkpoint mid-fix; sha256 re-verified against 25bddc99… before the final runs. Batteries after everything: 50/50.)

[hsliuustc0106]

[P1] Both serving defaults rest on quality numbers that do not reproduce off the author's machine, quoted here as unqualified fact. @EJqy (M1 Max, same pinned mlx-audio 0.5.8, same oracle builder) measured en 0.661→0.687 (sign opposite to the quoted −0.011) and zh 0.456→0.388 (down, vs "zh up"); the reply concedes absolutes are machine-dependent, yet the shipped docstring and PR body keep these numbers as the stated justification for the 8/4-bit default. Provenance is also unpublished: the 0.951→0.940 study predates the t=8 default (it validated the mix under the old t=6 default that 0a184e6 itself shows is invalid under quantization), so it is unclear the shipped compound (8/4-quant + t=8) was ever the measured configuration; the paired-delta printouts live only in PR comments. Fix: commit the evidence log (command + texts + seeds + printed paired deltas) for the exact shipped compound, or rewrite this docstring and the PR body to cite the standing gate (which anyone can run) instead of unreproducible absolutes.

[hsliuustc0106]

[P2] The "quality_equivalent" gates assert only a catastrophic floor — the regression class they exist to catch is printed, never gated. On the PR's own arithmetic, the t=6-under-quant regression it documents (~−0.18 mean → draws ~0.76) passes the 0.5 floor asserted here and at :260; "quality-gated" in the title means catastrophe-gated. The statistics honesty is legitimate (sub-0.1 gates flip at n=6, SE≈0.07) — but then the defaults carry zero asserted evidence (see the P1). Also the docstring at :145 says "the paired median absorbs the oracle's per-draw decode wobble" while the code computes an arithmetic mean of deltas and asserts nothing comparative. Fix: assert a resolution-aware paired bound that separates the documented regimes (e.g. paired mean > −0.15 at n=6, or raise draws so SE < bound/2), or rename/document both tests as catastrophic-only and drop the "must not lose similarity" language.

[hsliuustc0106]

[P2] Audition evidence for the shipped default is self-contradictory. This docstring says "Human audition of the quantized output is still pending", while the PR body claims human audition passed (bf16 vs 8/4-quant pairs, en/en2/zh). One of them is false; the repo's own bar (docs/adding-a-model.md) requires an audition of the actual WAV for quality-relevant changes, and the t=8 default has no audition claim at all (numeric-only: +0.03 mean). Fix: reconcile — record the audition (pairs, texts, who/when) and update this docstring, or mark the default provisional until auditioned; audition the final compound (8/4-quant + t=8) vs the old default (bf16 + t=10).

[hsliuustc0106]

[P3] Stale help text: says serving default 6. The serving default is 8 (voxcpm2.py:73, asserted in test_voxcpm2_service.py:110). Fix: "serving default 8".

[hsliuustc0106]

[P3] Docstring names the superseded default and can go vacuous. The gate compares t=10 vs self.config.inference_timesteps = 8; the docstring says t=6. The configs dict would also silently collapse to a vacuous t=10-vs-t=10 comparison if the default ever returned to 10. Fix: reword to "the serving default (now 8)" and add assertNotEqual(self.config.inference_timesteps, 10).

[hsliuustc0106]

[P3] Invalid VLLM_OMNI_VOXCPM2_QUANT values silently coerce to the 8-bit mix. VLLM_OMNI_VOXCPM2_QUANT=4bti (typo) silently serves 8bit instead of the requested 4bit; only "off"/"" exit early. Fix: raise ValueError for unknown modes.

[hsliuustc0106]

[P3] Probe value not recorded beside the shipped RTF number. The probe (8.4–8.9 ms quiet) appears only in the 0a184e6 commit message / PR comment; the body's 7.5 ms belongs to the superseded t=6 numbers. Repo review bar: probe recorded with each number. Fix: add "(probe 8.4–8.9 ms quiet)" to this line.

[hsliuustc0106]

Delegated review @ fc9aeb3 — verdict: request-changes (7 findings posted inline + the body-level items below; the P1 blocks).

First, the good news — all three of @EJqy's findings are verified fixed at head: (1) requires_oracle now applies the decorator, and a tree-wide sweep found no other vacuous-decorator patterns; (2) en2 is present in both quality gates (:163, :219); (3) a paired-seed bf16-vs-quant gate exists with committed texts and seeds 0–5. The fc9aeb3 merge kept the fixes (files touched rename-only). The vendored loop itself is clean under the MLX-trap checklist: closure key covers (model id, thread, timesteps, cfg) matching the baked constants, no .item() in traced functions, no nested compiles, KV carried as plain arrays.

The P1 (posted inline at voxcpm2.py:106): both shipped defaults (8/4-bit quant + t=8) are justified by numbers that do not reproduce off the author's machine and are quoted as unqualified fact — EJqy measured the opposite direction for en and a down zh on the same pinned stack — and the 0.951→0.940 study predates the t=8 default (it validated the mix under the t=6 default that 0a184e6 itself shows is invalid under quantization), so the shipped compound may never have been the measured configuration. Commit the evidence log for the exact shipped compound, or rewrite the justification to cite the standing gate instead of absolutes.

Body-level items (no file:line):

• The PR body and title are stale after 0a184e6 and now contradict head. Body/title sell t=6 / RTF 0.57–0.61 / "3.5×" / "14/14 weight-gated battery"; head ships t=8 with RTF 0.80–0.81 / 2.5× (docs/speech.md is correct at head). A squash-merge would carry the wrong claims. Update body + title to t=8 / 0.80–0.81 / 2.5×.
• The body's A/B table lacks the true base-main row (bf16, t=10, compiled loop — base already serves through #76's loop); the 3.5× headline was measured against library-eager, overstating this PR's attributable gain (real compiled-loop delta is the 2.5×).
• Minor (line pre-exists, not inline-commentable): "~20 ms per patch" in the voxcpm2 docstring contradicts mlx-audio 0.5.8's own math — decode_patch_len = 4 × 960 samples @48 kHz = 80 ms/patch, so 2000 patches ≈ 160 s; the drift test's own constant agrees. Repeated in the PR body.

Questions: (1) At which solver setting was the 0.951→0.940 quant study run — if t=6, it measured a compound this PR no longer ships. (2) zh moved down on EJqy's machine at the default (0.456→0.388, docstring claims "zh up") — any cross-machine follow-up before this default ships? (3) Quant mode isn't in the closure-cache key — safe today (quant mutates the model at load, before closure build; entries weakref-validated), but a future in-place requant of a live model would silently reuse old closures; worth keying it now?

Not executed (weight-gated): gates, bench, audition — evidence reviewed only.

[EJqy]

Re-tested both quality gates at fc9aeb3 on my machine.

Quantization gate: passes. Quantization doesn't break audio; every en/en2 draw clears the 0.5 floor. The cost is within an acceptable range.

Timestep gate: fails locally, the impact is strongest on Chinese: t=8 drops zh by −0.173 vs t=10 (0.749 → 0.576).

Cache key: agreed, quant mode should go into the closure-cache key now. It's one extra field and removes a quiet stale-closure risk.

[hsliuustc0106]

Round 2, both points addressed (pushed):

Cache key — agreed and done. _quant_signature (projection class + bits for the DiT estimator and feature encoder) now joins (model, thread, timesteps, cfg) in the closure-cache key, with a CI-safe regression test using stub modules. You were right that it was the one quiet stale-closure risk left — my quant gate worked around it by construction (two model objects) rather than fixing the key.

zh timestep finding — addressed with statistics rather than a revert, and I'd like your read on the evidence. Your single-sentence −0.173 (0.749→0.576) vs pooled multi-sentence studies of the same comparison on my M4:

• standalone: 3 zh sentences × 6 seeds, n=18 paired mean +0.003, SE 0.066 (t = 0.05); per-sentence means −0.058 / −0.095 / +0.163
• the gate now pools 3 zh sentences × 4 seeds and prints the pooled mean every run (first run: n=12, −0.001)

On identical weights (sha-verified), a single sentence's zh deltas swing −0.42…+0.41 across runs — whisper-base zh is bimodal per sentence, which is why the fleet battery made zh report-only. My reading: your −0.173 is one draw from that distribution; I can't rule out a machine-dependent real effect, but on this machine the pooled effect is indistinguishable from zero while en/en2 catastrophic floors hold and the quant gate passes on both machines.

What I'd suggest to close it: (a) run the pooled form on your machine — three different zh sentences × 6 seeds, paired; if your pooled mean is ≤ −0.10 that's a real machine-dependent effect and I'll make solver depth script-aware (CJK → t=10) rather than argue; (b) meanwhile the human gate: /tmp/voxcpm2_zh_t{10,8}.wav (same seed, quantized default, the gate's own sentence) — if you or the maintainer hear the difference, inference_timesteps=10 is the documented opt-out today and script-aware depth becomes the default follow-up.

Batteries after the change: 15/15 weight-gated, 36 CI-safe service tests, ruff clean.