[EJqy]

Refs #73.

This draft starts MOSS-TTS-Nano support through mlx-audio 0.5.7. The current changes provide the buffered cloning core; I plan to add CLI/HTTP integration, AR streaming, and performance validation in subsequent commits.

Stage 1 — Buffered cloning core (current changes)

• Add MossNanoConfig, model/codec loading, and a family-specific MossNanoService.
• Require reference audio for cloning, expose no preset voices, and accept but ignore ref_text.
• Validate requests before acquiring the generation lock.
• Convert generated audio to 48 kHz mono PCM16/WAV, with explicit stereo downmixing.
• Add tests using stub models for parameter forwarding, reference handling, output format, validation, and failure cleanup.

This stage is callable from Python. CLI/HTTP routing is not wired yet, and streaming requests are explicitly rejected.

Stage 2 — CLI and HTTP integration

• Select Nano from the checkpoint configuration and wire it into the existing TTS CLI and audio endpoints.
• Return the correct audio metadata and add routing/regression tests.

Stage 3 — AR streaming

mlx-audio’s Nano generate() currently buffers the result and rejects stream=True, although its internal AR loop generates codec frames sequentially.

I plan to adapt that loop to expose completed frames and feed them into the stateful codec decoder. The first playable PCM chunk should be available before generation finishes.

Validation will cover token parity, waveform consistency, EOS handling, partial final chunks, and cleanup after interruption.

Stage 4 — End-to-end validation and baseline

• Compare upstream buffered, service buffered, and incremental streaming under the same conditions.
• Report TTFA p50/p95, RTF distributions, chunk jitter, and peak memory, both standalone and with a chat model resident.
• Add the README entry and reproducible usage/benchmark commands.

The initial target is a correct eager implementation with a reproducible baseline. Compilation optimizations will be subsequent.

I’ll keep this PR in draft while these stages are in progress. Any feedback would be helpful.

[hsliuustc0106]

Stream parity question: VoxCPM2 (#71) chose the "finished-buffer chunks" interim for stream: true — the first (only) chunk lands when synthesis completes — so clients that always pass stream: true work uniformly across TTS families; Nano 400s instead until stage 3. _buffered_chunks already yields the finished buffer, so the same honest baseline would be ~3 lines here (yield the PCM once, or sliced at the interval). The explicit rejection is also defensible — no pretend-streaming — but if you keep it, stage 2's README/endpoint notes should say so explicitly. Which way do you want to go?

[EJqy]

I'll go with the finished-buffer interim to match VoxCPM2, so stream: true clients work uniformly.

[hsliuustc0106]

This except Exception also swallows the from mlx_audio.audio_io import read ImportError on line 105 — a server without the [tts] extra would classify every clone request as a 400 ("ref_audio could not be decoded as audio: No module named 'mlx_audio'") instead of a config error; load_moss_nano_model already raises the right RuntimeError for the missing dependency at boot. Suggest hoisting the import above the try and keeping the ValueError conversion for the decode itself only.

[hsliuustc0106]

Nit: there's a 30 s ceiling but no minimum — the Qwen clone path enforces \u22650.5 s (tts/service.py:204) on the grounds that a too-short reference produces garbage cloning. The 0.1 s test fixtures pass validation here. If OpenMOSS documents a floor for Nano, worth encoding it; if not, at least a token minimum (or a documented "you asked for it") would keep the two clone paths consistent.

[hsliuustc0106]

Note for stage 2: nothing in the serving stack calls close() today — server.py has no shutdown hook and VoxCPM2's pool likewise dies with the process — so this is currently uncalled surface (well tested, but unexercised in production). Either wire it into app shutdown when the endpoint lands or drop it until a caller exists.

[EJqy]

Agreed, I'll drop close() for now and reintroduce it alongside a shutdown hook once there's a caller.

[hsliuustc0106]

Follow-up verified good — close() removed, the finished-buffer stream interim matches VoxCPM2's shape, the ImportError hoisted out of the try (config error ≠ 400, with the comment), and the duration gate now has both bounds. All four round-1 inlines resolved correctly.

One new blocker class at this head, see the test-file comment: the battery itself is red — the follow-up commits changed behavior without updating its tests.

[hsliuustc0106]

The battery is red at this head — 10 failures/errors in this file, all one root-cause class: the follow-up commits changed behavior without updating the tests that pinned the old behavior.

Specifically (all reproduced at 12e84ee in a clean worktree):

• test_close_releases_model_and_prevents_further_generation — tests close(), which 9da492d removed (ERRORED: attribute gone)
• test_streaming_is_explicitly_unsupported — pins the removed 400-raise; speech_stream now serves finished-buffer chunks
• test_pcm16_wire_format_for_mlx_arrays (f32/f16/bf16) + test_pcm16_preserves_strided_sample_order — the expectations are off by 1 LSB on the −16383.5 boundary (01 c0 expected vs 00 c0 produced): looks like the conversion changed between rounding and truncation without the expectations moving. If round-half was the intent, _pcm16 should round before the int16 cast; if truncation is intended, update the fixtures.
• test_mono_remains_mono, test_resamples_24k…, test_duration_uses_frames_not_channel_count, test_buffered_output_is_downmixed… — ERRORED; from the tracebacks these hit the same removed/renamed surface (please check locally — they pass on your machine or not, which also tells us whether CI ever ran them).

Housekeeping nit: this file sits at tests/ while #106 established tests/tts/ for family tests — worth moving for consistency with the layout the repo just adopted.

[hsliuustc0106]

CLI additions reviewed — clean. The dispatch (model_type == moss_tts_nano peek), the validation matrix (--ref-audio required, --voice/--instruct rejected, --language auto-only, positive finite --temperature, integer --max-tokens), and the stub-based routing tests are all consistent with the VoxCPM2 and Qwen3 CLI seams. The RTF print parsing the written wav for duration is a nice touch (correct across sample rates).

[hsliuustc0106]

Re-review at 12e84ee (round 2).

Round-1 inlines: all four resolved correctly — close() dropped, finished-buffer stream interim (matching VoxCPM2, good call), ImportError hoisted (config error ≠ 400), duration bounds both ends. The CLI serving additions are clean too (inline posted): dispatch, validation matrix, and stub routing tests all consistent with the existing seams.

One blocker, and it's the battery itself: tests/tts/test_moss_nano.py is red at this head — 10 failures/errors, one root-cause class: the follow-up commits changed behavior without updating the tests that pinned the old behavior. Details inline (removed close()/stream-400 tests still present; _pcm16 expectations off by 1 LSB on the −16383.5 boundary across f32/f16/bf16; four more erroring on the same removed surface). Reproduced in a clean worktree at the frozen head, python -m unittest tests.tts.test_moss_nano tests.test_moss_nano_cli.

Verdict: still draft / not merge-ready — same overall shape as round 1 (the implementation direction is right; stage-gating on #73 holds), but the stale-test fix is required before this can go green, and the CI question from round 1 stands: these failures mean the suite hasn't run green on any machine since 9da492d. Suggested order: decide rounding-vs-truncation for _pcm16 (if round-half was intended, round before the int16 cast), update or delete the removed-surface tests, move test_moss_nano_cli.py under tests/tts/ per #106, and let CI actually run.

(cc @hsliuustc0106 for visibility — and thanks for the two thorough rounds on #98; the vacuous-decorator catch in particular was a real one.)

[hsliuustc0106]

Re-review at 818a6f5 (round 3).

Round-2 blocker: resolved. The stale-test class is gone — _pcm16 now pins the round-half result (±16384, i.e. rounding before the int16 cast — reasonable choice, and the boundary is now tested rather than accidental), the removed-surface tests were rewritten for the streaming API, test_moss_nano_cli.py moved under tests/tts/ (#106), and CI is green on this head (39 s). I re-ran everything locally in a clean worktree: CI-equivalent discover = 228 tests OK (64 skips), and the [tts]-gated battery in a fresh venv with mlx-audio 0.5.7 = 70 tests OK.

Stage 3 (AR streaming): delivered, and it's correct. I audited moss_nano_loop.py against the upstream 0.5.7 sdist directly:

• _sentence_frames is a faithful port of generate_audio_token_ids — same global-KV/frame-row feeding, same sampling order (so RNG consumption and seeded parity match), same per-sentence cache reset and repetition-penalty history, same n_vq padding. The cancel checks are pure additions at frame boundaries.
• iter_audio_frames mirrors upstream generate(): reference encoded once, same normalize/split calls, budget applied per text segment.
• Codec side: one streaming decoder per request spans all sentences — matching upstream's single whole-utterance decode; deriving the hop from decoder expansion ÷ channels instead of downsample_rate (input padding) is the right call, and every attribute validate_streaming_codec probes exists in 0.5.7.

I also ran the real-weight e2e on my machine (pulled both checkpoints via hf-mirror): all 5 pass — greedy + seeded token parity (EN & ZH) against upstream on real weights, real-codec partition parity vs whole decode at hop 3840, first audio strictly before generation finishes, buffered ≈ streamed within 4 LSB with no cross-request state leaks, and seed repeatability on the worker.

Stage 4 (bench): the harness works. Smoke-run: TTFA 0.81–0.87 s buffered → 0.047–0.073 s streamed, RTF ~0.30–0.36 in both modes, peak MLX memory lower when streaming (0.33 vs 0.50 GiB — incremental decode never materializes the full waveform), identical audio byte counts. That's the AR-streaming pitch validated on-silicon.

The concurrency layer is the part I looked hardest at, and it holds up. _PCMStream (single generation worker, maxsize-1 queue, cancel hook that unblocks a full put) × _AudioStream (ASGI adapter handling disconnect-before-first-next, disconnect-during-next, send failure, and task cancellation) — each race I tried to construct already has a deterministic test, including the queue-timeout-vs-final-put drain race. Lock release on cancel is verified at both the service and HTTP layers.

Two nits inline (anyio declaration; slow-client lock semantics worth a conscious sign-off) and one remaining stage-4 item: docs/speech.md has no Nano/streaming section yet — worth adding before this leaves draft, since that's where the speech endpoints are documented.

Verdict: approve as draft-stage code. Everything reviewable at this head is green and verified; what remains (baseline numbers write-up, docs entry, later compilation work) is the author's declared stage-4+/follow-up scope.

[hsliuustc0106]

P3: anyio is now imported at module level, but it is only a transitive dependency (via starlette) — pyproject.toml declares starlette>=0.37 only. It works today because starlette hard-requires anyio, but a direct import deserves a direct declaration. One-liner: add anyio>=3 to dependencies.

[hsliuustc0106]

P3 / sign-off requested: with the maxsize-1 queue, a client that stops reading without disconnecting parks the producer in the put-loop, which holds _lock (and the single worker) for as long as the client stalls — so batch-1 serialization now includes client pacing. This is a deliberate change from the round-2 interim, where slicing happened outside the lock and a slow client never blocked the next request. Bounded backpressure is the right default for true AR streaming (don't burn GPU on audio nobody is draining), and cancel-on-disconnect is tested at both layers — so I'm fine with it. Just flagging that a permanently-stalled TCP peer with no uvicorn write-timeout can wedge TTS serving; if that ever matters in practice, an idle-write timeout that trips cancel() would close the gap. No change required at this stage.

[hsliuustc0106]

Re-check (round 4) after the PR left draft state.

Head is unchanged since round 3 — still 818a6f5, the commit approved on 2026-10-09 (CI green there: lint + unit tests SUCCESS). No new commits or replies since, so the round-3 approval stands as-is: streaming parity vs mlx-audio 0.5.7, real-weight e2e, and the _pcm16/stale-test fixes all remain verified. No code re-review needed.

One item from the round-3 sign-off list is still open, and it's the only thing I'd ask for before merge:

docs/speech.md has no MOSS-TTS-Nano section. The file currently documents every other speech surface (VoxCPM2 config + quality gates, streaming semantics, voice cloning, one-shot CLI, performance), but this PR adds a second full TTS backend — new [tts] config, CLI subcommands, route behavior, and materially different streaming characteristics (TTFA ~0.05–0.07 s streamed vs ~0.85 s buffered, RTF ~0.3, lower peak MLX memory) — none of which is discoverable anywhere. Please add a ## MOSS-TTS-Nano section in this PR covering: checkpoint names (incl. the tokenizer repo), the config knobs, buffered vs streaming behavior and the maxsize-1 pacing semantics, and the measured bench numbers from round 3. Happy to review it as a docs-only follow-up commit in this same PR.

Carry-overs (both no-change-required, unchanged since round 3):

• P3: anyio is imported at module level in server.py but only transitively declared via starlette — anyio>=3 in pyproject.toml dependencies whenever convenient.
• Slow-client-stalls-_lock semantics: already signed off as deliberate backpressure; an idle-write timeout remains a possible future hardening, not needed now.

[hsliuustc0106]

Approving (round 4, maintainer call). Head is still 818a6f5 — identical to the round-3 approved commit, CI green, streaming parity and real-weight e2e verified there. The docs/speech.md ## MOSS-TTS-Nano section from my round-4 comment is downgraded to a non-blocking follow-up: please still add it (in this PR pre-merge if convenient, otherwise right after), since it's the only place users can discover the Nano config, streaming semantics, and bench numbers. Same for the anyio>=3 pyproject one-liner.