[cacheline999]

Tasks 1–2 of #68: a decoder-style ASR adapter and a buffered POST /v1/audio/transcriptions. No prefix cache or compile yet (that is task 5), and no streaming (task 4).

What's in it

• asr/ package
  • config.py: loader, default mlx-community/Qwen3-ASR-1.7B-4bit
  • capabilities.py: two-class matrix (decoder vs encoder-decoder) keyed on config.json model_type. An unserved family is rejected at boot with guidance, before any weights load (the tts/variants.py pattern)
  • service.py: validation + ASR lock. Request errors (decode, caps, language) raise before the lock is taken
• audio_io.decode_audio: lifted out of tts/generate.py so TTS reference clips and ASR uploads share one decode+resample path. decode_ref_audio keeps its base64 handling and its error messages
• POST /v1/audio/transcriptions: multipart file, language, prompt, response_format (json / text / verbose_json), temperature. srt/vtt and stream return a 400 for now
• serve --asr-model <repo>, combinable with --tts-model
• [asr] extra: mlx-audio (same pin) + python-multipart

Things worth a look

• model.generate instead of generate_transcription: the latter writes transcript.txt into the cwd as a side effect. Calling the model directly gives the same STTOutput without it.
• prompt → system_prompt: the static text head of the decoder prompt, i.e. the task-5 cache key. hotwords is an extension field that mlx-audio folds into the same head.
• language accepts an ISO-639-1 code or a name and is validated against the checkpoint's support_languages.
• CI: Starlette needs python-multipart for form parsing and CI only installs -e ., so the workflow now installs it too. Without it the route tests skip. Happy to move it into base deps instead if you'd rather.

Verification

• python -m unittest tests.test_asr: 20 tests, with the real-weights test running (checkpoint cached locally). ruff check --select E9,F . clean. test_audio_speech, test_clone_scaffold, test_tts_generate, test_variants still pass.
• Live check against a running serve --asr-model mlx-community/Qwen3-ASR-1.7B-4bit on an M-series Mac, with a clip made by macOS say (4.5 s): correct transcript, 0.40–0.47 s per request warm (1.5 s first request). Passing hotwords=Qwen and a prompt changed "Quen" to "Qwen". A bad language, srt, and a non-audio file each return a 400 with a message.

Round-trip gate (task 3)

scripts/asr_roundtrip.py synthesizes 6 fixed sentences per voice with the serving TTS checkpoint (Qwen3-TTS-12Hz-1.7B-CustomVoice-4bit), transcribes them with Qwen3-ASR-1.7B-4bit, and scores word-error-rate against the input text. It fails above --max-wer (default 0.15). Run on an M-series Mac with voices vivian and ryan, 12 utterances:

• mean WER 0.019, max 0.12, 10/12 exact, peak 5.40 GiB with both models resident, PASS
• an earlier run with a different sentence set scored 0.083; the two misses were "sea shells" → "seashells", a spelling variant rather than a recognition error, so I swapped that sentence out. TTS sampling is on (temperature 0.9), so single utterances vary run to run

Not done

• No latency table. The timings above are a single smoke clip, not the cool-machine interleaved A/B the repo asks for, so they are not a perf claim.
• Streaming deltas (task 4) and the prefix cache + compiled decode loop (task 5).

Refs #68.

[hsliuustc0106]

Both models-matrix rows (lines 57 and 64, unchanged in this diff) still read 🚧 planned — /v1/audio/transcriptions (#68) while this PR adds the endpoint to the API table and a working example. Worth flipping them in the same pass so the README is not internally inconsistent after merge: Qwen3-ASR decoder-style served (buffered; streaming = #68 task 4), whisper/parakeet/voxtral still planned.

Same catch-up applies to docs/speech.md: since #85 the per-endpoint parameter detail lives there (README is launch-only), but there is no ASR section yet — multipart fields, hotwords list semantics, ISO-code-or-name language, and the srt/vtt/stream 400s are all only discoverable from the code today. Fine as a fast-follow when task 4 lands if you prefer, but a 10-line section now would match how /v1/audio/speech is documented.

[hsliuustc0106]

Reviewer verification (head 63a2683)

Verified independently, not just read:

• mlx-audio 0.5.7 API surface — every claim in the PR body checks out against the pinned .venv: stt.utils.load_model; generate(audio, *, max_tokens, temperature, language, verbose, system_prompt, hotwords, …) (all kwargs keyword-only — the **kwargs call is the right shape); STTOutput fields incl. prompt_tokens/generation_tokens; language is returned as a list on one path (qwen3_asr.py:1516), so the scalar-flatten is load-bearing; support_languages (upstream's own typo'd name) matches the real checkpoint's config (model_type: "qwen3_asr", English language names identical to _ISO_TO_NAME — ISO codes will resolve, not 400); generate_transcription writes transcript.txt to cwd by default, so calling generate directly is the right call.
• audio_io.read resamples to the target rate (chunked-FIR path), so duration = size / 16000 is correct and the 8 kHz→16 kHz decode test is meaningful.
• Import hygiene — simulated CI's no-mlx-audio condition (sys.modules["mlx_audio"] = None): server + __main__ import and create_app() constructs fine; the new top-level asr.service import pulls only mlx.core/huggingface_hub (base deps). The extras discipline holds.
• Tests — tests.test_asr: 20/20 OK locally (1 skip = weight-gated real-model; the 1.5 GiB checkpoint is not cached on this machine, so RealModelTest was not independently re-run — the author's live serve check stands as the real-weights evidence). tests.test_audio_speech + tests.test_variants pass in the PR worktree (the decode_ref_audio → shared decode_audio refactor is error-message-identical). ruff check --select E9,F . clean.
• Merge vs new main — clean test-merge against 7ddd019 (feat(tts): VoxCPM2 serving (#71) + vendored compiled per-patch loop (#79) #76 VoxCPM2 + chore: PyPI publish groundwork — Apache-2.0, full package metadata, trusted-publishing workflow #90 PyPI landed under this PR; both also touch pyproject.toml, no conflicts).
• Starlette 1.7 details check out: the AssertionError catch matches formparsers.py:69, and file parts spool to disk unbounded (the 1 MB max_part_size only caps text fields), so the 25 MiB cap is the real limit.

Notes

• CI is action_required (approval gate) on both runs — the workflow change has never executed in CI. The equivalent condition is covered by the local no-mlx-audio run above; approving the run would close that gap.
• The python-multipart placement question from the PR body: CI-only install (as here) matches the extras discipline and is fine; moving it to base deps is also defensible (tiny, pure-Python) if form endpoints proliferate. Maintainer's call, no change requested.

One inline comment on README/docs catch-up. Nothing blocking — this is a clean, well-scoped tasks-1–3 slice; the two-class capability matrix and boot-time family rejection are exactly the tts/variants.py pattern transposed.

[hsliuustc0106]

[nit] The 25 MiB cap is enforced only after the whole upload is in RAM: await upload.read() here, then MAX_UPLOAD_BYTES is checked inside ASRService.transcribe (asr/service.py:64). A multi-GiB POST spikes memory before the 400 (Starlette spools the multipart body to disk, but this read materializes it). Cheap fix: check int(request.headers.get('content-length', 0)) before parsing, or read with a bound. Not blocking for a single-user local server.

[hsliuustc0106]

[nit] Docstring says "≈3 GB on the 4-bit defaults" but this PR's scripts/README.md entry says "~5.4 GiB peak" (and the PR body reports 5.40 GiB measured). One of the two is wrong — please align so the round-trip gate's footprint guidance matches what was measured.

[hsliuustc0106]

[nit] Comment says ISO-639-1, but the map includes yue (ISO 639-3) and fil (ISO 639-2/3). Only the comment is off — the codes themselves are right for Qwen3-ASR. Maybe "ISO-639-1/3 codes".

[hsliuustc0106]

Approve (with 3 nits, posted inline) — reviewed at 63a2683 per the delegated-review protocol.

Verified against the pinned mlx-audio 0.5.8 wheel (not installed in my env — claims checked from source, weights-gated claims not executed — evidence reviewed only):

• stt.utils.load_model exists with the expected signature; qwen3_asr config.model_type = "qwen3_asr" and support_languages present, so the boot-time capability gate and _resolve_language rest on real API.
• Model.generate (qwen3_asr.py:1227) accepts exactly the kwargs the service passes: max_tokens, temperature, language, verbose, system_prompt, hotwords.
• The generate_transcription side-effect claim is real: default output_path="transcript" writes transcript.txt into cwd (stt/generate.py:148) — calling model.generate directly is justified.
• hotwords → merge_hotwords folds into system_prompt (stt/utils.py:20) — matches the prompt-head/prefix-cache-key framing.
• audio_io.read(file, dtype, sample_rate, nchannels) signature matches; decode_ref_audio refactor preserves both error messages verbatim and the 24 kHz rate.
• starlette 1.7.0 raises AssertionError without python-multipart (formparsers.py:69) — the route's 500-with-guidance catch is correct for the pinned version.

Evidence bars (serving PR): met. asyncio.to_thread worker dispatch is exercised by the TestClient route tests, and the live serve smoke on real audio covers the load-adjacent path. Weight-gated tests follow the taxonomy (HF_HUB_OFFLINE, safetensors-presence gate, ReleaseAfterClass, one heavy model). Round-trip WER gate (task 3) is a nice addition — harness logic reviewed, WER numbers not re-executed here.

Process flag, not a code issue: CI has produced zero checks on this PR — it's a fork PR that modifies .github/workflows/ci.yml, so the workflow needs a maintainer "Approve and run" before it can go green. Please get a green run before merging, since the python-multipart CI change itself is what's unverified.

Nits (non-blocking, posted inline): cap-after-read in server.py:401; ≈3 GB vs ~5.4 GiB docstring/README mismatch in asr_roundtrip.py:11; ISO-639-1 comment covers 639-2/3 codes in asr/service.py:31.