From 014fbc1c5837b97d4d8ebe1eed149c0900831de2 Mon Sep 17 00:00:00 2001 From: localai-org-maint-bot <306269227+localai-org-maint-bot@users.noreply.github.com> Date: Mon, 24 Aug 2026 03:02:49 +0000 Subject: [PATCH] docs(IndexTTS): describe reference conditioning The model guide and server reference still said that IndexTTS ignored its required reference clip. The implementation now feeds the clip to both conditioning stages, and the feature table already records that behavior. Keep voice-cloning parity pending because vLLM-Omni is not pinned as an oracle. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-5 [Codex] --- README.md | 3 +++ docs/models/indextts-2-5.md | 19 ++++++++++++------- docs/reference/server.md | 7 ++++--- 3 files changed, 19 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 2e3cbbb06..1d513f2f4 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,9 @@ - **2026-08** **Hybrid CPU/GPU expert placement reaches four architecture families.** Qwen3-MoE, Qwen3.5/3.6, Nemotron-H, and DeepSeek-V2 can run routed experts on the CPU while the rest of the model stays on the selected accelerator. The end-to-end token and speed gates are still pending. +- **2026-08** **IndexTTS 2.5 now uses its reference audio.** The clip conditions both the talker + and S2Mel stages. Repeated runs with one clip are bit-identical, while different clips change + the output. Voice-cloning parity with vLLM-Omni remains unmeasured. - **2026-08** **MiniMax-Music3 generates music through the public API.** Every pipeline stage is implemented and gated. The server exposes it through `POST /v1/audio/speech`; no reference speed number is available yet. diff --git a/docs/models/indextts-2-5.md b/docs/models/indextts-2-5.md index 2bc6444e5..3bcae8a47 100644 --- a/docs/models/indextts-2-5.md +++ b/docs/models/indextts-2-5.md @@ -56,16 +56,21 @@ curl http://localhost:8000/v1/audio/speech \ ## Current limitations -The engine requires a reference clip but does not use the reference clip for -conditioning. The clip does not select the output voice. Reference-audio -conditioning remains incomplete. +The reference clip conditions both halves of the model. The engine projects a +CAMPPlus speaker vector into the talker's first conditioning row and passes the +same vector to the S2Mel front end as its style input. The clip must use a 16 +kHz sample rate because the engine has no resampler. + +This wiring does not establish voice-cloning parity. Repeated runs with one +clip are bit-identical, and different clips change the output, but vLLM-Omni is +not pinned as an oracle. The inferred emotion path is also not implemented. The vLLM-Omni quality and parity comparison is pending. Current structural -gates show that the pipeline renders audio. They do not establish output -quality or parity with the oracle. +gates show that the pipeline renders audio and that the reference path affects +the result. They do not establish output quality or parity with the oracle. -The inferred emotion path is not implemented. The request can provide text, -but the server exposes no named voice or speaking-rate control. +The request can provide text, but the server exposes no named voice or +speaking-rate control. See the [server reference](../reference/server.md) for the shared endpoint contract. See the diff --git a/docs/reference/server.md b/docs/reference/server.md index 99385ceba..8669bdc40 100644 --- a/docs/reference/server.md +++ b/docs/reference/server.md @@ -69,9 +69,10 @@ returns a 44.1 kHz stereo WAV. See the IndexTTS-2.5 loads from `--speech-model` and returns a 22.05 kHz mono WAV. Each request must include `reference_audio` as a data URL for a 16-bit PCM mono WAV. -The current engine does not use the reference clip for conditioning, so the -clip does not select the output voice. The vLLM-Omni quality comparison is also -pending. See the +The clip must have a 16 kHz sample rate. Its CAMPPlus speaker vector conditions +the talker and S2Mel stages. Different clips change the output, but this does +not establish voice-cloning parity. The inferred emotion path is not +implemented, and the vLLM-Omni quality comparison is pending. See the [owning model specification](../../.agents/specs/indextts-2-5.md) for the owned limitations and verification evidence.