Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,7 +309,6 @@ const client = createGenerationClient({
- `z-image-turbo`
- `qwen-image-edit`
- `qwen-audio-3.1-tts-flash`
- `qwen-audio-3.1-tts-plus`
- `gemini-3.1-flash-image-preview`
- `kling-text-to-video`
- `kling-image-to-video`
Expand Down Expand Up @@ -402,18 +401,17 @@ await client.generate({

## Text to speech

`qwen-tts`, `qwen-audio-3.0-tts-plus` and `qwen-audio-3.0-tts-flash` are removed in this release. Migrate to `qwen-audio-3.1-tts-flash`, or to `qwen-audio-3.1-tts-plus`, which this release adds; both need input of at least 15 Unicode code points.
`qwen-tts`, `qwen-audio-3.0-tts-plus` and `qwen-audio-3.0-tts-flash` are removed in 0.2.0, and `qwen-audio-3.1-tts-plus`, declared in 0.2.0, is withdrawn in 0.2.1. Migrate to `qwen-audio-3.1-tts-flash`; its input must have at least 15 Unicode code points.

Each TTS request accepts exactly one non-empty text block and returns one URL audio block. Choose the model from the requested voice mode and priority before constructing the request:

| Requirement | Model choice (default tier) |
| Requirement | Model choice |
| --- | --- |
| Create a voice from a text-only description, without reference audio | `qwen-audio-3.1-tts-flash` with a `voice_prompt` |
| Clone a voice from one reference recording | `qwen-audio-3.1-tts-flash`; pass one HTTP(S) reference URL and no `voice_prompt` |
| Use a default voice, including a delegated choice expressed only as any, random, suitable, or natural | `qwen-audio-3.1-tts-flash` with a short generic `voice_prompt`, such as a natural, neutral narrator |

- Qwen: `voice_prompt` design OR one-reference clone via `qwen-audio-3.1-tts-flash` (default) or `qwen-audio-3.1-tts-plus`; both require at least 15 Unicode code points.
- Tier: the table names `qwen-audio-3.1-tts-flash`, the default. In any row, use `qwen-audio-3.1-tts-plus` instead only when the user or an external policy selects it. Alibaba describes Flash as its low-latency tier and Plus as its quality tier for audiobooks and dubbing; the SDK has not measured either.
- Qwen: `voice_prompt` design OR one-reference clone via `qwen-audio-3.1-tts-flash`; requires at least 15 Unicode code points.
- Conflict: reference + redesign requires user choice before generation.
- Dependency: clone prior generated audio.
- Short text: input under 15 Unicode code points is rejected; ask the user to lengthen it.
Expand Down
4 changes: 2 additions & 2 deletions models/qwen-audio-3.1-tts-flash.yaml
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
schema: neta.generation.model.v1
model: qwen-audio-3.1-tts-flash
title: Qwen Audio 3.1 TTS Flash
description: "Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Tier: default;
Alibaba's low-latency tier. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio."
description: "Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Conflict: ask user;
never combine/reinterpret. Dependency: clone prior generated audio."
adapter:
type: openai.audioSpeech
content:
Expand Down
46 changes: 0 additions & 46 deletions models/qwen-audio-3.1-tts-plus.yaml

This file was deleted.

4 changes: 2 additions & 2 deletions src/adapters/audio-speech.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import type {
} from "../types.js";

const REQUEST_TIMEOUT_MS = 210_000;
const QWEN_MODELS: ReadonlySet<string> = new Set(["qwen-audio-3.1-tts-flash", "qwen-audio-3.1-tts-plus"]);
const QWEN_MODEL = "qwen-audio-3.1-tts-flash";

type TextBlock = Extract<GenerationContentBlock, { type: "text" }>;
type AudioBlock = Extract<GenerationContentBlock, { type: "audio" }>;
Expand Down Expand Up @@ -97,7 +97,7 @@ function validateQwen(input: ResolvedGenerationRequest, text: TextBlock, audio:

function validateAudioSpeechRequest(input: ResolvedGenerationRequest): void {
const { text, audio } = validateCommonContent(input);
if (QWEN_MODELS.has(input.declaration.model)) {
if (input.declaration.model === QWEN_MODEL) {
validateQwen(input, text, audio);
return;
}
Expand Down
32 changes: 11 additions & 21 deletions src/builtins.ts
Original file line number Diff line number Diff line change
Expand Up @@ -708,14 +708,13 @@ function geminiImageModel(
};
}

const QWEN_TTS_TEXT = "这是一段长度足够并且表达清晰自然的语音合成测试文本。";

function qwenAudio31TtsModel(model: string, title: string, tier: string): GenerationModelDeclaration {
return {
const audioSpeechModels = [
{
schema: MODEL_SCHEMA,
model,
title,
description: `Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Tier: ${tier}. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio.`,
model: "qwen-audio-3.1-tts-flash",
title: "Qwen Audio 3.1 TTS Flash",
description:
"Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio.",
adapter: { type: "openai.audioSpeech" },
content: {
input: [
Expand Down Expand Up @@ -748,32 +747,23 @@ function qwenAudio31TtsModel(model: string, title: string, tier: string): Genera
{
title: "Voice design",
request: {
model,
content: [{ type: "text", text: QWEN_TTS_TEXT }],
model: "qwen-audio-3.1-tts-flash",
content: [{ type: "text", text: "这是一段长度足够并且表达清晰自然的语音合成测试文本。" }],
meta: { voice_prompt: "一位沉稳干练的男性播音员声音,吐字清晰有力" },
},
},
{
title: "Voice clone",
request: {
model,
model: "qwen-audio-3.1-tts-flash",
content: [
{ type: "text", text: QWEN_TTS_TEXT },
{ type: "text", text: "这是一段长度足够并且表达清晰自然的语音合成测试文本。" },
{ type: "audio", source: { type: "url", url: "https://example.com/reference.mp3" } },
],
},
},
],
};
}

const audioSpeechModels = [
qwenAudio31TtsModel("qwen-audio-3.1-tts-flash", "Qwen Audio 3.1 TTS Flash", "default; Alibaba's low-latency tier"),
qwenAudio31TtsModel(
"qwen-audio-3.1-tts-plus",
"Qwen Audio 3.1 TTS Plus",
"Alibaba's quality tier (audiobooks, dubbing); only when user or external policy selects it",
),
},
] satisfies GenerationModelDeclaration[];

const meshyTaskParameters = {
Expand Down
51 changes: 17 additions & 34 deletions test/adapters/audio-speech.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,18 +68,15 @@ function requestBody(call: { init: RequestInit } | undefined): Record<string, un
}

describe("openai.audioSpeech adapter requests", () => {
it.each([
"qwen-audio-3.1-tts-flash",
"qwen-audio-3.1-tts-plus",
])("sends the fixed wire contract for %s and preserves Qwen input and voice prompt", async (model) => {
it("sends the fixed wire contract and preserves Qwen input and voice prompt", async () => {
const { client, calls } = recordingClient(() =>
routerSuccess({}, { headers: { "x-request-id": "request-primary", "x-oneapi-request-id": "request-fallback" } }),
);
const input = " 原样保留的朗读文本内容,不应被修改。\n";
const voicePrompt = " 沉稳清晰的声音。\n";

const output = await client.generate({
model,
model: "qwen-audio-3.1-tts-flash",
content: [{ type: "text", text: input }],
meta: { voice_prompt: voicePrompt },
});
Expand All @@ -91,7 +88,7 @@ describe("openai.audioSpeech adapter requests", () => {
new Headers({ Authorization: "Bearer secret-key", "Content-Type": "application/json" }),
);
expect(requestBody(calls[0])).toEqual({
model,
model: "qwen-audio-3.1-tts-flash",
input,
metadata: { voice_prompt: voicePrompt },
});
Expand Down Expand Up @@ -223,16 +220,6 @@ describe("openai.audioSpeech adapter validation", () => {
).toThrow("supports at most one reference audio");
});

it("accepts the qwen-audio-3.1-tts-plus declaration and a valid request", () => {
const declaration = getBuiltinGenerationModel("qwen-audio-3.1-tts-plus");
if (!declaration) throw new Error("qwen-audio-3.1-tts-plus declaration is unavailable");
const client = createGenerationClient({ models: [declaration], includeBuiltinModels: false, apiKey: "key" });

expect(() =>
client.validate({ model: "qwen-audio-3.1-tts-plus", content: [{ type: "text", text: QWEN_TEXT }, audio()] }),
).not.toThrow();
});

it("rejects a higgs-tts declaration at the audio speech adapter", () => {
const declaration = {
schema: "neta.generation.model.v1",
Expand All @@ -252,28 +239,24 @@ describe("openai.audioSpeech adapter validation", () => {
);
});

const QWEN_MODELS = ["qwen-audio-3.1-tts-flash", "qwen-audio-3.1-tts-plus"];

it.each(
QWEN_MODELS.flatMap((model) => [
{ model, text: "a".repeat(14) },
{ model, text: "😀".repeat(14) },
]),
)("rejects $model input below 15 Unicode code points ($text)", ({ model, text }) => {
it.each([
{ text: "a".repeat(14) },
{ text: "😀".repeat(14) },
])("rejects qwen-audio-3.1-tts-flash input below 15 Unicode code points ($text)", ({ text }) => {
const client = createGenerationClient({ apiKey: "key" });
expect(() => client.validate({ model, content: [{ type: "text", text }, audio()] })).toThrow(
"requires input of at least 15 Unicode code points",
);
expect(() =>
client.validate({ model: "qwen-audio-3.1-tts-flash", content: [{ type: "text", text }, audio()] }),
).toThrow("requires input of at least 15 Unicode code points");
});

it.each(
QWEN_MODELS.flatMap((model) => [
{ model, text: "a".repeat(15) },
{ model, text: ` ${"😀".repeat(15)} ` },
]),
)("accepts the $model input boundary ($text)", ({ model, text }) => {
it.each([
{ text: "a".repeat(15) },
{ text: ` ${"😀".repeat(15)} ` },
])("accepts the qwen-audio-3.1-tts-flash input boundary ($text)", ({ text }) => {
const client = createGenerationClient({ apiKey: "key" });
expect(() => client.validate({ model, content: [{ type: "text", text }, audio()] })).not.toThrow();
expect(() =>
client.validate({ model: "qwen-audio-3.1-tts-flash", content: [{ type: "text", text }, audio()] }),
).not.toThrow();
});

it.each<{ label: string; request: GenerateRequest }>([
Expand Down
22 changes: 7 additions & 15 deletions test/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,6 @@ describe("config", () => {
expect(byCategory.decision.sort()).toEqual([...expected.decision]);
for (const model of [
"qwen-audio-3.1-tts-flash",
"qwen-audio-3.1-tts-plus",
"suno_cover_chirp_v5",
"suno_image_to_song_chirp_v5",
"suno_infill_chirp_v5",
Expand Down Expand Up @@ -283,7 +282,7 @@ describe("config", () => {

it("publishes agent-discoverable audio speech declarations", () => {
const client = createGenerationClient();
const qwenModels = ["qwen-audio-3.1-tts-flash", "qwen-audio-3.1-tts-plus"];
const qwenModels = ["qwen-audio-3.1-tts-flash"];

for (const model of qwenModels) {
const declaration = client.getModel(model);
Expand All @@ -303,17 +302,13 @@ describe("config", () => {
expect(JSON.parse(client.stringifyModelConfig(model, { format: "json" }))).toEqual(declaration);
}

expect(client.getModel("qwen-audio-3.1-tts-flash")?.description).toBe(
"Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Tier: default; Alibaba's low-latency tier. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio.",
const flash = client.getModel("qwen-audio-3.1-tts-flash");
expect(flash?.description).toBe(
"Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio.",
);
expect(client.getModel("qwen-audio-3.1-tts-plus")?.description).toBe(
"Modes: voice_prompt design OR one-reference clone. Text: >=15 Unicode code points. Tier: Alibaba's quality tier (audiobooks, dubbing); only when user or external policy selects it. Conflict: ask user; never combine/reinterpret. Dependency: clone prior generated audio.",
expect(flash?.content.input.find((input) => input.type === "text")?.description).toContain(
"at least 15 Unicode code points",
);
for (const model of qwenModels) {
expect(client.getModel(model)?.content.input.find((input) => input.type === "text")?.description).toContain(
"at least 15 Unicode code points",
);
}

expect(client.getModel("higgs-tts")).toBeNull();
});
Expand All @@ -329,9 +324,6 @@ describe("config", () => {
expect(readme).toContain("Conflict: reference + redesign requires user choice before generation");
expect(readme).toContain("Dependency: clone prior generated audio");
expect(readme).toContain("Short text: input under 15 Unicode code points is rejected");
expect(readme).toContain(
"In any row, use `qwen-audio-3.1-tts-plus` instead only when the user or an external policy selects it",
);
expect(readme).not.toMatch(/quality prioritized over latency|latency prioritized over maximum quality/);
});

Expand Down Expand Up @@ -568,7 +560,7 @@ describe("config", () => {

it("does not publish the retired Qwen preview field", async () => {
const client = createGenerationClient({ apiKey: "test" });
for (const model of ["qwen-audio-3.1-tts-flash", "qwen-audio-3.1-tts-plus"]) {
for (const model of ["qwen-audio-3.1-tts-flash"]) {
expect(client.stringifyModelConfig(model)).not.toContain("preview_text");
}
expect(await readFile(join(process.cwd(), "README.md"), "utf8")).not.toContain("preview_text");
Expand Down
8 changes: 0 additions & 8 deletions test/live/audio-speech-live.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,14 +52,6 @@ liveDescribe("audio speech live router smoke", () => {
meta: { voice_prompt: "一位沉稳干练的男性播音员声音,吐字清晰有力" },
},
},
{
name: "qwen plus voice design",
request: {
model: "qwen-audio-3.1-tts-plus",
content: [text(`这是高质量音色设计端到端测试,运行编号${runId}。`)],
meta: { voice_prompt: "一位沉稳干练的男性播音员声音,吐字清晰有力" },
},
},
{
name: "qwen voice clone",
request: {
Expand Down