Skip to content

feat(tts): add Phonon engine via ptts-openai-server - #62

Closed
shreyaskarnik wants to merge 2 commits into
mainfrom
feat/phonon-tts
Closed

shreyaskarnik wants to merge 2 commits into
mainfrom
feat/phonon-tts

Conversation

@shreyaskarnik

Copy link
Copy Markdown
Owner

Adds engines.phonon(), a TTS adapter for Phonon, Gradium's small on-device model (phonon-31m-en, Apache-2.0).

Draft: every network call in the tests is mocked. This has not yet been run against a real Phonon server.

Why an HTTP adapter, not the npm package

The phonon-tts npm package is a WebAssembly build that needs a browser (Web Workers, Cache API) and doesn't run in Node. The runtime repo also ships an OpenAI-compatible server, which this adapter talks to:

docker run -p 127.0.0.1:8880:8880 -e PTTS_LANG=en ghcr.io/gradium-ai/ptts-openai-server:0.5.0
tts: { engine: engines.phonon({ voice: 'Zoey' }) }

So there is no new dependency, no importOptional, and no argo doctor row. Like mlxAudio, it is mentioned in doctor's "no engine installed" hint.

Behaviour

  • Server: baseUrl defaults to PHONON_TTS_URL, then http://localhost:8880. The other options are voice (default default) and timeoutMs (default 180 s).
  • Audio: it requests pcm (s16le, mono, 24 kHz, already Argo's rate) and builds the Float32 WAV directly, with no ffmpeg pass at 1×.
  • Speed: the server returns 400 for any speed other than 1.0, so speed is never sent. Scene speed goes through ffmpeg atempo via convertToWav, as in the ElevenLabs and Gemini adapters.
  • Long text: input over 4096 characters (the server's limit) is split at sentence boundaries and the pieces are joined.
  • Kokoro voice names: Argo's global default voice is Kokoro's af_heart, which the server rejects. Names shaped like Kokoro's (xx_name) fall back to the engine's voice, with one warning per name. Phonon voice names are capitalised (Zoey, Easton), so a real one can't be caught.
  • Errors: an unreachable server prints the docker run command. A 400 that mentions the voice points to GET /v1/audio/voices.

Open questions for review

  1. Is a dedicated engine worth it now that feat(tts): OpenAI engine talks to any OpenAI-compatible server #58 shipped? engines.openai({ baseUrl: 'http://localhost:8880/v1' }) already reaches this server. What engines.phonon() adds: the Kokoro-name fallback, speed handled without a 400, splitting past 4096 characters, and server-specific errors. That's a modest gain, and the alternative is a README recipe.
  2. timeoutMs above 300 s won't work as-is. This adapter uses Node's built-in fetch, so the undici headersTimeout from OpenAI engine: requests over ~300 s fail with openai >= 5, whatever the timeout #59 applies. Options: cap and document it, or accept a fetchOptions / dispatcher as feat(tts): pass fetchOptions through engines.openai() to lift the 300 s limit #61 did for engines.openai().
  3. af_heart fallback vs. throwing. Falling back is friendlier; throwing is stricter. I'd keep the fallback.
  4. Language is fixed per server (PTTS_LANG), so a scene's lang is ignored. This is relevant to the multi-language design in Multi-language support #55. The fr/de/es/pt 40M models are CC BY-NC, i.e. non-commercial.
  5. The docs pin image tag 0.5.0. It needs bumping as upstream releases.

Checklist

https://claude.ai/code/session_01A5noi8MogUjD2mdirpzXmY

engines.phonon() talks to Gradium's OpenAI-compatible Phonon server
(ghcr.io/gradium-ai/ptts-openai-server, default http://localhost:8880,
overridable via baseUrl or PHONON_TTS_URL). The phonon-tts npm package is
browser-only (WASM in a Web Worker + Cache API), so no npm dependency is
added.

- Requests response_format=pcm (s16le mono 24 kHz) and builds the Float32
  WAV directly; no ffmpeg pass at 1x.
- Never sends speed (server rejects != 1.0); non-1x speed goes through
  convertToWav atempo like ElevenLabs/Gemini.
- Kokoro voice names (Argo's af_heart default) fall back to the engine's
  voice option with a one-time warning.
- Splits input over the server's 4096-char limit.
- Doctor comment/hint, README and tts-engines skill reference updated.

Claude-Session: https://claude.ai/code/session_01A5noi8MogUjD2mdirpzXmY
@shreyaskarnik

Copy link
Copy Markdown
Owner Author

Closing: since #58, engines.openai({ baseUrl: 'http://localhost:8880/v1' }) already reaches Phonon's ptts-openai-server, so a dedicated engine adds too little to carry. The branch is kept for reference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant