Repository navigation
feat(tts): add Phonon engine via ptts-openai-server - #62
Closed
shreyaskarnik wants to merge 2 commits into
Closed
shreyaskarnik wants to merge 2 commits into
shreyaskarnik wants to merge 2 commits into
Conversation
engines.phonon() talks to Gradium's OpenAI-compatible Phonon server (ghcr.io/gradium-ai/ptts-openai-server, default http://localhost:8880, overridable via baseUrl or PHONON_TTS_URL). The phonon-tts npm package is browser-only (WASM in a Web Worker + Cache API), so no npm dependency is added. - Requests response_format=pcm (s16le mono 24 kHz) and builds the Float32 WAV directly; no ffmpeg pass at 1x. - Never sends speed (server rejects != 1.0); non-1x speed goes through convertToWav atempo like ElevenLabs/Gemini. - Kokoro voice names (Argo's af_heart default) fall back to the engine's voice option with a one-time warning. - Splits input over the server's 4096-char limit. - Doctor comment/hint, README and tts-engines skill reference updated. Claude-Session: https://claude.ai/code/session_01A5noi8MogUjD2mdirpzXmY
Owner
Author
|
Closing: since #58, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
engines.phonon(), a TTS adapter for Phonon, Gradium's small on-device model (phonon-31m-en, Apache-2.0).Draft: every network call in the tests is mocked. This has not yet been run against a real Phonon server.
Why an HTTP adapter, not the npm package
The
phonon-ttsnpm package is a WebAssembly build that needs a browser (Web Workers, Cache API) and doesn't run in Node. The runtime repo also ships an OpenAI-compatible server, which this adapter talks to:So there is no new dependency, no
importOptional, and noargo doctorrow. LikemlxAudio, it is mentioned in doctor's "no engine installed" hint.Behaviour
baseUrldefaults toPHONON_TTS_URL, thenhttp://localhost:8880. The other options arevoice(defaultdefault) andtimeoutMs(default 180 s).pcm(s16le, mono, 24 kHz, already Argo's rate) and builds the Float32 WAV directly, with no ffmpeg pass at 1×.speedother than 1.0, sospeedis never sent. Scene speed goes through ffmpegatempoviaconvertToWav, as in the ElevenLabs and Gemini adapters.af_heart, which the server rejects. Names shaped like Kokoro's (xx_name) fall back to the engine'svoice, with one warning per name. Phonon voice names are capitalised (Zoey,Easton), so a real one can't be caught.docker runcommand. A 400 that mentions the voice points toGET /v1/audio/voices.Open questions for review
engines.openai({ baseUrl: 'http://localhost:8880/v1' })already reaches this server. Whatengines.phonon()adds: the Kokoro-name fallback, speed handled without a 400, splitting past 4096 characters, and server-specific errors. That's a modest gain, and the alternative is a README recipe.timeoutMsabove 300 s won't work as-is. This adapter uses Node's built-infetch, so the undiciheadersTimeoutfrom OpenAI engine: requests over ~300 s fail with openai >= 5, whatever the timeout #59 applies. Options: cap and document it, or accept afetchOptions/ dispatcher as feat(tts): pass fetchOptions through engines.openai() to lift the 300 s limit #61 did forengines.openai().af_heartfallback vs. throwing. Falling back is friendlier; throwing is stricter. I'd keep the fallback.PTTS_LANG), so a scene'slangis ignored. This is relevant to the multi-language design in Multi-language support #55. The fr/de/es/pt 40M models are CC BY-NC, i.e. non-commercial.0.5.0. It needs bumping as upstream releases.Checklist
src/tts/engines/phonon.ts,engines.phononfactory and exportstests/tts/phonon.test.ts: 15 tests (fetch mocked)skills/argo-guide/references/tts-engines.md, CLAUDE.md, doctor hinttsc --noEmitclean;tests/ttsandtests/optional-deps.test.tspass (153 tests), rebased on currentmain(includes feat(tts): OpenAI engine talks to any OpenAI-compatible server #58 and feat(tts): pass fetchOptions through engines.openai() to lift the 300 s limit #61)ptts-openai-server(argo tts generateon a demo, then listen to the clips)https://claude.ai/code/session_01A5noi8MogUjD2mdirpzXmY