Skip to content

Multi-language support #55

Open
Open

Description

@lordrip

Hi, I had an idea yesterday about combining playwright with a voice synthesizer to record the same script in multiple languages, I started the implementation and after having a working POC, I thought, "what if someone already had the same idea and implemented it already", funny enough, I found this project and it seems it already does what I was looking for.

The main difference with this project vs my POC as far as I can tell, is the multi-language perspective. Would you be open to add this functionality to the project? If this is something that could be interesting for the project, I think I could contribute it with some guidance.

Thanks in advance 🙏

Activity

  1. shreyaskarnik commented on Oct 27, 2026

    @shreyaskarnik
    Owner

    Thanks for opening this, and for #58! Yes, I'd welcome multi-language support, and I'm happy to help guide it.

    Some context on what already exists, so you can reuse it:

    • Per-scene lang is already in the clip cache key (src/tts/cache.ts) and is used as the Whisper language hint.
    • video.contextOptions.locale already reaches Playwright, so the recorded app can switch language.
    • export.variants (src/pipeline.ts, the "Viewport-native variants" loop) already does TTS → re-record → export under another name, with each run in its own .argo/<demo>-<name>/ folder. Multi-language fits most naturally as a sibling of that loop.

    One constraint that shapes the design: a translation needs its own recording, not just new audio. Translated narration often runs 15–25% longer, and narration.durationFor() sets the script's waits from clip length, so laying French audio over the English recording would drift.

    A rough shape I'd suggest (open to your POC's approach):

    1. Translations: demos/<demo>.scenes.<lang>.json sidecars with the same scene names, translated text and overlay copy. argo validate checks that scene names match the base manifest.
    2. Config: a locales: [{ code: 'fr', voice?, engine?, contextOptions?: { locale: 'fr-FR' } }] list, with per-language voice/engine overrides.
    3. Pipeline: per language, TTS → record → align → export to videos/<demo>.<lang>.mp4. Expose the language to the demo script as an ARGO_LOCALE env var, for apps without locale-based routing.
    4. CLI: argo pipeline <demo> --locale fr plus a way to build every language. Subtitles and chapters come out per language for free.

    Things to keep in mind:

    • Kokoro (the default engine) is effectively English-only. Multi-language demos will lean on OpenAI, ElevenLabs, Gemini, Sarvam, or an OpenAI-compatible server via your baseUrl work in feat(tts): OpenAI engine talks to any OpenAI-compatible server #58.
    • Every export path has to handle locales: the pipeline, argo export, preview export, and variants (see CLAUDE.md). README and skills/argo-guide need updates too.

    Could you post a short design sketch here first (manifest layout, config shape, CLI) before opening a PR? Happy to iterate on it with you.

  2. lordrip commented on Oct 27, 2026

    @lordrip
    ContributorAuthor

    Hi @shreyaskarnik, thanks for the reply, absolutely, that's my goal, first get the design right and agreed, then implement.

    a translation needs its own recording, not just new audio.

    Yes, I agree, we would need to record everything, we're just gonna reuse the scene steps.

    I'll draft something and post it in this very same issue. Thanks once again.

  3. shreyaskarnik commented on Oct 27, 2026

    @shreyaskarnik
    Owner

    Sounds good, looking forward to the sketch! 🙌

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions