Skip to content

Add Translate calls guide - #878

Open
Manny-r31 wants to merge 2 commits into
mainfrom
manny/translate-calls-guide
Open

Manny-r31 wants to merge 2 commits into
mainfrom
manny/translate-calls-guide

Conversation

@Manny-r31

Copy link
Copy Markdown
Collaborator

Closes #769

Summary

  • New guide: Translate calls (/docs/platform/voice/translate-calls). A task-first capability guide covering SWML, Relay, and REST, modeled on the recent voice guides (call streaming, record calls) and structured to match the Transcribe calls guide in Add Transcribe Calls guide #829.
    • Capability matrix, then how translation works: each party's original audio is replaced by translated speech, remote-caller vs local-caller, and a table of which party speaks and hears which language.
    • First run via SWML: a small server answers, starts live_translate with a webhook and live_events, connects the caller, and prints each translated utterance.
    • Start translation on a live call via Relay and REST, read the three webhook events (told apart by channel_data.content_disposition), options (voices, tone filters, speech detection), AI summaries (at session end and on demand), and stopping translation.
    • Running example: a Spanish-speaking rider calls Bayview Taxi and talks to Ada, the English-speaking dispatcher.
  • Fixed the Server SDK live_translate reference pages.
    • The Python and TypeScript Relay pages documented {"action": "start", "source_language": …}, which the platform rejects. They now document {"start": {from_lang, to_lang, direction, …}}, {"summarize": {…}}, {"inject": {…}}, and stop.
    • The TypeScript Relay page now puts statusUrl in the options object.
    • The TypeScript REST example now wraps the action in { action: { start: … } }, matching the pinned @signalwire/sdk@2.0.5.
    • The Python REST page title and description now cover start, summarize, and stop.
  • Cross-links: a card on the voice overview, a link from the SWML live_translate reference, and the llms.txt bullet now points to the guide instead of the SDK reference.
  • No nav change (the voice folder is folder-driven) and no changelog entry (documentation work).

Notes for reviewers

  • Depends on Add Transcribe Calls guide #829. The guide links /docs/platform/voice/transcribe-calls. Merge Add Transcribe Calls guide #829 first. Both PRs add a card to the voice overview, so expect a small conflict there.
  • Needs a live test before merge. Nothing here was tested on a real call.
    • Which party speaks which language. The order of direction decides it: the first-listed side is recognized in to_lang and spoken to the other party in from_lang. This is not intuitive, and the language table, the troubleshooting bullet, and the SDK reference wording depend on it. The guide's config (from_lang: es-ES, to_lang: en-US, direction: [local-caller, remote-caller]) should give a Spanish-speaking caller and an English-speaking callee.
    • Translation started before connect carries over to the connected party. The SWML and Relay first runs rely on this.
    • Webhook payload examples are built from the event shapes, not captured from a call, and are labeled as shortened. Worth checking that primary_* fields carry the from_lang conversation.
    • Retry behavior for utterance events (connection failure or 5xx) follows the same webhook mechanism Add Transcribe Calls guide #829 verified for transcription.
  • inject is left out of the guide. The SDK reference pages list inject.message and inject.direction as required but make no claim about what direction means, because its meaning is unclear.
  • Not fixed here (follow-up for the SWML spec and reference):
    • from_voice / to_voice default: the reference says elevenlabs.josh; the platform default is an Azure multilingual voice.
    • vad_silence_ms Deepgram default: documented as 300; it is 700.
    • live_events sends final translated utterances, not partial results.
    • inject.direction is shown with a default of local-caller; it is required with no default.
    • The spec examples use Polly.Joanna. Voice prefixes are case-sensitive, so this falls back to the default voice; lowercase polly. works.
    • debug_level is documented but has no customer-visible effect.
    • The end-of-session conversation log is sent whenever a webhook is set, not only with ai_summary.
    • status_url on the REST and Relay command has no observed effect for translation. Events go to start.webhook.
  • Upstream SDK issues (not fixed here):
    • The TypeScript REST JSDoc for liveTranslate names source_lang / target_lang, which don't exist.
    • The Python SWML builder's typed stub for live_translate takes a positional config, but the runtime method is keyword-only (live_translate(action=…)).
    • TypeScript Relay liveTranslate("stop") fails strict typing. The guide uses { stop: {} }, which the platform also accepts.
    • TypeScript Relay returns {} on an error response instead of throwing.

Verification

  • All 39 code blocks were extracted and checked. JSON and YAML parse. bash -n is clean. Python passes py_compile and pyflakes. TypeScript passes tsc --strict against @signalwire/sdk@2.0.5.
  • The Python and TypeScript SWML builder chains were run; their output matches the JSON and YAML shown beside them.
  • The sample webhook servers were fed sample events and print them correctly.
  • Every internal link was checked against its target's slug:, and every #anchor by hand.
  • yarn fern-md-check: all 2831 MDX files valid. No yml changed, so fern-check was not needed.

Preview

  • /docs/platform/voice/translate-calls
  • /docs/platform/voice
  • /docs/swml/reference/calling/live-translate
  • /docs/server-sdks/reference/python/relay/call/live-translate
  • /docs/server-sdks/reference/typescript/relay/call/live-translate
  • /docs/server-sdks/reference/python/rest/calling/live-translate
  • /docs/server-sdks/reference/typescript/rest/calling/live-translate

🤖 Generated with Claude Code

Add a task-first guide for real-time call translation across SWML,
Relay, and REST, and fix the action shape on the Server SDK
live_translate reference pages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Translate calls with AI

2 participants