Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion fern/llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ Feature pages are grouped by channel. SDK pages come first when available, follo
- [Transfer calls with SIP REFER](/docs/platform/voice/sip-refer): Transfer SIP calls with SWML, Relay, or REST, then handle the result or fall back to a bridged transfer.
- [Call legs](/docs/platform/voice/call-legs): What a call leg is, how connects, forwards, and fan-outs create legs, how to tell legs apart by their identifiers, and how each leg appears in logs, callbacks, and charges.
- [Transcribe calls](/docs/platform/voice/transcribe-calls): Transcribe a live call as it happens with SWML, Relay, or REST and receive each utterance on your webhook, then summarize the conversation with AI, stop transcribing, or transcribe the whole call after it ends.
- [Live translation](/docs/server-sdks/reference/python/relay/call/live-translate): Start or stop live translation on a call.
- [Translate calls](/docs/platform/voice/translate-calls): Translate a live call between two languages with SWML, Relay, or REST so each party hears the other in their own language, then read the translated conversation on your webhook, choose voices and tone filters, get AI summaries, and stop translating.
- [Stream call audio](/docs/platform/voice/call-streaming): Stream a live call to your own WebSocket server with SWML, Relay, or REST, then save or process the audio as it arrives, and play audio back into the call on a bidirectional stream.
- [Conferencing](/docs/server-sdks/reference/python/relay/call/join-conference): Join a call to an ad-hoc audio conference.
- [Secure payments](/docs/server-sdks/reference/python/relay/call/pay): Collect payment information on a call.
Expand Down
3 changes: 3 additions & 0 deletions fern/products/platform/pages/calling/voice/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,9 @@ SignalWire's advanced APIs and elastic cloud infrastructure make it a breeze to
<Card title="Call legs" icon="regular share-nodes" href="/docs/platform/voice/call-legs">
Why one call shows up as several records, and how to tell the legs apart.
</Card>
<Card title="Translate calls" icon="regular language" href="/docs/platform/voice/translate-calls">
Let a caller and your team talk in their own languages, with each side translated live.
</Card>
</CardGroup>

{/* Core links */}
Expand Down
1,055 changes: 1,055 additions & 0 deletions fern/products/platform/pages/calling/voice/translate-calls.mdx

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -1,38 +1,144 @@
---
title: "live_translate"
slug: reference/python/relay/call/live-translate
description: Start or stop live translation on a call.
description: Start, summarize, or stop live translation on a call.
max-toc-depth: 3
---

Start or stop live translation on the call. Translates spoken audio into
another language in real-time.
[swml-live-translate]: /docs/swml/reference/calling/live-translate
[translate-calls]: /docs/platform/voice/translate-calls

Start, summarize, or stop live translation on the call. Live translation recognizes what each
party says, translates it, and speaks the translation to the other party in place of the original
audio. The `action` dictionary takes the same shape as the [`live_translate`][swml-live-translate]
SWML method: a single `start`, `summarize`, `inject`, or `stop` key.

<Info>
See the [translate calls guide][translate-calls] for which language each party speaks and for the
webhook events.
</Info>

## **Parameters**

<ParamField path="action" type="dict" required={true} toc={true}>
Action configuration for the live translation.
The action to perform. Set exactly one of `start`, `summarize`, `inject`, or `stop`.
</ParamField>

<Indent>
<ParamField path="action.action" type="str" required={true} toc={true}>
The action to perform.

- `"start"` -- begin live translation
- `"stop"` -- end live translation
<ParamField path="action.start" type="dict" toc={true}>
Start a live translation session.
</ParamField>

<Indent>

<ParamField path="action.start.from_lang" type="str" required={true} toc={true}>
One of the two languages of the conversation, such as `"es-ES"`. `from_voice` speaks it.
</ParamField>

<ParamField path="action.start.to_lang" type="str" required={true} toc={true}>
The other language of the conversation, such as `"en-US"`. `to_voice` speaks it.
</ParamField>

<ParamField path="action.start.direction" type="list[str]" required={true} toc={true}>
The sides of the call to translate: `"remote-caller"`, `"local-caller"`, or both. The order
decides which side speaks which language.
</ParamField>

<ParamField path="action.start.webhook" type="str" toc={true}>
The URL that receives translation events as HTTP `POST` requests.
</ParamField>

<ParamField path="action.start.from_voice" type="str" default="azure.en-US-RyanMultilingualNeural" toc={true}>
The voice that speaks translations into `from_lang`, as `<engine>.<voice>` with a lowercase
engine name, such as `"azure.es-ES-ElviraNeural"`.
</ParamField>

<ParamField path="action.start.to_voice" type="str" default="azure.en-US-RyanMultilingualNeural" toc={true}>
The voice that speaks translations into `to_lang`, such as `"elevenlabs.josh"`.
</ParamField>

<ParamField path="action.start.filter_from" type="str" toc={true}>
A tone filter for speech in `from_lang`: `"polite"`, `"rude"`, `"professional"`,
`"shakespeare"`, `"gen-z"`, or a custom instruction prefixed with `"prompt:"`.
</ParamField>

<ParamField path="action.start.filter_to" type="str" toc={true}>
A tone filter for speech in `to_lang`, with the same values as `filter_from`.
</ParamField>

<ParamField path="action.start.live_events" type="bool" default="False" toc={true}>
Whether to send each translated utterance to the webhook as it is spoken.
</ParamField>

<ParamField path="action.start.ai_summary" type="bool" default="False" toc={true}>
Whether to add AI summaries to the event sent to the webhook when the session ends.
</ParamField>

<ParamField path="action.start.ai_summary_prompt" type="str" toc={true}>
The prompt that tells the AI how to summarize the conversation when `ai_summary` is enabled.
</ParamField>

<ParamField path="action.start.speech_engine" type="str" default="deepgram" toc={true}>
The speech recognition engine: `"deepgram"` or `"google"`.
</ParamField>

<ParamField path="action.source_language" type="str" toc={true}>
Source language code (e.g., `"en-US"`).
<ParamField path="action.start.speech_timeout" type="int" default="60000" toc={true}>
The timeout for speech recognition, in milliseconds.
</ParamField>

<ParamField path="action.target_language" type="str" toc={true}>
Target language code (e.g., `"es-ES"`).
<ParamField path="action.start.vad_silence_ms" type="int" toc={true}>
The silence, in milliseconds, that ends an utterance. Defaults to `700` with Deepgram and `500`
with Google.
</ParamField>

<ParamField path="action.start.vad_thresh" type="int" default="400" toc={true}>
Voice activity detection threshold, from `0` to `1800`.
</ParamField>

</Indent>

<ParamField path="action.summarize" type="dict" toc={true}>
Request an AI summary of the conversation so far, while translation continues. Pass an empty
dictionary to send the summary to the webhook from `start` with the default summary prompt.
</ParamField>

<Indent>

<ParamField path="action.summarize.webhook" type="str" toc={true}>
The URL that receives the summary.
</ParamField>

<ParamField path="action.summarize.prompt" type="str" toc={true}>
The prompt that tells the AI how to summarize the conversation.
</ParamField>

</Indent>

<ParamField path="action.inject" type="dict" toc={true}>
Insert a message into the session, to be translated and spoken on the call.
</ParamField>

<Indent>

<ParamField path="action.inject.message" type="str" required={true} toc={true}>
The message to translate and speak.
</ParamField>

<ParamField path="action.inject.direction" type="str" required={true} toc={true}>
The side of the call: `"remote-caller"` or `"local-caller"`.
</ParamField>

</Indent>

<ParamField path="action.stop" type="dict" toc={true}>
Stop the live translation session on both sides. Pass an empty dictionary.
</ParamField>

</Indent>

<ParamField path="status_url" type="str | None" default="None" toc={true}>
URL to receive translation results via webhook.
A status URL sent with the command. Translation events go to `action.start.webhook`.
</ParamField>

## **Returns**
Expand All @@ -41,7 +147,7 @@ another language in real-time.

## **Example**

```python {14}
```python {14-21}
from signalwire.relay import RelayClient

client = RelayClient(
Expand All @@ -54,14 +160,17 @@ client = RelayClient(
async def handle_call(call):
await call.answer()

# Start live translation from English to Spanish
# Start live translation between Spanish and English
await call.live_translate({
"action": "start",
"source_language": "en-US",
"target_language": "es-ES",
"start": {
"from_lang": "es-ES",
"to_lang": "en-US",
"direction": ["local-caller", "remote-caller"],
"webhook": "https://example.com/translation-events",
}
})

# Let the call proceed...
# Connect the caller to the other party, then let the call proceed...
await call.wait_for_ended()

client.run()
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
---
title: "live_translate"
slug: reference/python/rest/calling/live-translate
description: Start live translation on an active call via REST.
description: Start, summarize, or stop live translation on an active call via REST.
max-toc-depth: 3
---

Start live translation on an active call. Spoken audio is transcribed and
translated in real time to a target language.
Start, summarize, or stop live translation on an active call. Spoken audio is transcribed and
translated in real time, and each party hears the other in their own language. Pass the action
as `action`: a `start`, `summarize`, or `inject` object, or the string `"stop"`.

## **Request**

Expand Down
Loading
Loading