Repository navigation
Add Transcribe Calls guide - #829
Conversation
Devon-White
left a comment
There was a problem hiding this comment.
I checked this against mod_openai main (translate_transcribe.c, webhook.c), mod_infrastructure master (relay_apis.c, relay.c, swml.c, swml_schema.c), prime-rails (the calling.transcript.* callback path), and the published signalwire-sdk 3.4.1 / 3.7.5 and @signalwire/sdk 2.0.5 / 3.7.1 packages. The platform facts are accurate. What needs to change is in the TypeScript samples, plus a few prose rules.
Before merge
- Coordinate with #908. It unpins Server SDK installs across the guides, and on the latest TypeScript SDK four samples here break.
calling.liveTranscribetakes the action positionally, so{ action }nests it and the platform rejects it.transcribeStoptakescontrol_idpositionally.addVerb("transcribe", …)throwsSchemaValidationErrorunless you pass{ skipValidation: true }. Each one has a suggestion inline. Python is unaffected. - Language names in prose. Lines 102, 383, 500, 934–936 and 1023 name Python, TypeScript, or per-language methods, which
mdx-capability-guidesrules out. Suggestions inline.
Should fix
- Line 885: both
stopforms work on every surface. There's no Relay/REST split. - Line 628 tip: from source, the end-of-session event can arrive unsigned when
live_eventsis off. Worth confirming before the tip says every request is signed. - Line 583: the REST reference example doesn't match the request, so this should be a cURL block.
Verified correct
- Action shapes and
startkeys.startneedslangand adirectionarray (parse_transcribe_json).remote-calleris the read leg andlocal-callerthe write leg.speech_engineisdeepgram(default) orgoogle(get_speech_engine). Sendingstartwhile a session runs stops it first (openai_transcribe_init→openai_transcribe_stop). - Summaries.
summarizefalls back to thestartwebhook and the default prompt, andai_summary_promptapplies only to the end-of-session summary. On failure, a mid-call summary sendsError creating summary, and the end-of-session event omitsconversation_summary. - Webhook models. All three match what mod_openai sends:
content_dispositionvalues,channel_datafields, and microsecond timestamps- every
utterancefield, withsilence_before_msandconfidenceomitted when zero - the top-level
confidence(nullwhen the engine reports none) languageinconversation_log, and theremote_caller_*/local_caller_*metrics
- Live webhook delivery. 10 s timeout. A connection error, timeout, or 5xx is retried, a 4xx isn't, and signed requests don't follow redirects. Basic auth comes from URL credentials.
transcribeandtranscribe_stop.status_urlis the only option, andtranscribe_resultissuccessorfailed. RESTcalling.transcriberequirescontrol_id. In the post-call callback,params.textis dropped when empty (.presence+.compactinRelay::TranscriptCallbackSerializer) andparams.idis present. The callback has a 5 s read timeout (Signalwire::REQUEST_TIMEOUT), and non-2xx responses are retried (raise Signalwire::RetryableError, Sidekiqretry: 2).- Relay reference fix. The old
{"action": "start", "language": …}example really is rejected:actiongets keys outsidestart,stop,summarize. - Links and snippets. Every link target and
#anchorresolves. Thecalling.live_transcribe,calling.transcribe, andcalling.transcribe.stoprequest examples exist, and so do the three new webhook operation IDs.
Couldn't verify: what a live call prints (no live call was placed, per your notes), including whether the TTS prompt and the far side of a connect come through as local-caller. The write-leg direction supports both.
|
@Devon-White Thanks! Addressed everything except the 3.x TypeScript changes:
|
Description
/docs/platform/voice/transcribe-calls). A task-first capability guide covering SWML, Relay, and REST. It walks through a first call that sends each utterance to a webhook, then covers reading the webhook events, options, AI summaries (when the session ends and on demand), stopping transcription, and transcribing the whole call after it ends withtranscribe, including thecalling.call.transcribestate events that also reachstatus_url. It also links the AI sidecar guide and call streaming.live_transcribe, built from the TypeSpec. There are three new Calling webhooks: utterance, summary, and conversation log. The event name isn't in the request body, so the pages and the guide show how to tell the events apart bychannel_data.content_disposition.live_transcribereference pages (Python and TypeScript). Their examples passed{"action": "start", "language": …}, which the platform rejects. Both SDKs forwardactionunchanged, so it must be an object with onestart,summarize, orstopkey.stopandsummarizealso work as bare strings.live_transcribereference pages now also liststart.hintsand the engine-dependentvad_silence_msdefault, matching the SWML reference.live_transcribereference:live_eventssends final utterances, not partial results; the page now says so. Added a Webhooks section.transcribereference, and theai_sidecarreferencellms.txtbullet now points to the guideNotes for reviewers
@signalwire/sdk@2.0.5, so they need updating when Install the latest Server SDK in guides instead of pinned versions #908 unpins installs. On 3.x,liveTranscribe(callId, action)andtranscribeStop(callId, controlId)take their arguments positionally, andaddVerb("transcribe", …)needs{ skipValidation: true }. Whichever of this PR and Install the latest Server SDK in guides instead of pinned versions #908 merges second applies Devon's inline suggestions (lines 571, 825, 905, 969, 1070) and unpins the six install lines. Python is unaffected.connect. The guide sayslocal-callerincludes the other party of aconnect. That follows from the audio direction.X-SignalWire-Signature) and send credentials in the URL (user:pass@host) as HTTP Basic auth. Retries differ: live events are retried on a connection error or a 5xx response, and post-callstatus_urlrequests on any non-2xx response. The guide's samples leave the webhook routes open to stay short, and a tip covers both protections.live_eventsoff, the end-of-session event can be sent without a signature (cloud-product#22323). The tip tells readers who verify signatures to keeplive_eventson. Remove that sentence once the fix ships.AICallInfoenvelope.fern/apis/signalwire-rest/openapi.yamlgrows by about 400 lines, all of it the three new webhooks.live_translatereference pages have the sameactionshape bug.Preview
/docs/platform/voice/transcribe-calls/docs/platform/voice/docs/platform/voice/record-calls/docs/swml/reference/calling/live-transcribe/docs/swml/reference/calling/transcribe/docs/swml/reference/calling/ai-sidecar/docs/server-sdks/reference/python/relay/call/live-transcribe/docs/server-sdks/reference/typescript/relay/call/live-transcribe/docs/apis/rest/webhooks/live-transcribe-utterance-webhook/docs/apis/rest/webhooks/live-transcribe-summary-webhook/docs/apis/rest/webhooks/live-transcribe-conversation-log-webhookType of Change
Related Issues
Closes #768.
Testing
Behavior verified against the platform and the API spec.
yarn build:specssucceeds after mergingmain.WebhookPayloadSnippetuses the operationIds, per main's component change.yarn fern-md-check: all MDX files valid.yarn fern-check: only the redirects check fails locally, because it needsFERN_TOKEN.signalwire-sdk==3.4.1and@signalwire/sdk@2.0.5:bash -n.py_compileandpyflakes.tsc --strict.status_url.fern-check. Every#anchorchecked by hand.Tests
Checklist