Repository navigation
Document supervisor call monitoring (listen in, whisper, barge) - #658
hey-august wants to merge 10 commits into
Conversation
…visor-call-monitoring # Conflicts: # fern/apis/signalwire-rest/openapi.yaml # fern/products/server-sdks/pages/reference/python/agents/function-result/join-conference.mdx # fern/products/server-sdks/pages/reference/python/relay/call/join-conference.mdx # fern/products/server-sdks/pages/reference/python/rest/compat/conferences/update-participant.mdx # fern/products/server-sdks/pages/reference/typescript/rest/compat/conferences/update-participant.mdx # fern/products/swml/pages/reference/methods/calling/join_conference.mdx # specs/swml/calling/Methods/join_conference/main.tsp # specs/swml/calling/tsp-output/@typespec/json-schema/SWMLObject.json
Devon-White
left a comment
Devon-White
left a comment
There was a problem hiding this comment.
Draft, and it needs a rebase: git merge-tree against current main gives a content conflict in fern/products/platform/pages/calling/voice/overview.mdx, because main has added cards to Popular guides since this branch was cut. specs/swml/calling/Methods/join_conference/main.tsp and its tsp-output have also moved on main (#898 and the join_conference fixes). After rebasing, rerun yarn build:specs rather than resolving the generated files by hand. The join_conference_result mismatch listed under "Not in this PR" is already fixed on main (joined/failed).
I checked every behavioral claim against mod_infrastructure (new_conference_controller.c, swml.c, swml_schema.c, relay.c, relay_apis.c), mod_laml, prime-rails, the stack mod_conference mixer, and the latest Server SDKs (signalwire-sdk 3.7.5, @signalwire/sdk 3.7.1).
Must change before merge
- Every supervisor join beeps the whole conference.
beepdefaults to"true", and the enter and exit tones play to every participant, customer included. All three samples needbeep: "false", as a string. Details inline. - Mid-call switching may not work for SWML-joined conferences, and the SDK sample is entirely that path. For Relay conferences, Rails sends the participant row's own
idas the member UUID, while the conference controller looks members up by call segment ID. This needs a live test before the "whether the conference was joined from SWML or cXML" claim ships. Details inline. - The TypeScript sample doesn't compile on the current SDK. It pins
@signalwire/sdk@2.0.5, and its version<Note>documents the gap rather than fixing it. Unpin, usedial(from, to, { swml }), and delete the note. - Coaching needs a started conference, not just the agent present (
not_ready_for_coach). Add this to the whisper section and to Limits. - "Keep the console in sync" misses
modify. Coaching changes arrive only asparticipant-modifywhenmodifyis subscribed, and cXML has no event for them at all. - Page shape. This is a multi-surface capability guide, but it doesn't follow
mdx-capability-guides. It has no## Choose an approach/## Prepare(placeholder table) /## How … worksskeleton. The joins appear only as raw YAML: there's no Server SDK SWML builder (SWMLBuilder/SwmlBuilder, served bySWMLService) first, no JSON, and no WebSocket (Relay) path, even though Relayjoin_conferenceacceptsmuted,beep, andcoach(relay_apis.ccall_join_conference;Call.join_conference/joinConferencein both SDKs). Bringing in the supervisor re-teaches call placement instead of linking/docs/platform/voice/make-and-receive-calls. A shape that fits the content:## How supervisor monitoring works(the mode table), then one##per approach (SWML, then WebSocket (Relay)), each with join-to-listen, join-to-whisper, and switch-modes tasks, and the REST switch shared as one section. - Ships-with-the-page items. It has no
fern/llms.txtbullet for the new page (the root index is hand-maintained). The overview card is there; keep it when you resolve the conflict.
Verified correct
- Coach direction (the core fix).
coachnames the coached member, and the joining call becomes the coach.conference_node_joinlooks upmember_to_coachby thecoachvalue and callsconference_member_coach(&member, member_to_coach)inmod_conference. The mixer (mod_conference.c) writes a coach's frame only to its target's coach frame and skips the main mix. The rewrites on the four SDK pages, the SWML reference and.tsp, and the cXML noun are right. - A muted coach is heard by nobody. A muted member never sets
MFLAG_HAS_AUDIO, and the mixer skips members without it. - Several coaches can coach one agent. The target's coach frame is the sum of all coaches.
- A participant can't coach itself. Checked in both join and update ("Cannot coach yourself").
- The coach leaves with the agent.
conference_test_bridged.c: "hang up the coached member - the coach will be removed automatically". The coach's dial status staysansweredand the call continues. - Join failures. SWML
coachmust be a UUID (ks_json_check_string_is_uuid). A failed coach join givesDialCallStatus=busyin cXML, and in SWMLjoin_conference_result=failedwith the script continuing. - REST participant update.
Muted,Coaching, andCallSidToCoachexist (API::Laml::ConferenceParticipant::Contracts::Update). Error 16002 covers both a missingCallSidToCoachand a coached call that isn't in the conference. The participant record exposescoachingandcall_sid_to_coach. There's no REST create-participant route (only: [:index, :show, :destroy]plusupdate). The six transitions in the table are consistent with the controller's mute and coach handling. - Same project. The conference registrar key includes the project.
- Python sample.
client.calling.dial(from_=…, to=…, swml=…)matches 3.7.5.py_compileandpyflakesare clean. Neither SDK has a Compatibility REST client. - Links. All internal links resolve against current slugs or operation IDs.
Couldn't verify
- Whether the Relay-conference participant update reaches the member (item 2). This needs a live call.
| ```yaml title="SWML" | ||
| version: 1.0.0 | ||
| sections: | ||
| main: | ||
| - join_conference: | ||
| name: support-floor | ||
| muted: true | ||
| ``` |
There was a problem hiding this comment.
The listen-in join beeps the whole conference. beep defaults to "true", and conference_node_add_member (new_conference_controller.c) turns that into beep_on_join/beep_on_leave. On add-member and on leave the controller calls conference_node_play(node, …ENTER_SOUND/EXIT_SOUND), which plays to every participant, the customer included. So a supervisor following this sample is announced by a tone on the way in and out, which defeats silent monitoring, and line 64's "leaves no marker of its own" isn't true as written.
Every supervisor join on the page needs beep: "false", and it has to be the string. The SWML validator (swml_schema.c, SWML_CHECK_OPTIONAL_STRING_MATCHES_PARAM(beep, …, "true,false,onEnter,onExit")) and the Relay check (relay_apis.c call_join_conference) both reject a YAML boolean false.
| ```yaml title="SWML" | |
| version: 1.0.0 | |
| sections: | |
| main: | |
| - join_conference: | |
| name: support-floor | |
| muted: true | |
| ``` | |
| ```yaml title="SWML" | |
| version: 1.0.0 | |
| sections: | |
| main: | |
| - join_conference: | |
| name: support-floor | |
| muted: true | |
| beep: "false" | |
| ``` |
| ```yaml title="SWML" | ||
| version: 1.0.0 | ||
| sections: | ||
| main: | ||
| - join_conference: | ||
| name: support-floor | ||
| coach: b3877ee3-6f3c-4985-8066-6d24e3f65e12 | ||
| ``` |
There was a problem hiding this comment.
Same beep problem as the listen-in sample. A coach that joins with the default beep plays the enter tone to the customer, then the exit tone when the coach leaves.
| ```yaml title="SWML" | |
| version: 1.0.0 | |
| sections: | |
| main: | |
| - join_conference: | |
| name: support-floor | |
| coach: b3877ee3-6f3c-4985-8066-6d24e3f65e12 | |
| ``` | |
| ```yaml title="SWML" | |
| version: 1.0.0 | |
| sections: | |
| main: | |
| - join_conference: | |
| name: support-floor | |
| beep: "false" | |
| coach: b3877ee3-6f3c-4985-8066-6d24e3f65e12 | |
| ``` |
| coach: b3877ee3-6f3c-4985-8066-6d24e3f65e12 | ||
| ``` | ||
|
|
||
| The agent must be in the conference when the supervisor joins. If the call ID in `coach` is not a current participant, the join fails: in cXML, `<Dial>` completes with a `DialCallStatus` of `busy` and control passes to its action URL; in SWML, `join_conference_result` reports the failure and the script continues. |
There was a problem hiding this comment.
The conference must also have started, not just contain the agent. conference_node_join (new_conference_controller.c) refuses a coach with not_ready_for_coach when the conference doesn't exist yet ("can't create a conference with a coach") or when node->state != CM_CS_STARTED. An agent who is alone and waiting, for example before the customer leg lands, can't be coached yet. Both refusals map to conference_dial_status=busy, so the outcomes you describe are right (DialCallStatus=busy in mod_laml.c's <Conference> loop, join_conference_result: failed with the script continuing in swml_method_join_conference). The REST path has the same gate: conference_node_coach returns not_ready_for_coach before the start, and Rails' 16002 check only looks at the participant row, so that request can return success without coaching starting.
| The agent must be in the conference when the supervisor joins. If the call ID in `coach` is not a current participant, the join fails: in cXML, `<Dial>` completes with a `DialCallStatus` of `busy` and control passes to its action URL; in SWML, `join_conference_result` reports the failure and the script continues. | |
| The agent must be in the conference, and the conference must have started, when the supervisor joins. If the call ID in `coach` is not a current participant, or the conference is still waiting for participants, the join fails: in cXML, `<Dial>` completes with a `DialCallStatus` of `busy` and control passes to its action URL; in SWML, `join_conference_result` is `failed` and the script continues. |
|
|
||
| ## Switch modes during the call | ||
|
|
||
| Markup sets a participant's state at join time. Changing it afterward is a cross-call operation, so it belongs to the REST API rather than to the supervisor's own SWML or cXML: [Update a Participant][update-participant] on the Compatibility REST API changes `Muted`, `Coaching`, and `CallSidToCoach` for any participant, whether the conference was joined from SWML or cXML. A supervisor console with three buttons makes one request per press. |
There was a problem hiding this comment.
I couldn't confirm that this works for a conference joined from SWML, and the end-to-end sample relies on it. For a cXML conference the update reaches the member by call SID: Blade::Laml::ConferenceParticipantUpdateSerializer sends call_id: participant.laml_call_id. A conference created by SWML join_conference (or by a Calling API dial with inline SWML, as in the SDK sample) is a Relay conference. For those, API::Laml::ConferenceParticipant::Operations::Update uses Blade::Relay::ConferenceParticipantUpdateSerializer, which sends uuid: participant.id. That's the relay_conference_participants row's own primary key: Kafka::Relay::Event::Calling::Conferencing::Participant::Join creates the row without setting id, and it's the only place that creates one. On the other side, conference_action_conference_participant_update finds the member with conference_node_member_locate(node, segment_id), and that hash is keyed by the call's segment ID. From source, the request would get "Conf member not found" and the supervisor's mode wouldn't change, even though the REST response and the participant record say it did.
I may be missing a mapping, but the PR's testing boxes are unchecked. Before this ships, run the Python sample against a live space and confirm the supervisor's audio actually changes. If it doesn't, either the page scopes mid-call switching to cXML conferences, or it's a backend bug to raise before we document the flow.
|
|
||
| The request is form-encoded, addressed by the conference SID and the supervisor's call ID: | ||
|
|
||
| ```bash title="Listen in to whisper" |
There was a problem hiding this comment.
Code block title. House style is <Language> — <client>, and the host should be the same placeholder the rest of the page uses. The page mixes example.signalwire.com, your-space.signalwire.com, $PROJECT_ID, and "your-project-id". mdx-capability-guides asks for <YOUR_SPACE>, <YOUR_PROJECT_ID>, <YOUR_API_TOKEN> and so on, explained once in a | Value | Replace with | table before the first code block. No compatibility-API reference example covers coaching, so a cURL block is the right call here.
| ```bash title="Listen in to whisper" | |
| ```bash title="cURL — Compatibility API" |
| <Note title="TypeScript `dial()` signature depends on the SDK version"> | ||
| The TypeScript sample pins `@signalwire/sdk@2.0.5`, whose `dial()` takes a single options object. The [TypeScript `dial` reference][sdk-ts-dial] documents the positional `dial(from, to, options)` form of a newer release. Match the form to the version you install. | ||
| </Note> |
There was a problem hiding this comment.
Delete this note once the sample uses the current signature. Pin-dependent notes are what we're removing in #908.
|
|
||
| ## Keep the console in sync | ||
|
|
||
| Subscribe to participant events so the console's buttons reflect state set anywhere, including a mute from another tool. In SWML, set `status_callback` and `status_callback_event` on the `join_conference` that creates the conference; in cXML, set `statusCallback` and `statusCallbackEvent` on `<Conference>`. The `join`, `leave`, `mute`, `hold`, and `speaker` events carry each participant's mute state and coaching state: `muted` and `coaching` in SWML callbacks, `Muted`, `Coaching`, and `CallSidToCoach` in cXML callbacks. |
There was a problem hiding this comment.
A coaching switch doesn't arrive on any of the events listed here. coach-start/coach-stop fire conference::participant-coach/participant-coach-stop. new_conference_controller.c maps both to participant-modify and forwards them only when status_callback_event contains modify. cXML's statusCallbackEvent has no modify, so a cXML console never hears about a coaching change made over REST. (Laml::ConferenceStatusCallbackSerializer and the cXML event table on main list no participant-modify.) The relay-format SWML payload also carries the coached call as call_id_to_coach, which is how the join_conference reference on main names it.
| Subscribe to participant events so the console's buttons reflect state set anywhere, including a mute from another tool. In SWML, set `status_callback` and `status_callback_event` on the `join_conference` that creates the conference; in cXML, set `statusCallback` and `statusCallbackEvent` on `<Conference>`. The `join`, `leave`, `mute`, `hold`, and `speaker` events carry each participant's mute state and coaching state: `muted` and `coaching` in SWML callbacks, `Muted`, `Coaching`, and `CallSidToCoach` in cXML callbacks. | |
| Subscribe to participant events so the console's buttons reflect state set anywhere, including a mute from another tool. In SWML, set `status_callback` and `status_callback_event` on the `join_conference` that creates the conference, and include `modify`: a coaching change made over REST arrives only as a `participant-modify` event. In cXML, set `statusCallback` and `statusCallbackEvent` on `<Conference>`; cXML has no event for a coaching change, so read the participant record after each switch. The `join`, `leave`, `mute`, `hold`, and `speaker` events carry each participant's mute and coaching state: `muted`, `coaching`, and `call_id_to_coach` in SWML callbacks, and `Muted`, `Coaching`, and `CallSidToCoach` in cXML callbacks. |
| ## Limits | ||
|
|
||
| - **Conference only.** A directly bridged two-party call cannot be monitored. Put agent calls in a conference from the start if supervisors may need to join. | ||
| - **The agent must be present first.** A `coach` that names a call outside the conference fails the join, and a REST update naming one returns error 16002. |
There was a problem hiding this comment.
| - **The agent must be present first.** A `coach` that names a call outside the conference fails the join, and a REST update naming one returns error 16002. | |
| - **The agent must be present first, and the conference must have started.** A `coach` that names a call outside the conference, or a coach who joins before the conference starts, fails the join. A REST update naming a call outside the conference returns error 16002. |
| title: Supervisor call monitoring | ||
| slug: /voice/supervisor-call-monitoring | ||
| description: Listen in on a live call silently, whisper to an agent without the customer hearing, or barge in, using SignalWire conferences from SWML, cXML, REST, or the Server SDKs. | ||
| max-toc-depth: 3 |
There was a problem hiding this comment.
Title and slug, nits. mdx-capability-guides wants a verb-first task title, e.g. "Monitor and coach live calls" or "Listen in, whisper, and barge on calls". The sibling guides use a bare slug (voice/record-calls, voice/call-queues), so drop the leading slash on line 3 to match.
| <Card title="Update a Participant" href="/docs/compatibility-api/rest/conference-participants/update-participant" icon="regular rectangle-api"> | ||
| The REST request that switches a participant between modes. | ||
| </Card> | ||
| <Card title="Server SDK dial" href="/docs/server-sdks/reference/python/rest/calling/dial" icon="regular server"> |
There was a problem hiding this comment.
Point this card at something every SDK reader can use. The house rule keeps per-language reference pages out of guide links. Swap this card for the unified calling guide (/docs/platform/voice/make-and-receive-calls#place-a-call-via-swml) or drop it.
Description
Add a guide for supervisor call monitoring, and correct errors in relevant references and examples.
New
/docs/platform/voice/supervisor-call-monitoring.Leads with the conference-only constraint, maps listen in / whisper / barge onto
muted+coach, shows the join in SWML, the REST participant update thatswitches modes mid-call, and an end-to-end Server SDK flow in Python and TypeScript.
Includes limits, a "Not legal advice" consent warning, and a disambiguation list
(call whisper, AI barge,
ai_sidecar,stream/tap).Fixes
coachdirection was inverted on all four SDKjoin-conferencepages, the cXML<Conference>noun, and the SWMLjoin_conferencereference and.tsp. Theparameter names the participant being coached; the call that sets it is the coach
(verified against source code).
coachnow says UUID call ID, matching the validator and its own example.C# string fixed, unverifiable 13240 claim replaced with the actual behavior
(
DialCallStatus=busy).update-participantno longer reads as mute/hold only; coaching example added.Not in this PR
specs/entirely (needs.tspwork).join_conference_resultdocumented values don't match the engine (joined/failed).Type of Change
Related Issues
https://github.com/signalwire/cloud-product/issues/20825
Testing
Checklist