diff --git a/quest/m2/README.md b/quest/m2/README.md index 6424bd4a59..aaf63ea5af 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -119,6 +119,12 @@ before format-specific metadata. Unrelated areas can proceed in parallel. - [JS track end](/quest/m2/2318-js-net-remaining-capability-gaps-vs-rs-moq-net-setup-role.md) - js/net declares a track end ahead of the live edge and observes the publisher's SUBSCRIBE_END - [E2EE](/quest/m2/e2ee/README.md) - TypeScript and Rust peers interoperate over encrypted broadcasts no relay can decrypt - [Processor](/quest/m2/processor/README.md) - a customer-run worker publishes an on-demand contribution with scoped access +- [LiveKit Agents adapter (Python)](/quest/m2/livekit-agents-python.md) - a `livekit-agents` voice session runs against a MoQ relay through a MoQ audio input and output and an announce-prefix runner +- [LiveKit client shim](/quest/m2/livekit-shim.md) - a drop-in `livekit-client` facade over `@moq/room`, media and core events first +- [LiveKit shim data surfaces](/quest/m2/livekit-shim-data.md) - publishData, text streams, and RPC on the data convention +- [Data convention](/quest/m2/data-convention.md) - one request/response shape over the catalog's `json` and `binary` sections, plus the `window` mode, so rooms, teleop, and voice stop hand-rolling reply channels +- [Active speaker](/quest/m2/room-active-speaker.md) - audio level and speaking signals on every `@moq/room` member, and a `moq-audio` meter for native apps +- [LiveKit Agents adapter (Node)](/quest/m2/livekit-agents-node.md) - the Python adapter ported to `@livekit/agents` - [#3056](/quest/m2/3056-watch-video-decoder-captures-the-rewind-generation-at.md) - watch: the video decoder resets on a declared discontinuity - [#933](/quest/m2/933-video-rotation-metadata-not-propagated-from-mobile-camera.md) - the catalog rotation follows the live camera's orientation - [#2075](/quest/m2/2075-mirror-catalog-reservation-gating-in-moq-hang-js-hang.md) - @moq/publish gates the first catalog snapshot until every reserved track is described diff --git a/quest/m2/data-convention.md b/quest/m2/data-convention.md new file mode 100644 index 0000000000..4e826e0bbf --- /dev/null +++ b/quest/m2/data-convention.md @@ -0,0 +1,63 @@ +# [M] Data convention + +## Goal + +One request/response convention over the catalog's existing `json` and +`binary` data sections (#3109), so rooms, teleoperation, voice agents, and +any app stop hand-rolling reply channels: a caller's `stream`-mode track of +`{id, method, params}` answered on the callee's `stream`-mode track of +`{id, ok, result}` or `{id, error}`, with the callee advertised in its +catalog and the caller discovered from an announce prefix. The `window` +mode moq-json already implements joins the catalog's known modes, every +in-tree data track declares itself, and the hang draft carries all of it. + +Today three consumers each invented a reply channel: the Pipecat +transport's fixed-name `transcript.json.z` track, teleop's proposed `rpc` +tracks, and pronto's `status` echoing a command `sequence`. `@moq/room`'s +chat window is not in the catalog at all. + +## Plan + +- Modes: add `window` (the retained run moq-json's `js/json/src/window` + writes) as a JSON-only known mode. `KnownMode` in + `js/hang/src/catalog/mode.ts` and `Mode` in `rs/hang/src/catalog/mode.rs` + are shared by both sections today, and `@moq/binary` and `moq-binary` + implement only snapshot and stream, so support detection splits by + section (`jsonModeSupported` and `binaryModeSupported`, or per-section + known sets) and a binary `window` entry stays unreadable rather than + appearing supported. Datagram delivery is a track `Info` property, not a + mode, and stays out of the catalog. +- Request/response: `moq-json` gains typed `Request`/`Response` producers + and consumers over its stream mode in JS and Rust: the caller keeps the + correlation id and its own timeout, the callee answers each id once, and + a lagged reader surfaces the error rather than retrying. Discovery is the + direction teleop and Voice already use: the callee's response track is a + `json` entry with a `schema` naming the method set, and callers are found + under an announce prefix (operator publishes, robot subscribes; + `request/` and `response/` subtrees). +- Reliability is what `moq-net` gives (a group is one QUIC stream; a reader + that falls past `MAX_GROUP_CACHE` is told so). The docs say that instead + of promising delivery. Non-goals: byte-stream file transfer, delivery + receipts, per-participant addressing beyond a path and a scoped token, + and a cross-host timebase. +- Adopt in the same change: `@moq/room` `Chat` and its native twin + `moq_room::chat::Publisher` (`rs/moq-room/src/chat.rs`) declare their + `window` track in the `json` section, so Rust and JavaScript room + catalogs agree; `rs/moq-native/examples/chat.rs` and + `rs/moq-json/examples/telemetry.rs` declare theirs. The draft gains the + `window` mode and the request/response shape, and `doc/concept`'s + catalog page gains both, per the cross-package checklist for an `rs/hang` + catalog change. A catalog without data sections parses unchanged, which + the existing `deserialize_section` leniency already guarantees. +- Tests: `window` round trip in both languages, a request answered across a + local relay, a lagged caller surfacing the error, and an unknown mode + still passing through verbatim. + +## Related + +- [Robot teleoperation primitive](/quest/m3/teleop/robot.md) - the `rpc`, + `telemetry`, and `command` tracks are the first non-room consumer +- [LiveKit shim data surfaces](/quest/m2/livekit-shim-data.md) - + `publishData`, streams, and RPC map onto this +- [Catalog track identity](/quest/m3/catalog-tracks.md) - whatever it + decides about immutable definitions applies to data entries too diff --git a/quest/m2/livekit-agents-node.md b/quest/m2/livekit-agents-node.md new file mode 100644 index 0000000000..0ea7dee0d0 --- /dev/null +++ b/quest/m2/livekit-agents-node.md @@ -0,0 +1,24 @@ +# [M] LiveKit Agents adapter (Node) + +## Goal + +The Python adapter's design ported to `@livekit/agents`: an `AudioInput` +(`ReadableStream`) and `AudioOutput` (`captureFrame`, `flush`, +`clearBuffer`) over `@moq/net` and `@moq/hang`, plus the same announce-prefix +runner, published from `js/` as `@moq/livekit-agents`. Done when the stock +Node voice-agent example answers a browser publisher through a local relay. + +## Plan + +- `AgentSession.start({agent})` builds `RoomIO` only when a room is passed + (`agent_session.ts`), and their tests assign `session.input.audio` + directly, so the shape is the same as Python. `@livekit/rtc-node` stays a + dependency for `AudioFrame`. +- Reuse the Python adapter's track naming, barge-in replacement, RTVI turn + mapping, and tests one for one, so a browser publisher is agnostic to which + runtime answers. + +## Required + +- [LiveKit Agents adapter (Python)](/quest/m2/livekit-agents-python.md) - + settles the design this ports diff --git a/quest/m2/livekit-agents-python.md b/quest/m2/livekit-agents-python.md new file mode 100644 index 0000000000..12c8facd2d --- /dev/null +++ b/quest/m2/livekit-agents-python.md @@ -0,0 +1,48 @@ +# [M] LiveKit Agents adapter (Python) + +## Goal + +A `livekit-agents` voice agent runs against a MoQ relay instead of a LiveKit +room, with its STT, LLM, TTS, turn detection, and plugins unchanged: a +`moq-livekit-agents` package in `py/` supplies an `AudioInput` and +`AudioOutput` over MoQ and a runner that starts one `AgentSession` per +announced request broadcast and publishes its response. Done when a stock +LiveKit voice-agent example answers a browser publisher through a local relay +with only the I/O wiring changed. + +## Plan + +- The framework already runs without a room. `AgentSession.start(agent)` + with `session.input.audio` and `session.output.audio` assigned is how its + own test harness (`tests/fake_io.py`) and `console` mode + (`cli/tcp_console.py`) work, and `get_job_context(required=False)` keeps a + session independent of a Worker. Subclass `voice/io.py`'s `AudioInput` + (async iterator of `rtc.AudioFrame`) and `AudioOutput` (`capture_frame`, + `flush`, `clear_buffer`, and `on_playback_finished` when the track drains). +- Media the way Pipecat's MoQ transport does it (`pipecat.transports.moq`): + the `moq` package encodes and decodes Opus inside the FFI, the agent's + audio is one hang audio track replaced by a fresh one on barge-in so + catalog-aware players follow the successor, and frames cross as int16 at + the session's sample rate. `livekit-rtc` stays a dependency for the frame + type; no WebRTC connection is ever opened. +- The runner replaces dispatch: consume an announce prefix, start a session + per `request/` broadcast, publish at `response/`, end the session on + unannounce. The disjoint subtrees are the loop guard. A scoped token + (subscribe `request/**`, publish `response/**`) is the whole credential; + there is no registration and no dispatch API. +- Client turn signals ride the same RTVI JSON track the Pipecat transport + uses, mapped onto `session.interrupt()` and `session.commit_user_turn()`, + so one browser publisher works with either framework's agent. +- The README states what `RoomIO` provides and this does not: participant + attributes, `lk.chat` text input, transcription forwarding, `SessionHost`, + and RPC. Text and transcript surfaces follow the + [data convention](/quest/m2/data-convention.md) when it lands. +- Tests: a fake `AgentSession` I/O round trip over a local relay, barge-in + replacing the output track, and unannounce ending the session. Pin + `livekit-agents` to the 1.8 line and record the `io.py` surface it relies + on, since that module is not covered by their stability promise. + +## Related + +- [LiveKit Agents adapter (Node)](/quest/m2/livekit-agents-node.md) +- [LiveKit client shim](/quest/m2/livekit-shim.md) - the client-side half diff --git a/quest/m2/livekit-shim-data.md b/quest/m2/livekit-shim-data.md new file mode 100644 index 0000000000..d9929db590 --- /dev/null +++ b/quest/m2/livekit-shim-data.md @@ -0,0 +1,26 @@ +# [M] LiveKit shim data surfaces + +## Goal + +`publishData`, text streams, and RPC in the LiveKit client shim work instead +of throwing: reliable `publishData` is a `stream`-mode `binary` track and +lossy a `snapshot`-mode one, text streams are `stream`-mode `json` tracks +keyed by topic, and RPC is the data convention's request/response pair +addressed by the callee's identity path. Byte streams with progress stay +unimplemented and throw. Done when a LiveKit sample that uses each of the +three runs against a relay unchanged. + +## Plan + +- Every track the shim publishes is declared in the participant's + `camera.hang` catalog, so a plain `@moq/room` client can read a LiveKit + app's data too. The README maps each LiveKit method to its track and its + delivery guarantee. +- Tests: a reliable and a lossy `publishData` round trip, a topic stream + read by a late joiner (only what the stream retains), and an RPC with a + timeout and an error reply. + +## Required + +- [LiveKit client shim](/quest/m2/livekit-shim.md) +- [Data convention](/quest/m2/data-convention.md) diff --git a/quest/m2/livekit-shim.md b/quest/m2/livekit-shim.md new file mode 100644 index 0000000000..8426713838 --- /dev/null +++ b/quest/m2/livekit-shim.md @@ -0,0 +1,43 @@ +# [L] LiveKit client shim + +## Goal + +A drop-in `livekit-client`-compatible JS package (e.g. `@moq/livekit`) that +runs a multi-participant room entirely over MoQ. The v1 surface is media plus +core events: Room connect/disconnect, local and remote participants, +camera/mic/screenshare publish, auto-subscribe, and the TrackSubscribed event +family; data surfaces (publishData, streams, RPC) are stubbed. The token slot +takes an ordinary moq-auth token, no LiveKit JWT parsing. Done when an +off-the-shelf LiveKit JS sample runs against a MoQ relay with only the import +and the connect URL/token changed. + +## Plan + +- The shim is a LiveKit-API facade over `@moq/room` (`js/room`, landed in + #3634): the room is a path prefix in the connection URL and token root, + participants are discovered from the announce stream, identity is the path + before `camera.hang` and `screen.hang`, and a screenshare's + announce/unannounce is its lifecycle. The shim groups the two broadcasts + per identity into one RemoteParticipant and maps catalog entries to + TrackPublications. +- Build on `@moq/publish` and `@moq/watch`. LiveKit quality hints map to + the receiver-driven pixel target where they can (`setVideoQuality` picks + the rendition, `adaptiveStream` follows the rendered size); a hint the + target cannot express (`setVideoFPS`, per-layer bitrate caps) throws a + clear unsupported error rather than no-oping, so an app never believes a + limit is active. +- v1 is identity-only: `participant.identity` comes from the path and muted + state derives from catalog track presence. Names and coarse state come + from `@moq/room`'s `hang/*.json` metadata in a follow-up, not a rival + scheme; `ActiveSpeakersChanged` waits on + [active speaker](/quest/m2/room-active-speaker.md). +- Tokens come from `@moq/room` `claims()`: publish under `/**` + only, so participants cannot publish at each other's paths. +- `publishData`, text and byte streams, and RPC throw a clear + not-implemented error in v1; [shim data surfaces](/quest/m2/livekit-shim-data.md) + maps them onto the data convention afterwards. + +## Related + +- [LiveKit Agents adapter (Python)](/quest/m2/livekit-agents-python.md) - + the agent-side half of the same migration diff --git a/quest/m2/room-active-speaker.md b/quest/m2/room-active-speaker.md new file mode 100644 index 0000000000..23df3bd2da --- /dev/null +++ b/quest/m2/room-active-speaker.md @@ -0,0 +1,27 @@ +# [S] Active speaker + +## Goal + +`@moq/room` exposes who is talking: every `Remote` and the `Local` carry an +audio level signal and a debounced `speaking` boolean, and the `Room` +exposes the ordered active-speaker set, so a grid can highlight the speaker +and the LiveKit shim can emit `ActiveSpeakersChanged`. `moq-room` is +media-free (announce events and chat; decoded audio lives in `moq-audio` +and the application), so the native half is a level meter in `moq-audio` +over decoded PCM with the same thresholds, which a native app wires itself; +no room media layer is added. + +## Plan + +- Measure on decoded audio, independent of the member's playback mute: + `Member.muted` defaults to true and gates only the output today, so the + level analysis runs on the decoded frames before that gate (or keeps the + decoder subscribed while the sink is silent), and a muted-for-me member + still reads as speaking. Measuring on the wire is not an option since the + relay never decodes. +- Level is a smoothed RMS in dBFS; `speaking` uses an attack and release + threshold with a short hold, exposed through the existing `@moq/signals` + idiom. The `moq-audio` meter uses the same numbers and a test vector + shared with the JS one. +- The demo at `demo/web/src/meet.html` highlights the loudest tile as proof, + with every remote left muted. diff --git a/quest/m3/README.md b/quest/m3/README.md index 4f316d313b..1327411827 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -23,8 +23,8 @@ worth settling now. Deferral does not by itself abandon a feature. - [iOS capture](/quest/m3/mobile-capture-ios.md) - camera and screen capture if the mobile ownership decision selects Rust - [Android capture](/quest/m3/mobile-capture-android.md) - NDK/JNI capture and codec backends if the mobile ownership decision selects Rust - [Mobile completion](/quest/m3/mobile-completion.md) - verify the selected native/mobile path before closing #700 +- [C++ SDK](/quest/m3/cpp-sdk.md) - an idiomatic wrapper over libmoq's C ABI, with the OBS plugin as its first consumer - [Linux OBS GPU input](/quest/m3/obs-linux-gpu.md) - publish OBS compositor frames without CPU readback on a validated Linux graphics/encoder combination -- [LiveKit client shim](/quest/m3/livekit-shim.md) - a media compatibility facade over the room SDK - [QUIC capacity probing](/quest/m3/quic-probe.md) - compare useful early retransmissions with padding and no probing - [Embedded video](/quest/m3/video-embedded.md) - EGL import in the renderer, so moq-video presents on a Pi @@ -42,7 +42,8 @@ worth settling now. Deferral does not by itself abandon a feature. - [#1838](/quest/m3/1838-tr-101-290-monitoring-requirements-broadcast-contribution.md) - TR 101 290 monitoring: requirements (broadcast/contribution health metrics) - [Teleoperation](/quest/m3/teleop/README.md) - MoQ carries robot video down and control up on one session as a library capability - [SIP media stack](/quest/m3/sip-stack.md) - terminate one inbound SIP audio call leg and expose it as Opus frames +- [SIP DTMF](/quest/m3/sip-dtmf.md) - RFC 4733 keypad events become a data track beside the caller's audio +- [SIP transfer](/quest/m3/sip-transfer.md) - blind transfer over REFER and warm transfer over a second leg, driven through the data convention - [Carrier voice](/quest/m3/carrier-voice/README.md) - determine whether MoQ should be the call fabric for programmable carrier voice -- [LiveKit WebRTC bridge](/quest/m3/livekit-webrtc-bridge.md) - a go/no-go verdict, backed by a spike, on per-track LiveKit-to-MoQ bridging - [Vision worker](/quest/m3/processor-vision.md) - a documented customer-run vision worker proves the processor contract - [Common Access Tokens](/quest/m3/cat/README.md) - a moq-transport client presents a CAT in SETUP and `moq auth serve` admits it with the scope its `moqt` claim names diff --git a/quest/m3/cpp-sdk.md b/quest/m3/cpp-sdk.md new file mode 100644 index 0000000000..f849a3c975 --- /dev/null +++ b/quest/m3/cpp-sdk.md @@ -0,0 +1,25 @@ +# [M] C++ SDK + +## Goal + +C++ developers (robotics tooling, engines, native apps) get a thin, +idiomatic wrapper over `libmoq`'s C ABI: RAII handles, `std::span` payloads, +callbacks or futures for announce and subscribe, and hang audio, video, and +data tracks, installed as a CMake package beside `moq.pc`. The OBS plugin +(`cpp/obs`) moves onto it as the first consumer, and the wrapper's surface +tracks the Go, Swift, and Kotlin bindings one for one. + +## Plan + +- Header-first in `cpp/moq`, generated where the C ABI already carries the + shape (cbindgen output in `rs/libmoq`), handwritten where ownership or + callbacks need C++ semantics. C++20, no exceptions across the boundary. +- Ship through `libmoq`'s existing release pipeline (`build.sh`, CMake + config), with a smoke that publishes and watches a clock through a local + relay on Linux, macOS, and Windows. +- Unity is deliberately not a target; a C# binding is a separate decision. + +## Required + +- [#2152](/quest/m2/2152-libmoq-c-abi-catch-up-with-the-moq-ffi-surface.md) - + the C ABI has to carry sessions before a wrapper is worth publishing diff --git a/quest/m3/livekit-shim.md b/quest/m3/livekit-shim.md deleted file mode 100644 index 1e1253f46b..0000000000 --- a/quest/m3/livekit-shim.md +++ /dev/null @@ -1,36 +0,0 @@ -# [L] LiveKit client shim - -## Goal - -A drop-in `livekit-client`-compatible JS package (e.g. `@moq/livekit`) that -runs a multi-participant room entirely over MoQ. The v1 surface is media plus -core events: Room connect/disconnect, local and remote participants, -camera/mic/screenshare publish, auto-subscribe, and the TrackSubscribed event -family; data surfaces (publishData, streams, RPC) are stubbed. The token slot -takes an ordinary moq-auth token, no LiveKit JWT parsing. Done when an -off-the-shelf LiveKit JS sample runs against a MoQ relay with only the import -and the connect URL/token changed. - -## Plan - -- The shim is a LiveKit-API facade over `@moq/room`, which carries hang.live's convention: the room is a path prefix in the connection URL and token root, - participants are discovered from the bare announce stream, identity is the - next path segment, and each participant publishes `/camera` - (camera + mic, hd/sd renditions) and `/screen` (screenshare, - whose announce/unannounce is the screenshare lifecycle). The shim groups - the two paths per identity into one RemoteParticipant and maps catalog - entries to TrackPublications. -- Build on `@moq/publish` and `@moq/watch`. LiveKit quality hints - (setVideoQuality, adaptive settings) map to the receiver-driven pixel - target, or no-op gracefully. -- v1 is identity-only: `participant.identity` comes from the path and muted - state derives from catalog track presence. Names and coarse state are a - follow-up wired to the room SDK's `hang/*.json` metadata, not a rival - scheme. -- Recommend tokens scoped to `put: /` so participants cannot - publish at each other's paths (hang.live grants `put` on the whole room - subtree today). - -## Related - -- [WebRTC bridge evaluation](/quest/m3/livekit-webrtc-bridge.md) diff --git a/quest/m3/livekit-webrtc-bridge.md b/quest/m3/livekit-webrtc-bridge.md deleted file mode 100644 index a36d29916d..0000000000 --- a/quest/m3/livekit-webrtc-bridge.md +++ /dev/null @@ -1,27 +0,0 @@ -# [M] LiveKit WebRTC bridge evaluation - -## Goal - -A go/no-go verdict, backed by a working spike, on bridging LiveKit rooms to -MoQ per-track over WebRTC using LiveKit's Rust SDK: a native participant -joins the room and republishes its tracks into MoQ, optionally the reverse. -The verdict decides whether this becomes a maintained gateway. - -## Plan - -- The zero-code paths already exist and bound the value: LiveKit Egress - pushes RTMP/SRT into the existing gateways today (composited and - transcoded), and `moq export rtc --connect` publishes WHIP into LiveKit - Ingress. LiveKit has no WHIP/WHEP egress, so per-track LiveKit to MoQ - requires joining the room as a participant via livekit rust-sdks - (Apache 2.0, no Chrome/GStreamer). -- Spike it in-tree: a rust-sdks participant subscribes to all tracks and - republishes via moq-net + moq-mux, with codec passthrough where possible - (Opus/VP8/VP9; H.264 through the Annex-B importer). -- Assess dependency weight of the webrtc stack, per-track fidelity versus the - Egress paths, simulcast handling, and where a real gateway would live (edge - versus standalone). - -## Related - -- [LiveKit client shim](/quest/m3/livekit-shim.md) diff --git a/quest/m3/sip-dtmf.md b/quest/m3/sip-dtmf.md new file mode 100644 index 0000000000..dc027c7f6a --- /dev/null +++ b/quest/m3/sip-dtmf.md @@ -0,0 +1,25 @@ +# [M] SIP DTMF + +## Goal + +`moq-sip` surfaces the caller's keypad: RFC 4733 telephone-events on the +negotiated RTP leg become a `stream`-mode data track beside the caller's +audio, declared in the catalog under the data convention, and the embedder +can send digits back on an outbound leg. In-band tone detection stays out. + +## Plan + +- Negotiate `telephone-event/8000` in the SDP answer beside Opus and G.711; + decode event, end bit, and duration into one JSON frame per digit + (`{digit, duration_ms}`) with the RTP timestamp mapped onto the audio + clock so a transcript can interleave them. +- Sending: the embedder writes the same frames and the crate emits the RTP + events with the standard three end packets. +- Test against a softphone and a PBX that sends both RFC 4733 and no DTMF + at all (the track exists but stays empty). + +## Required + +- [SIP media stack](/quest/m3/sip-stack.md) - the leg this extends +- [Data convention](/quest/m2/data-convention.md) - the track shape and + its catalog entry diff --git a/quest/m3/sip-stack.md b/quest/m3/sip-stack.md index 422399ce35..45c786adb9 100644 --- a/quest/m3/sip-stack.md +++ b/quest/m3/sip-stack.md @@ -11,8 +11,7 @@ consumable by an embedder. ## Plan -- Open with an evaluation, spike-backed like the - [WebRTC bridge verdict](/quest/m3/livekit-webrtc-bridge.md): drive a real +- Open with a spike-backed evaluation: drive a real inbound call through the candidate full Rust SIP stacks (ezk-sip, rvoip, and whatever else is current) and adopt the one that holds up. Fallback if none do: an existing parser crate (e.g. rsip) for message/SDP syntax plus a @@ -31,3 +30,5 @@ consumable by an embedder. so every embedder gets answer-with-silence for free. - The SIP edge gateway and line provisioning that consume this crate as an inbound-call product are moq.pro (downstream) work. +- [SIP DTMF](/quest/m3/sip-dtmf.md) and [SIP transfer](/quest/m3/sip-transfer.md) - + the extensions that lift the v1 boundaries above diff --git a/quest/m3/sip-transfer.md b/quest/m3/sip-transfer.md new file mode 100644 index 0000000000..de682df423 --- /dev/null +++ b/quest/m3/sip-transfer.md @@ -0,0 +1,28 @@ +# [M] SIP transfer + +## Goal + +`moq-sip` moves a live call: a blind transfer sends REFER for a target URI +and follows the NOTIFY subscription to completion, and a warm transfer holds +the caller, originates a second leg to the target, and bridges the two legs +when the embedder says so. The embedder drives both through the same +request/response track the data convention defines, so an agent or an app +can transfer without SIP-shaped code. + +## Plan + +- Blind: REFER with `Refer-To`, handle the implicit subscription and the + `sipfrag` NOTIFYs, and end the original dialog on success. Refuse REFER + from the far end (405) in v1. +- Warm: reuse the origination quest's outbound leg; hold is re-INVITE with + `a=sendonly`; bridging is the embedder swapping which leg's Opus feeds + which, not a media mixer. +- Test blind against a PBX that accepts REFER and one that rejects it, and + warm end to end with a softphone as the target. + +## Required + +- [SIP media stack](/quest/m3/sip-stack.md) +- [SIP call origination](/quest/m3/carrier-voice/sip-originate.md) - warm + transfer needs the outbound leg +- [Data convention](/quest/m2/data-convention.md) - the control channel diff --git a/quest/m3/teleop/robot.md b/quest/m3/teleop/robot.md index a4c98c9e0a..44c1bd1627 100644 --- a/quest/m3/teleop/robot.md +++ b/quest/m3/teleop/robot.md @@ -61,11 +61,12 @@ The framing is where the guarantee lives, not the subscription flags: ### Contents -- The catalog section, through `moq-mux`'s `CatalogExt` and - `RenditionConfig`. hang stays media-only; the `CatalogExt` doc example is - already a `telemetry` section (`rs/moq-mux/src/catalog/tracks.rs`, with a - `gps` rendition in its tests), so this is the designed seam and needs no hang - schema change. +- The catalog section: `telemetry`, `command`, and `rpc` are entries in + the catalog's `json` section (#3109), which the browser package can + discover, superseding the earlier plan of an app-private `CatalogExt` + section (`rs/moq-mux/src/catalog/tracks.rs`); the `rpc` tracks are the + shared [data convention](/quest/m2/data-convention.md)'s request/response + shape rather than a teleop-only one. - Announce-prefix fan-in, generalised from `rs/moq-boy/src/input.rs`. - The two delivery classes, as `moq-json`'s snapshot and stream modes with the group structure and `Info::latency_max` each one needs. @@ -81,6 +82,10 @@ Port `moq-boy` onto the crate in the same change, as the no-arbitration case. It is the only existing consumer, and if the abstraction cannot express crowd control then it is the wrong abstraction. +## Required + +- [Data convention](/quest/m2/data-convention.md) - the request/response shape the `rpc` tracks adopt + ## Related - [arbitration](/quest/m3/teleop/arbitration.md) - which controller is obeyed