Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion dart/moq_ffi/lib/src/moq.dart
Original file line number Diff line number Diff line change
Expand Up @@ -12693,7 +12693,7 @@ void _checkApiChecksums() {
throw UniffiInternalError.panicked("UniFFI API checksum mismatch");
}
if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_recv_datagram() !=
29049) {
17412) {
throw UniffiInternalError.panicked("UniFFI API checksum mismatch");
}
if (uniffi_moq_ffi_checksum_method_moqtrackconsumer_recv_group() != 60887) {
Expand Down
6 changes: 6 additions & 0 deletions doc/concept/standard.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ because they do not issue joining fetches. Other publishers may replay a cached
backlog for that filter; selecting the next group instead would leave static
tracks waiting for a group that never arrives.

A moq-lite datagram is a single-frame group, so on moq-transport it travels
as an `OBJECT_DATAGRAM` at object 0 whose Group ID is the sequence, and a relay
forwards it without renumbering. A datagram carrying any other Object ID, or a
status other than Normal, is dropped. JavaScript does not yet carry datagrams
on moq-transport.

Several project drafts extend the IETF wire without breaking it, since `SETUP`
ignores unknown parameters: [cluster](/draft/moq-cluster) routing hop lists,
[solicit](/draft/moq-solicit) to make announcements opt-in,
Expand Down
2 changes: 1 addition & 1 deletion doc/lib/rs/moq-net.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ above ([hang](/lib/rs/hang)); relays and CDNs implement only this.
- **Tracks** carry groups with a priority, a retention window, and a timescale. Subscribers set their own priority and max age and can change them live.
- **Groups** are written frame by frame and delivered on independent streams. Old groups are cached for fetch-by-sequence; stale groups are skipped per the subscriber's budget.
- **Track ends**: `finish()` ends a track at its live edge, while `finish_at(n)` declares the exclusive end ahead of it and still accepts the groups below. A subscriber awaits it with `finished()`. A remote track ends only once every group below its end has arrived or was dropped; one reset before its header arrived is skipped after the subscription's max age on moq-lite (one second without one), or after one second on IETF.
- **Datagrams** send a single small frame unreliably on moq-lite 05+.
- **Datagrams** send a single small frame unreliably on moq-lite 05+ and moq-transport.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update binding API comments for moq-transport datagrams

Although this page and the wrapper READMEs now advertise moq-transport datagrams, the generated binding API documentation remains contradictory: rs/moq-ffi/src/consumer.rs:633-636 explicitly says they are unavailable over IETF moq-transport, while rs/libmoq/src/api.rs:3617-3618 still documents only moq-lite. Users reading the exported API rather than the README will incorrectly treat the new path as unsupported, so update both source comments alongside this capability change. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L27-L27

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 8632aeb: the moq-ffi recv_datagram and libmoq moq_consume_datagrams comments now include moq-transport, and the Dart bindings are regenerated for the new checksum.

(Written by Claude Opus 5.5)

- **Routes** record the relay hops and a cost, which is what the relay [cluster](/bin/relay/cluster) routes on. A hop of 0 marks the chain anonymous: `Route::is_anonymous()` is true, and that route ranks below every fully identified one. `Route::source()` says where a delivered route entered: `Source::Local`, or `Source::Peer(hop)` when a handle marked `origin::Producer::peer()` announced it. `origin::Consumer::local()` sees only the local ones.
- **Stats** counters per broadcast and session, drained by [`moq-stats`](https://docs.rs/moq-stats).

Expand Down
3 changes: 2 additions & 1 deletion drafts/draft-lcurley-moq-e2ee.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,8 @@ An Object ID above `2^32-1` MUST be refused as `identity` before encryption or d

## Datagrams
A datagram uses `domain = 0x01`, its 64-bit sequence as `group`, and `frame = 0`.
MoQ Transport has no datagram mapping in this profile; shared vectors cover grouped tracks on both transports and datagrams on moq-lite only.
On MoQ Transport, a datagram is an Object at ID 0 in an OBJECT_DATAGRAM whose Group ID is the sequence.
Shared vectors cover grouped tracks on both transports and datagrams on moq-lite only.

## Nonce
The 96-bit AES-GCM nonce is:
Expand Down
2 changes: 1 addition & 1 deletion go/wrapper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,7 @@ Raw tracks support best-effort datagrams alongside groups: `TrackProducer.Append
sends one `Frame` and returns its sequence number, while `TrackConsumer.RecvDatagram`
and `TrackConsumer.Datagrams` receive them in arrival order. Payloads are capped at
1200 bytes. Datagram delivery requires a datagram-capable transport and lite-05 or
newer moq-lite; IETF moq-transport, pre-lite-05, WebSocket, and TCP paths do not
newer moq-lite, or moq-transport; pre-lite-05, WebSocket, and TCP paths do not
deliver them, and there is no stream fallback.

## Versioning
Expand Down
2 changes: 1 addition & 1 deletion kt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ The `dev.moq` package is intentionally thin: Kotlin has extension functions, so
- **Fetched media**: `fetchMediaGroup(...).frames()` streams the decoded frames of one retained group, then completes.
- **Duration extensions** (`Durations.kt`): the FFI carries microseconds as integers, so `stats.rtt`, `backoff.initial`, `frame.timestamp`, and their siblings read back as a `kotlin.time.Duration`.
- **`logLevel(...)`**: configures native Rust tracing without importing the raw bindings package.
- **Raw datagrams**: `TrackProducer.appendDatagram(Frame(payload, timestampUs))` sends one best-effort frame and returns its sequence; `TrackConsumer.recvDatagram()` and `datagrams()` receive them. Payloads are capped at 1200 bytes, require a datagram-capable transport plus lite-05 or newer moq-lite, and have no stream fallback.
- **Raw datagrams**: `TrackProducer.appendDatagram(Frame(payload, timestampUs))` sends one best-effort frame and returns its sequence; `TrackConsumer.recvDatagram()` and `datagrams()` receive them. Payloads are capped at 1200 bytes, require a datagram-capable transport plus lite-05 or newer moq-lite or moq-transport, and have no stream fallback.
- **`MoqException.isShutdown`** (`Errors.kt`): true for the graceful `Cancelled`/`Closed` cases.

## Versioning
Expand Down
2 changes: 1 addition & 1 deletion py/moq-rs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ Every handle whose cleanup is `cancel()` is an async context manager, so exiting
- **`Catalog`**. `.audio: dict[str, Audio]`, `.video: dict[str, Video]`, `.display`, `.rotation`, `.flip`.
- **`Frame`**. `.payload: bytes`, `.timestamp_us: int`. The unit of every write and every raw read.
- **`MediaFrame`**. `.payload: bytes`, `.timestamp_us: int`, `.keyframe: bool`. Returned by media subscriptions. `keyframe` marks a group start or video keyframe; for audio it is true only at a group start.
- **`Datagram`**. `.sequence: int`, `.timestamp_us: int`, `.payload: bytes`. Delivered only on datagram-capable transports and lite-05 or newer moq-lite.
- **`Datagram`**. `.sequence: int`, `.timestamp_us: int`, `.payload: bytes`. Delivered only on datagram-capable transports with lite-05 or newer moq-lite, or moq-transport.
- **`Audio`**. `.codec`, `.sample_rate`, `.channel_count`, `.bitrate`, `.description`.
- **`Video`**. `.codec`, `.coded: Dimensions`, `.display_aspect`, `.bitrate`, `.stalled`, `.framerate`, `.description`. A true `.stalled` recommends temporarily avoiding the rendition without making it unavailable.
- **`Subscription`**. Subscriber delivery preferences: priority, staleness, and optional group range.
Expand Down
2 changes: 2 additions & 0 deletions quest/m1/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ transport, benchmark tooling); worktrees isolate commits, not semantics.
- [Subgroup refusal](/quest/m1/ietf-subgroup-refusal.md) - a non-zero subgroup stream from a moq-transport peer ends that stream, never the session
- [Moxygen compatibility](/quest/m1/moxygen/README.md) - one subgroup per group, whole-group FETCH, and one datagram per group, never a full moxygen pass
- [Fetch without SUBSCRIBE](/quest/m1/ietf-fetch-only.md) - a relay fetches from an IETF upstream without subscribing, finished tracks included, with End of Track always reported
- [Datagram range](/quest/m1/datagram-range.md) - a subscriber gets only the datagrams its subscription asked for, on both protocols, not the buffered backlog
- [JavaScript FETCH](/quest/m1/js-fetch.md) - generic on-demand group serving and IETF FETCH for browser publishers
- [JavaScript moq-transport datagrams](/quest/m1/js-ietf-datagram.md) - `@moq/net` carries datagrams over moq-transport as `OBJECT_DATAGRAM`, like Rust
- [Archive](/quest/m1/archive/README.md) - record selected tracks to any object_store and replay them over FETCH or derived HLS, on the catalog and store the release ships
- [Wildcard](/quest/m1/wildcard/README.md) - a relay resolves subscriptions against advertised prefixes, a service claims the prefix it could serve and refuses the rest instead of enumerating broadcasts, and the browser player treats a covering claim as availability
- [Tooling](/quest/m1/tooling/README.md) - justfiles become a one-line menu over `sh/`, one impact map scopes CI, and every workflow step runs a recipe
Expand Down
29 changes: 29 additions & 0 deletions quest/m1/datagram-range.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# [S] Datagram range

## Goal

A subscriber receives only the datagrams its subscription asked for, on
moq-lite and on moq-transport, in Rust and JavaScript. A new subscriber
is not handed datagrams from before its start.

## Plan

Datagrams share the group sequence namespace, but their cursor ignores the
subscription's start and end. The model buffers the last 64 per track, and a
new subscriber's cursor starts at the oldest. So a late joiner, or a relay
fanning out a fresh downstream, gets stale datagrams first. Both protocols do
this today, and the moxygen line kept moq-transport matching moq-lite.

Settle the rule once and apply it to both protocols. It could be the cursor
starting at the live edge, the subscribe range bounding datagrams the way it
bounds groups, or both. Prefer fixing it in the model over filtering in each
session.

Watch the edge cases: a datagram at the start group when a frame offset
skips object 0, SUBSCRIBE_UPDATE moving the range, and a datagram that
lands before the subscription's alias or id is known. A test must tell a
filtered datagram apart from one dropped for any other reason.

## Related

- [Moxygen compatibility](/quest/m1/moxygen/README.md) - brought datagrams to moq-transport with moq-lite's behavior
2 changes: 1 addition & 1 deletion quest/m1/e2ee/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ The Rust and TypeScript cores expose the same surface, and nothing else:

- Deterministic secret-derived physical names hide catalog, codec, role, quality, timeline, and custom-track semantics. Authorized clients derive the encrypted catalog track name, then learn the remaining opaque names from its decrypted contents. Every catalog representation is encrypted; Rust publishers must not emit a plaintext MSF catalog.
- A platform that forwards and meters protected bytes must never preview, record, archive, transmux, transcode, transcribe, compose, or inspect them, rejecting those paths before opening a processing session or writing product state. Applications needing those operations terminate E2EE outside the platform. A platform classifies protected broadcasts by its own credential or product state, never by name; the moq.pro (downstream) exclusion classifier and dashboard work stay downstream.
- The first proof covers browser TypeScript and native Rust publication and playback in both directions, with grouped audio and video over both moq-lite and MoQ Transport. Shared vectors cover groups and moq-lite datagrams; MoQ Transport has no datagram delivery.
- The first proof covers browser TypeScript and native Rust publication and playback in both directions, with grouped audio and video over both moq-lite and MoQ Transport. Shared vectors cover groups and moq-lite datagrams; JavaScript has no MoQ Transport datagram delivery yet.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the normative E2EE datagram mapping

This now says only JavaScript lacks MoQ Transport datagram delivery, but drafts/draft-lcurley-moq-e2ee.md:228-230 still normatively states that the profile has no MoQ Transport datagram mapping and limits shared vectors to moq-lite. Protected Rust datagrams now use this path, so the draft contradicts the implementation and would lead E2EE interop work to omit or reject it. Update the draft's mapping and vector requirements in this change. (Written by GPT-5.6 Sol)

AGENTS.md reference: AGENTS.md:L27-L27

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 8632aeb: the draft now maps a MoQ Transport datagram to an OBJECT_DATAGRAM at Object ID 0 whose Group ID is the sequence, so the identity is unchanged. Shared vectors stay transport-independent; rs/moq-e2ee/tests/transport.rs gains datagrams_over_ietf.

(Written by Claude Opus 5.5)


## Quests

Expand Down
21 changes: 21 additions & 0 deletions quest/m1/js-ietf-datagram.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# [M] JavaScript moq-transport datagrams

## Goal

`@moq/net` sends and receives datagrams over moq-transport as
`OBJECT_DATAGRAM`, matching Rust: one Object at ID 0 is a single-frame group
whose Group ID is the sequence. A JavaScript publisher's datagrams reach a Rust
subscriber, and a Rust publisher's reach a JavaScript one.

## Plan

Port `rs/moq-net/src/ietf/datagram.rs` and the session's send and receive
loops. Decode every draft's Type flags, drop what the model cannot carry the
same way Rust does, and close the session on a malformed Type.

The integration test `ietf does not deliver datagrams` flips to delivery on
every supported draft, and `just test interop --all` covers both directions.

## Related

- [Datagram range](/quest/m1/datagram-range.md) - the subscribe range for datagrams, settled on both protocols
1 change: 0 additions & 1 deletion quest/m1/moxygen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ Docs stay inline in the change that makes them stale. No new guide.
## Quests

- [Sparse FETCH ranges](/quest/m1/moxygen/fetch-span.md) - a FETCH costs the groups it returns, not the span of its range
- [Datagram groups](/quest/m1/moxygen/datagram.md) - an IETF datagram that is one object in a group arrives as a moq-lite datagram group

## Related

Expand Down
23 changes: 0 additions & 23 deletions quest/m1/moxygen/datagram.md

This file was deleted.

2 changes: 1 addition & 1 deletion rs/libmoq/src/api.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3615,7 +3615,7 @@ pub extern "C" fn moq_consume_track_cancel(track: u32) -> i32 {
/// touched again, so release `user_data` there. The terminal callback fires even after
/// [moq_consume_datagrams_cancel]. Read each datagram with [moq_consume_datagram] and release
/// it with [moq_consume_datagram_free]. Datagrams arrive only over datagram-capable
/// transports and lite-05 or newer moq-lite; there is no stream fallback.
/// transports on moq-transport or lite-05 and newer moq-lite; there is no stream fallback.
///
/// Returns a non-zero handle to the subscription on success, or a negative code on failure.
///
Expand Down
17 changes: 13 additions & 4 deletions rs/moq-e2ee/tests/transport.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//! Grouped tracks on moq-lite and MoQ Transport; datagrams on moq-lite.
//! Grouped tracks and datagrams on moq-lite and MoQ Transport.

mod support;

Expand Down Expand Up @@ -104,10 +104,9 @@ async fn grouped_over_ietf() {
.expect("timed out");
}

#[tokio::test]
async fn datagrams_over_lite() {
async fn datagram_roundtrip(version: &str) {
tokio::time::timeout(TEST_TIMEOUT, async {
let mut fixture = connect_protected("moq-lite-05".parse().unwrap(), "audio").await;
let mut fixture = connect_protected(version.parse().unwrap(), "audio").await;
fixture
.producer
.append_datagram(Timestamp::from_millis(9).unwrap(), b"opus")
Expand All @@ -123,3 +122,13 @@ async fn datagrams_over_lite() {
.await
.expect("timed out");
}

#[tokio::test]
async fn datagrams_over_lite() {
datagram_roundtrip("moq-lite-05").await;
}

#[tokio::test]
async fn datagrams_over_ietf() {
datagram_roundtrip("moq-transport-21").await;
}
2 changes: 1 addition & 1 deletion rs/moq-ffi/src/consumer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -633,7 +633,7 @@ impl MoqTrackConsumer {
/// Receive the next best-effort datagram in arrival order.
///
/// Returns `None` when the track ends. Datagram delivery is unavailable over
/// IETF moq-transport, pre-lite-05 moq-lite, and stream-only transports.
/// pre-lite-05 moq-lite and stream-only transports.
/// Datagrams are a separate cursor from groups, so this works alongside either
/// group order, never commits the track to one, and progresses while a group
/// read is pending.
Expand Down
3 changes: 2 additions & 1 deletion rs/moq-net/src/fuzz.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ const IETF_VERSIONS: &[ietf::Version] = &[
const LITE_KINDS: u8 = 21;

/// How many types [`ietf_wire`] dispatches over.
const IETF_KINDS: u8 = 39;
const IETF_KINDS: u8 = 40;

/// Split the two selector bytes off the input: a version and a type.
fn select(data: &[u8], versions: usize) -> Option<(usize, u8, &[u8])> {
Expand Down Expand Up @@ -318,6 +318,7 @@ pub fn ietf_wire(data: &[u8]) -> bool {
36 => roundtrip::<ietf::Location, _>(rest, version, stable),
37 => roundtrip::<ietf::FetchObject, _>(rest, version, stable),
38 => roundtrip::<ietf::PublishNamespaceUpdate, _>(rest, version, stable),
39 => roundtrip::<ietf::ObjectDatagram, _>(rest, version, stable),
_ => unreachable!("kind is taken modulo IETF_KINDS"),
}
}
Expand Down
Loading
Loading