diff --git a/quest/m1/qos/README.md b/quest/m1/qos/README.md index 0a0f2deed0..3d70c72e36 100644 --- a/quest/m1/qos/README.md +++ b/quest/m1/qos/README.md @@ -52,3 +52,5 @@ on `main`. - [Media stats](/quest/m1/stats/README.md) - publishers and viewers report their own media, transport, and playback health, the media half of a verdict +- [Loss delay](/quest/m3/cut-through/loss-delay.md) - an ingress counter of + bytes a loss held back by at least one RTT, on the same rows diff --git a/quest/m3/README.md b/quest/m3/README.md index 02d73b4c1c..8695f91439 100644 --- a/quest/m3/README.md +++ b/quest/m3/README.md @@ -26,6 +26,7 @@ deleted when it goes stale; git history keeps it. - [#3201: moq-uring: use SENDMSG_ZC for large UDP GSO trains](/quest/m3/3201-moq-uring-use-sendmsg-zc-for-large-udp-gso-trains.md) - complete the prerequisite issue first - [#3204](/quest/m3/3204-moq-uring-register-tx-pool-buffers-for-zero-copy-sends.md) - moq-uring: register TX-pool buffers for zero-copy sends - [QUIC GCC](/quest/m3/quic-gcc.md) - receive timestamps in ACKs and a measured verdict on delay-based congestion control for media egress, shipping as `RealTime` +- [Cut-through](/quest/m3/cut-through/README.md) - a relay forwards bytes behind a QUIC stream hole before the retransmission fills it, if a lossy-hop bench says it is worth it - [AF_XDP UDP path](/quest/m3/af-xdp.md) - the kernel-bypass verdict on today's virtio hosts that gates DPDK - [Unreal prototype](/quest/m3/unreal.md) - a UE5 module on the C++ package with exceptions disabled, rendering a subscribed broadcast to a texture - [Conan remote](/quest/m3/cpp-conan.md) - a remote we own serves the same tarball to `conan install` diff --git a/quest/m3/cut-through/README.md b/quest/m3/cut-through/README.md new file mode 100644 index 0000000000..7b85b846b5 --- /dev/null +++ b/quest/m3/cut-through/README.md @@ -0,0 +1,79 @@ +# Cut-through group streams + +## Goal + +A relay forwards group stream bytes that arrive past a QUIC stream hole +instead of holding them until the retransmission fills it. Payload after a +lost packet reaches the next hop at its own output offset while the hole is +still being recovered, so a loss on one hop no longer delays everything behind +it on the next. + +The line first measures whether this is worth it, then builds it only if the +measurement says so. A measured no-go deletes the build quests; the loss-delay +metric and the bench stay either way. + +Non-goals: end consumers (players, `moq-cli`) keep ordered reads, since a +decoder needs whole frames. No MoQ wire change; QUIC already allows a sender to +send stream offsets in any order. + +## Plan + +Decided 2026-10-02: the line moved from m2 to m3. It is a speculative +optimization with no consumer asking for it, so it parks until one does and +is deleted if it goes stale. The QUIC primitives land in `moq-quic` after +[the hard fork](/quest/m1/quic/fork/README.md) rather than in moq-dev/noq. + +Decided 2026-09-30: + +- Measure first, and plan the build now so the verdict only decides go or + no-go, not the design. The build quests require the [bench](/quest/m3/cut-through/bench.md). +- The gain is roughly the time the relay would spend bursting the held bytes + downstream after the hole fills: small when the egress hop has headroom, + larger for a big I-frame on a tight hop. The bench sweeps exactly that. +- A hole that covers a frame header blocks every later frame on that stream: + the next header's input position and every output header's length depend on + earlier header values (lite's timestamp delta, IETF's object id delta). A hole + inside a payload blocks nothing, since the frame sizes locate both the input + and the output bytes. Passing a hole into later frames needs several + in-flight frames per group; the relay quest prototypes both scopes and keeps + the one the bench justifies. +- The opportunity is measured on the real two-hop path, not against a direct + connection: the gain exists only with a relay in the middle. +- The frame model gains an offset write and a range read, crate-private in + `moq-net`, since the lite and IETF publishers and subscribers that use them + live in the same crate. Cut-through frames keep the received chunks by + offset instead of copying into a pre-allocated buffer, so memory tracks bytes + received and the per-session pre-allocation `Budget` (`model/frame.rs`) + does not apply. The group cache already charges each frame by bytes written + (#4609), so several in-flight frames charge only what has arrived. +- `web-transport-trait` gains an unordered chunk read and an offset write whose + defaults report unsupported, and moq-net falls back to ordered I/O. Only the + `moq-quic` backends implement them; browsers, qmux, and iroh keep the defaults. + Additive, so it lands on `main`. +- Always on wherever both of the relay's streams support it. No config knob. +- The relay's own egress is `moq-quic` for every native and browser viewer, so browser + viewers benefit too; only the relay's side needs the feature. + +Public API: additive `web-transport-trait` methods; the loss-delay counter on +`moq-stats` ingress rows. Wire: none for MoQ; the counter is a `moq-stats` +field. + +End to end: once [the relay quest](/quest/m3/cut-through/relay.md) lands, re-run the +bench on the same sweep and record before and after in this README before +closing the line. No new doc page: nothing user-facing changes beyond the +stats field, which its quest documents inline. + +## Required + +- [Loss delay](/quest/m3/cut-through/loss-delay.md) - relays report, per broadcast, the ingress bytes a loss held back by at least one RTT +- [Bench](/quest/m3/cut-through/bench.md) - a lossy relay hop swept over loss, frame size, and egress headroom, measuring the post-hole drain time, with a go or no-go verdict +- [Offset writes in moq-quic](/quest/m3/cut-through/quic.md) - send streams accept writes past a gap and send them right away +- [Frame ranges](/quest/m3/cut-through/frames.md) - frames fill out of order, several at once per group, and a reader sees each range as it lands +- [Transport trait](/quest/m3/cut-through/transport.md) - `web-transport-trait` carries unordered reads and offset writes, implemented for `moq-quic` +- [Relay cut-through](/quest/m3/cut-through/relay.md) - lite and IETF group streams read out of order and write each range at its output offset + +## Related + +- [Unordered qmux](/quest/m2/p2p/unordered.md) - the same head-of-line problem on the data channel transport +- [Hierarchical stream scheduling](/quest/m1/quic/scheduler.md) - orders streams; offset writes order ranges within one +- [QoS](/quest/m1/qos/README.md) - the loss-delay counter follows its per-broadcast ingress row conventions diff --git a/quest/m3/cut-through/bench.md b/quest/m3/cut-through/bench.md new file mode 100644 index 0000000000..bdbe9dc11f --- /dev/null +++ b/quest/m3/cut-through/bench.md @@ -0,0 +1,46 @@ +# [M] Benchmark head-of-line blocking at a relay hop + +## Goal + +A nightly bench measures how long a frame takes to complete at the viewer when +the publisher's hop to the relay loses packets, and how much of that a +cut-through relay could win back. It ends with a written go or no-go for the +rest of the [cut-through line](/quest/m3/cut-through/README.md). + +## Plan + +Decided 2026-09-30: real stacks, not a mock. The loss recovery being measured +belongs to the QUIC stack, so a mock transport with a made-up retransmit model would +partly measure its own assumptions. + +- Publisher, then a seeded `moq-shaper` (loss and delay), then `moq-relay` over + QUIC, then a clean or rate-limited hop to the viewer. Reuse the shaper wiring + from `rs/moq-relay/tests/drills.rs`. +- Sweep loss rate, frame size (audio-sized through a large I-frame), and egress + headroom (egress rate relative to the media rate). +- Report per-frame completion at the viewer, from publish to last byte, p50 and + p99. The opportunity is the drain time on this same two-hop path: per frame, + the time from the relay's ingress hole filling to the frame completing at the + viewer, minus the same for frames with no hole. That excess is the burst + cut-through removes. No direct publisher-to-viewer control: the gain exists + only with a relay in the middle, and one QUIC hop already reassembles out of + order, so a direct run measures a different topology (review of #4616). +- Split the held bytes three ways: bytes in the frame that has the hole, bytes + in later frames whose headers were known while the hole was open, and bytes + behind a lost header. Only the first two are forwardable; the last blocks with + or without cut-through and is reported but excluded from the opportunity. The + first split shows what [multiple in-flight frames](/quest/m3/cut-through/frames.md) + add beyond the current frame. Report the drain estimate as an upper bound, and + include both a payload-loss and a header-loss case to validate the split. +- Take publisher and subscriber counts as parameters, so + [relay cut-through](/quest/m3/cut-through/relay.md) reuses the rig for its + fan-out sweep. +- Record the loss-delay counter from the same runs, if + [Loss delay](/quest/m3/cut-through/loss-delay.md) has landed, so production + numbers can be read against the lab. +- It runs on wall-clock time, so it is a nightly bench, not a unit test. Wire it + into the nightly workflow. + +Write the numbers and the verdict into this line's README. The maintainer makes +the go or no-go call from them; no threshold is set in advance. On a no-go, +delete the build quests and keep the bench. diff --git a/quest/m3/cut-through/frames.md b/quest/m3/cut-through/frames.md new file mode 100644 index 0000000000..da2e6588ed --- /dev/null +++ b/quest/m3/cut-through/frames.md @@ -0,0 +1,42 @@ +# [L] Frames fill out of order + +## Goal + +A `moq-net` frame accepts payload ranges at any offset, a group can hold several +in-flight frames at once, and a reader can consume each range as soon as it +lands. Existing in-order readers and writers behave exactly as before. + +## Plan + +Decided 2026-09-30: + +- `frame::Producer` gains an offset write and `frame::Consumer` gains a read + that yields `(offset, Bytes)` ranges in arrival order. Both are + `pub(crate)`: the only users are the lite and IETF sessions in the same crate. +- A frame written out of order keeps the received chunks by offset instead of + copying into a pre-allocated buffer, so its memory tracks bytes received and + the per-session pre-allocation `Budget` (`model/frame.rs`) does not apply. + Each in-flight frame keeps the existing charge-by-bytes-written cache + accounting (`charge_partial` in `model/group.rs`), per frame. The + in-order readers (`poll_read_chunk`, `poll_read_all`) still see contiguous + bytes up to the first hole, and `finish` still requires every byte. +- Overlapping or out-of-bounds ranges are errors. +- A group holds several in-flight frames, not only the one `partial` in + `GroupState` (`model/group.rs`), and `create_frame_owned` stops rejecting a + second open frame. Frames are still created in order: frame N+1 exists only + once N's header, and so its size, is known, because that is what locates + N+1's header. The group consumer still yields frames in order. Decided + 2026-09-30 after review of #4616, so a hole in frame N does not hold back + N+1's bytes. +- Today's single sequential writer (`model/frame.rs`, the one `written` + watermark and its safety comments) is the invariant being changed; keep the + ordered fast path, including the whole-frame zero-copy install, unchanged. + +Extend `rs/moq-net/benches/group.rs` so the ordered path shows no regression, +and test that a frame filled in shuffled ranges reads back identical through +both readers, and that frame N+1's ranges reach a range reader while N still +has a hole. + +## Required + +- [Bench](/quest/m3/cut-through/bench.md) - a go verdict diff --git a/quest/m3/cut-through/loss-delay.md b/quest/m3/cut-through/loss-delay.md new file mode 100644 index 0000000000..fbf9337e40 --- /dev/null +++ b/quest/m3/cut-through/loss-delay.md @@ -0,0 +1,46 @@ +# [M] Report bytes delayed by loss at relay ingest + +## Goal + +The relay's `moq-stats` ingress rows carry, per broadcast, a cumulative +`loss_delay_bytes` counter: stream bytes that arrived behind a hole and stayed +unreadable for at least one smoothed RTT. Compared with bytes received, it +shows how much of a publisher's media a lossy uplink held back, and it is the +production number that sizes the cut-through opportunity. + +## Plan + +Decided 2026-09-30: one monotonic byte counter, no hold-time histogram, on the +ingress (`Role::Subscriber`) rows next to +[publisher timeliness](/quest/m1/qos/publisher-timeliness.md)'s fields. +Independent of the bench; it can start once `moq-quic` lands. + +- `moq-quic` counts per receive stream. The assembler stamps each chunk buffered past + the contiguous frontier with its arrival time; when the frontier passes it, + bytes held at least one smoothed RTT are counted. Plain reordering fills + within an RTT and is not counted. The count is by arrival and gap fill, not by + when the application reads, so it stays meaningful once reads go unordered. + When a stream ends with a hole still open (reset, stop, or session close), + buffered bytes that have already waited one smoothed RTT are counted then. +- `moq-quic` exposes the per-stream count, and `web-transport-trait`'s + `RecvStream` gains an accessor returning `Option` (`None` when the backend + cannot see it), following the trait's `Stats` convention. Release the trait + and implement it in the in-tree `web-transport-moq`, as + [poll_acked](/quest/m2/quic-ack-hook.md) does. +- The lite and IETF subscribers read the count at each frame boundary and when + the group stream ends, adding the delta to the broadcast's ingress row, so a + long group does not leave the row stale. Nothing on the per-byte path. +- Update the stats section of `doc/bin/relay/config.md`. + +Test: a seeded loss on a `moq-quic` pair counts the held bytes; a reorder shorter than +an RTT counts nothing. + +Public API: an additive trait accessor. Wire: a new `moq-stats` field. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the `moq-quic` assembler this counts in + +## Related + +- [QoS](/quest/m1/qos/README.md) - the per-broadcast ingress conventions this follows diff --git a/quest/m3/cut-through/quic.md b/quest/m3/cut-through/quic.md new file mode 100644 index 0000000000..9dc6c1009f --- /dev/null +++ b/quest/m3/cut-through/quic.md @@ -0,0 +1,32 @@ +# [L] Offset writes in moq-quic + +## Goal + +A send stream in `moq-quic` accepts a write at an offset past unwritten +bytes, sends it right away, and fills the gap when those bytes are written +later. + +## Plan + +Decided 2026-09-30: no write-ahead limit beyond QUIC's own flow control. + +- An offset write reserves the gap and queues the range. Stream and connection + credit is charged up to the highest written offset, as it already is on the + receiver's side. +- Within a stream, lower offsets are sent first once they are written, and + retransmits keep the existing stream priority path. +- `finish` waits until every range below the final size is written; a write + that overlaps an existing range is an error. +- The append-only `write` and `write_chunks` stay as they are; the offset write + is a new method, since a writer that never skips pays nothing. +- Tests in `moq-quic`'s simulated pair: an out-of-order writer delivers the + same bytes as an ordered one, flow control blocks at the highest offset, FIN + waits for the gap, and a reset with a gap outstanding is clean. + +The change lands in-tree with its consumer, per the +[QUIC line's rules](/quest/m1/quic/README.md); no fork release or pin bump. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - `moq-quic` exists and MoQ runs on it +- [Bench](/quest/m3/cut-through/bench.md) - a go verdict diff --git a/quest/m3/cut-through/relay.md b/quest/m3/cut-through/relay.md new file mode 100644 index 0000000000..f0ddee7649 --- /dev/null +++ b/quest/m3/cut-through/relay.md @@ -0,0 +1,56 @@ +# [L] Relay cut-through for group streams + +## Goal + +The lite and IETF sessions read incoming group streams out of order and write +each payload range to every subscriber's stream at its output offset, so bytes +behind an ingress hole reach the next hop before the hole is filled. On the +[bench](/quest/m3/cut-through/bench.md), cut-through shrinks the post-hole +drain time on the same two-hop path. + +## Plan + +Decided 2026-09-30: always on when the ingress and egress streams both support +it, no config knob; otherwise today's ordered path. + +- Ingest: read the stream header in order, then switch to unordered reads. + `Reader` may already hold bytes past the header in its buffer, so hand that + suffix over with its absolute stream offset rather than dropping or + re-basing it; test a transport chunk that carries both the header and the + first payload bytes. + Frame headers are parsed from the contiguous prefix; once a header is known, + its frame's payload ranges go into the frame with the + [offset write](/quest/m3/cut-through/frames.md) wherever they land. A hole + covering the next header blocks later frames, because that header's position + depends on it. Lite: `FrameIngest` in `lite/subscriber.rs`; IETF: + `GroupIngest` in `ietf/subscriber.rs`. `Reader::poll_read_frame` in + `coding/reader.rs` is the ordered path they share today. +- Egress: each subscriber's output offset for a payload range is its stream + prefix plus every earlier frame's encoded header and size, plus this frame's + header, plus the range offset. That is known once every earlier input header + has been parsed, including across a lite and IETF translation, since output + header lengths depend on values (timestamp and object id deltas). Write the + header, then the ranges with the + [offset write](/quest/m3/cut-through/transport.md). Lite: `GroupServe` in + `lite/publisher.rs`; IETF: `GroupServe` in `ietf/publisher.rs`. +- The fetch paths stay ordered. +- Tests over the mock transport with shuffled chunk delivery: every subscriber + version receives byte-identical streams to the ordered path, and a + header-covering hole holds later frames. + +Prototype both scopes on the bench: cut-through within the current frame only, +and across in-flight frames, where N+1's payload passes N's hole once N's +header is parsed. Keep the one the numbers justify and delete the other; a +regression test asserts N+1 is forwarded before N's hole fills when the +multi-frame scope ships. + +Sweep the bench over publisher and subscriber counts as well as the network +axes. Range bookkeeping, wakeups, and offset writes run per served +subscriber, so total CPU grows with the subscribers a range reaches; the cost +per served subscriber, and the cost from table entries a range does not touch, +must stay flat against the ordered path. Re-run it and record the result in the line's README. + +## Required + +- [Frame ranges](/quest/m3/cut-through/frames.md) - the model primitive +- [Transport trait](/quest/m3/cut-through/transport.md) - the I/O primitive diff --git a/quest/m3/cut-through/transport.md b/quest/m3/cut-through/transport.md new file mode 100644 index 0000000000..ec2962e3ef --- /dev/null +++ b/quest/m3/cut-through/transport.md @@ -0,0 +1,33 @@ +# [M] Unordered reads and offset writes in web-transport-trait + +## Goal + +A released `web-transport-trait` lets a receive stream yield chunks with their +offsets in arrival order and lets a send stream write at an offset. The +`moq-quic` backends implement both; every other backend reports that it cannot. + +## Plan + +Decided 2026-09-30: additive methods with defaults, not a moq-net-private +extension trait, which would need per-session detection behind the generic +`Session`. + +- `RecvStream` gains an unordered chunk read returning `(offset, Bytes)`, and + `SendStream` gains an offset write, both in the poll style. Their defaults + report unsupported in the return type (as + [poll_acked](/quest/m2/quic-ack-hook.md) does, since a default cannot build + `Self::Error`), and the caller falls back to ordered I/O. +- quinn's assembler, which `moq-quic` keeps, allows no ordered read after an + unordered one, so the unordered mode is entered once per stream and never + left. +- Implement both in the in-tree `web-transport-moq` over the unordered + `read_chunk` and the [offset write](/quest/m3/cut-through/quic.md), and + natively in moq-uring's QUIC stream (today `rs/moq-uring/src/quic/noq/stream.rs`, + renamed by the [switch](/quest/m1/quic/fork/switch.md)), which reads ordered + and drops the chunk offset. moq-tokio's poll adapter forwards them. + `web-transport-wasm`, qmux, and iroh keep the defaults. +- Release `web-transport-trait` and bump its pin here. + +## Required + +- [Offset writes in moq-quic](/quest/m3/cut-through/quic.md) - the send-side primitive this exposes