From af2edbcb326d51559cdbe4b63ee2ef0b42979adc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 30 Sep 2026 11:51:42 -0700 Subject: [PATCH 1/3] quest(quic): hard-fork quinn in-tree as moq-quic Replace the moq-dev/noq soft fork with an in-tree hard fork of quinn, delete multipath and the upstream quest, and plan endpoint sharding, moq-uring copy removal, and the io_uring I/O boundary redesign. Co-Authored-By: Claude Opus 5.5 --- quest/m1/README.md | 2 +- quest/m1/bbr-ack-cleanup.md | 4 ++ quest/m1/perf/README.md | 1 + quest/m1/perf/uring-copies.md | 31 +++++++++++ quest/m1/quic/README.md | 69 +++++++++++------------- quest/m1/quic/bbr-app-limited-edges.md | 4 ++ quest/m1/quic/bbr-app-limited.md | 11 ++-- quest/m1/quic/fork/README.md | 72 ++++++++++++++++++++++++++ quest/m1/quic/fork/bbr3.md | 27 ++++++++++ quest/m1/quic/fork/import.md | 20 +++++++ quest/m1/quic/fork/stream-slots.md | 20 +++++++ quest/m1/quic/fork/switch.md | 32 ++++++++++++ quest/m1/quic/peer-limits.md | 1 + quest/m1/quic/reliable-reset.md | 4 ++ quest/m1/quic/scheduler.md | 4 ++ quest/m1/quic/shard.md | 37 +++++++++++++ quest/m1/quic/upstream.md | 72 -------------------------- quest/m2/README.md | 2 +- quest/m2/multipath-spike.md | 64 ----------------------- quest/m2/quic-bbr-loss-parity.md | 4 -- quest/m2/quic-bbr-natural-drain.md | 1 - quest/m2/quic-deadline.md | 4 +- quest/m2/quic-io-boundary.md | 34 ++++++++++++ quest/m2/quic-keep-alive.md | 1 - 24 files changed, 329 insertions(+), 192 deletions(-) create mode 100644 quest/m1/perf/uring-copies.md create mode 100644 quest/m1/quic/fork/README.md create mode 100644 quest/m1/quic/fork/bbr3.md create mode 100644 quest/m1/quic/fork/import.md create mode 100644 quest/m1/quic/fork/stream-slots.md create mode 100644 quest/m1/quic/fork/switch.md create mode 100644 quest/m1/quic/shard.md delete mode 100644 quest/m1/quic/upstream.md delete mode 100644 quest/m2/multipath-spike.md create mode 100644 quest/m2/quic-io-boundary.md diff --git a/quest/m1/README.md b/quest/m1/README.md index 12f1f067c0..c74d353d0b 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -91,7 +91,7 @@ QUIC studies there on that rule. - [Audio rendition pick](/quest/m1/audio-ranked.md) - single-track FLV/RTMP and WHEP serve the best audio rendition, not the first by name - [FLV catalog stream](/quest/m1/flv-catalog-stream.md) - on dev, `flv::Export` takes a catalog stream like fmp4, replacing `with_select` - [io_uring flow control](/quest/m1/uring-flow-control-windows.md) - the relay's io_uring workers honor the QUIC flow-control windows instead of refusing them -- [Own the QUIC stack](/quest/m1/quic/README.md) - the moq-noq fork carries ACK progress, reliable reset, hierarchical scheduling, deadlines, peer limits, and qmux +- [Own the QUIC stack](/quest/m1/quic/README.md) - quinn hard-forked in-tree as `moq-quic`, carrying BBR, reliable reset, hierarchical scheduling, peer limits, and endpoint sharding - [QoS](/quest/m1/qos/README.md) - broadcast health: relay starvation and timeliness histograms, on dev - [Catalog track alias](/quest/m1/catalog-track-alias.md) - catalog rendition keys become aliases with an optional `track` name, so one catalog lists renditions from several broadcasts - [Media stats](/quest/m1/stats/README.md) - publishers announce a stats track in the catalog, viewers answer a soliciting catalog through a per-catalog `.echo` broadcast, and a Rust encoder adapts to them diff --git a/quest/m1/bbr-ack-cleanup.md b/quest/m1/bbr-ack-cleanup.md index bb0db79437..a0b9db7614 100644 --- a/quest/m1/bbr-ack-cleanup.md +++ b/quest/m1/bbr-ack-cleanup.md @@ -53,6 +53,10 @@ immutable fork release, and pin the corrected dependency chain here before completing this quest. Do not wait for the broader QUIC stack release. Update internal packet-lifetime comments inline; no new user guide is needed. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork + ## Related - [BBR starvation edges](/quest/m1/quic/bbr-app-limited-edges.md) - also edits `bbr3/mod.rs`; one owner there at a time diff --git a/quest/m1/perf/README.md b/quest/m1/perf/README.md index 4eb4d4ad0f..dc7aa2ce62 100644 --- a/quest/m1/perf/README.md +++ b/quest/m1/perf/README.md @@ -66,3 +66,4 @@ the rest. - [Ingest batch](/quest/m1/perf/ingest-batch.md) - relay ingest pays one lock, wake, and clock read per chunk burst instead of per chunk - [#3122](/quest/m1/perf/3122-moq-uring-2-5-of-relay-cpu-is-vdso-clock-reads-the-drive.md) - moq-uring: ~2.5% of relay CPU is vdso clock reads; the drive loop and its callers each re-read Instant::now() - [#3199](/quest/m1/perf/3199-moq-uring-remove-sq-indirection-and-per-enter-ring-fd.md) - moq-uring: remove SQ indirection and per-enter ring fd lookup +- [Remove moq-uring copies](/quest/m1/perf/uring-copies.md) - egress, stream send and receive, and datagram receive stop copying where the QUIC core already allows it, on `dev` after the fork diff --git a/quest/m1/perf/uring-copies.md b/quest/m1/perf/uring-copies.md new file mode 100644 index 0000000000..591596afc0 --- /dev/null +++ b/quest/m1/perf/uring-copies.md @@ -0,0 +1,31 @@ +# [S] Remove moq-uring's avoidable copies + +## Goal + +moq-uring stops copying and allocating on paths where the QUIC core already +supports zero-copy, with a measured before/after. + +## Plan + +Found in the 2026-09-30 fork planning; none needs a QUIC API change: + +- Egress: `quic/noq/connection.rs` copies each GSO train from `scratch` into + the TX buffer. Hand the TX pool's `Vec` to `poll_transmit` directly; it + encrypts in place and only reserves when capacity is short. +- Stream send: moq-net's `write_chunk(Bytes)` falls back to the trait's + default `poll_write_buf`, which copies through `poll_write(&[u8])`. Route + it to the core's `write_chunks` so large chunks are kept by reference. +- Stream receive: moq-uring uses the trait's default `poll_read_chunk`, which + allocates an 8 KiB `BytesMut` and copies each read. Return the core's + `Bytes` directly. +- Datagram receive: `endpoint.rs` allocates a `BytesMut` per GRO segment. + Copy once per completion and split it into segments. + +The clock-read fix stays in +[#3122](/quest/m1/perf/3122-moq-uring-2-5-of-relay-cpu-is-vdso-clock-reads-the-drive.md). +Measure with the echo benches and `just bench`. This branches from `dev`, +where the fork lands; paths above are pre-fork names. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the copies are removed on `moq-quic`, not the frozen fork diff --git a/quest/m1/quic/README.md b/quest/m1/quic/README.md index d09e79e75b..785326dc89 100644 --- a/quest/m1/quic/README.md +++ b/quest/m1/quic/README.md @@ -2,12 +2,12 @@ ## Goal -MoQ owns the QUIC features it needs. noq, the Quinn-derived stack n0 -maintains for Iroh, is the parent; the moq-dev fork carries what MoQ needs on -MoQ's schedule and offers it upstream when it is general. One core serves the -tokio backend, the thread-per-core `moq-uring` backend, iroh, and qmux. The -features are BBR correctness, reliable stream resets, hierarchical stream -scheduling with per-broadcast fairness, and wider limits for relay peers. +MoQ owns the QUIC features it needs in an in-tree stack, `moq-quic`, hard +forked from quinn (see [the fork](/quest/m1/quic/fork/README.md) for why +quinn and not noq). One core serves the tokio backend, the thread-per-core +`moq-uring` backend, and qmux; iroh stays on upstream noq. The features are +BBR correctness, reliable stream resets, hierarchical stream scheduling with +per-broadcast fairness, wider limits for relay peers, and endpoint sharding. Per-stream acknowledgment progress, per-stream deadlines, qmux on the shared stream state machine, and the experiments (GCC, receive timestamps, the egress profile, media probing, L4S, careful resume, deadline keep-alive) live in @@ -15,35 +15,29 @@ profile, media probing, L4S, careful resume, deadline keep-alive) live in ## Plan -Everything here ships from moq-dev/noq, the fork MoQ publishes as `moq-noq*`. -One stack carries every change on MoQ's own QUIC paths; a build with the `iroh` -feature also compiles upstream noq, and iroh connections are outside what these -quests reach. +Decided in the 2026-09-30 plan: the moq-dev/noq soft fork is replaced by a +hard fork of quinn in `rs/`, so every QUIC change here waits for +[the fork](/quest/m1/quic/fork/README.md) and lands on `dev`. Retarget this +line's PR (#3975) to `dev` when the fork starts. moq-dev/noq is frozen to +security patches for `main`. -The seven BBR correctness fixes shipped in moq-noq 1.3.1 (#4206). The -remaining BBR quests here and [BBR ACK cleanup](/quest/m1/bbr-ack-cleanup.md) -all edit `bbr3/mod.rs`, so one owner should work there at a time. -Controller-level regressions extend the shared test `Sim` in `bbr3/mod.rs` -with only what each needs, rather than adding another simulation loop; a fix -at the transport boundary still needs a transport test through the real -callbacks. The existing loops stay, since the fork merges upstream weekly and -a port would conflict. Each BBR fix ships in a fork patch release without -waiting for the remaining transport features. The comparison against -Google's BBR is part of the separate -[natural drain study](/quest/m2/quic-bbr-natural-drain.md). +Older quests say "the fork" or "noq"; read that as `moq-quic`. Their steps to +publish a fork release, pin it, or offer a change upstream are superseded: +a change lands in-tree with its consumer, and upstreaming is optional. -Rules the line keeps: +The seven BBR correctness fixes shipped in moq-noq 1.3.1 (#4206) and move to +`moq-quic` with the [BBR3 port](/quest/m1/quic/fork/bbr3.md). The remaining +BBR quests here and [BBR ACK cleanup](/quest/m1/bbr-ack-cleanup.md) all edit +`bbr3/mod.rs`, so one owner should work there at a time. Controller-level +regressions extend the shared test `Sim` in `bbr3/mod.rs` with only what each +needs, rather than adding another simulation loop; a fix at the transport +boundary still needs a transport test through the real callbacks. The +comparison against Google's BBR is part of the separate +[natural drain study](/quest/m2/quic-bbr-natural-drain.md). -- a carried change lists its upstream PR or the reason it has none; -- the fork's packages are `moq-noq-proto`, `moq-noq`, and `moq-noq-udp`, - never a crate that impersonates the parent; -- published MoQ crates depend on crates.io releases of the fork, never a - workspace-only Cargo patch or a mutable branch; -- each feature quest releases the fork crate it changes and records the - parent noq commit it is based on; there is no separate release quest; -- MoQ's config names congestion families (`Loss`, `Delay`, and `RealTime` - once GCC ships), never algorithms; noq's public `Controller` trait is the - seam experiments plug into, and MoQ owns which algorithm each family means. +MoQ's config names congestion families (`Loss`, `Delay`, and `RealTime` once +GCC ships), never algorithms; the `Controller` trait is the seam experiments +plug into, and MoQ owns which algorithm each family means. The scheduling contract has three levels: strict subscription priority, byte-fair service between send groups at the same priority, then the @@ -54,12 +48,11 @@ This is a transport API change, not a MoQ wire change. Decided in the 2026-09-30 audit: deadlines, qmux, BBR loss parity, ECN measurement, ACK progress, and the ACK hook moved to m2, since no m1 quest -consumes them. The release quest was deleted: the fork already publishes to -crates.io (moq-noq 1.3.2, web-transport-moq 2.0.0) with no workspace patch, -and the qmux crate release folds into [qmux](/quest/m2/quic-qmux.md). +consumes them. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - quinn in-tree as `moq-quic`, with BBR3 and lazy stream slots, and MoQ switched onto it - [BBR idle burst](/quest/m1/quic/bbr-app-limited.md) - a fork regression proves a burst after a long idle is paced at the learned bandwidth, closing #4219 - [Mark BBR starvation wherever the source runs dry](/quest/m1/quic/bbr-app-limited-edges.md) - partial polls count, local send caps do not, receiver credit is pinned - [Deliver the application close before io_uring teardown](/quest/m1/quic/uring-close.md) - @@ -71,8 +64,8 @@ and the qmux crate release folds into [qmux](/quest/m2/quic-qmux.md). the lossy scalar; retransmits follow the same order - [Relay peers get wider limits](/quest/m1/quic/peer-limits.md) - MAX_STREAMS and MAX_DATA are raised after SETUP identifies a cluster peer -- [Upstream the fork](/quest/m1/quic/upstream.md) - every general carried - change is offered to n0-computer/noq once its shape has settled +- [Shard the endpoint](/quest/m1/quic/shard.md) - the library shards a + server endpoint across cores by default, replacing the `--workers` group ## Related @@ -80,8 +73,6 @@ and the qmux crate release folds into [qmux](/quest/m2/quic-qmux.md). per-broadcast fairness policy on cluster sessions - [Starvation](/quest/m1/qos/starvation.md) - the first consumer of ACK progress: how far behind viewers are, from the relay's point of view -- [Multipath spike](/quest/m2/multipath-spike.md) - a noq capability that - MoQ does not use yet - [Discover media headroom](/quest/m2/quic-probe.md) - test useful-media pacing before adding redundant probe traffic - [L4S on the backbone](/quest/m2/quic-ecn.md) - an ECT(1) option in the fork, an `ecn` config knob, and a dualpi2 measurement - [Careful resume on reconnect](/quest/m2/quic-careful-resume.md) - a redial starts at the previous connection's rate diff --git a/quest/m1/quic/bbr-app-limited-edges.md b/quest/m1/quic/bbr-app-limited-edges.md index fc47d4d255..fdcc2b07b2 100644 --- a/quest/m1/quic/bbr-app-limited-edges.md +++ b/quest/m1/quic/bbr-app-limited-edges.md @@ -32,6 +32,10 @@ drains, a sender blocked only by `send_window`, and a stream blocked by receiver credit. Builds on the seven fixes released in moq-noq 1.3.1. No public API or wire change is intended. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork + ## Related - [BBR idle burst](/quest/m1/quic/bbr-app-limited.md) - the first app-limited fix this extends diff --git a/quest/m1/quic/bbr-app-limited.md b/quest/m1/quic/bbr-app-limited.md index f0ea70c90c..47c10053ad 100644 --- a/quest/m1/quic/bbr-app-limited.md +++ b/quest/m1/quic/bbr-app-limited.md @@ -27,13 +27,12 @@ minutes, send 250 KB, and assert the pacing rate stays at or above about If it still stalls, find the remaining cause, such as ProbeRTT entered during the idle or a stale `bw_shortterm`, and fix it in the fork. Dropping the estimate after a long idle is a policy change for the m2 study, not this -quest. The `iroh` feature uses upstream noq, which lacks #5; offering it -there belongs to the upstream quest. +quest. -## Closes +## Required -- [#4219](https://github.com/moq-dev/moq/issues/4219) - the first send after an idle period is paced at a trickle +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork -## Related +## Closes -- [Upstream the fork](/quest/m1/quic/upstream.md) - offer the starvation fix upstream +- [#4219](https://github.com/moq-dev/moq/issues/4219) - the first send after an idle period is paced at a trickle diff --git a/quest/m1/quic/fork/README.md b/quest/m1/quic/fork/README.md new file mode 100644 index 0000000000..c016b0574e --- /dev/null +++ b/quest/m1/quic/fork/README.md @@ -0,0 +1,72 @@ +# Hard-fork quinn into the monorepo as moq-quic + +## Goal + +MoQ's QUIC stack lives in `rs/` as `moq-quic` (the sans-IO core), +`moq-quic-udp`, `moq-quic-tokio`, and `web-transport-moq`, forked from +quinn-rs/quinn `main`. Every MoQ QUIC path (moq-tokio and moq-uring) runs on +it, the `moq-noq*` dependencies are gone, and a QUIC change lands in the same +PR as the MoQ code that needs it. + +Out of scope: iroh keeps upstream noq (the `iroh` feature still compiles it for +iroh's controller factory), and multipath, NAT traversal, and QAD are not +carried. + +## Plan + +Decided in the 2026-09-30 plan, with reasons, so later sessions don't reopen +them: + +- **Hard fork, no merges.** The moq-dev/noq soft fork never merged upstream + (its weekly sync job failed on a workflow-permission error), n0-computer/noq + slowed to 5-7 commits a month in Aug-Sep, and n0's strict semver limits + what we can upstream, not what a fork can change. Upstream fixes are + cherry-picked by hand; offering our changes upstream is optional courtesy, + not tracked work. +- **Base on quinn, not noq.** quinn-proto is 32.9k lines against noq-proto's + 58.5k, about 15k of which is multipath, NAT traversal, and QAD that MoQ + never negotiates. Stripping those from noq was estimated at 2-3 weeks with + loss-recovery risk, and would leave every quinn fix a hand port. quinn ships + security fixes the day the advisory publishes (11 in 2026), and staying + close to its code keeps those cherry-picks cheap. The cost is re-porting + BBR3, the controller callbacks, and lazy stream slots, which is why MoQ + moved to noq in the first place (#1706, #3342, #3789). +- **Multipath is deleted**, and with it the multipath spike. +- **In-tree**, so `just check` and CI cover the stack, one PR spans the core + and its consumers, and `main` and `dev` each carry their own copy instead of + double-landing fixes on the 1.3 and 2.0 fork lines. +- **Named by role**: `moq-quic`, `moq-quic-udp`, `moq-quic-tokio`; + `web-transport-moq` keeps its name. Each crate's README and license credit + quinn and noq. +- **Lands on `dev`.** moq-tokio exposes `noq::Endpoint`, + `noq::TransportConfig`, and `noq::Incoming` publicly, so the crate swap is + a break. +- **Security folds into the fork.** Rebasing on quinn picks up the 2026 + advisories noq is missing; `main` stays on `moq-noq` until `dev` releases. + [noq reassembly cap](/quest/m0/noq-reassembly-cap.md) still finishes on the + old fork for `main`. +- **moq-dev/noq is frozen**: security patches for `main` only, archived once + no released MoQ crate depends on it. The other + [QUIC quests](/quest/m1/quic/README.md) wait for this line and land + in-tree on `dev`. + +This README's own work: delete moq-dev/noq's `moq-sync.yml`, mark its README +frozen, and document advisory triage in `rs/moq-quic/README.md`. `cargo audit` +cannot match renamed crates, so the triage is: watch quinn-rs/quinn's security +advisories and releases, check each fix against `moq-quic`, and port it with +its regression test. Every carried change in moq-dev/noq's `CHANGELOG-MOQ.md` +is either ported by a child quest or recorded as not applicable in the +[switch](/quest/m1/quic/fork/switch.md) PR. + +## Required + +- [Import quinn](/quest/m1/quic/fork/import.md) - quinn's three crates build and test in-tree as `moq-quic*`, verbatim at a recorded commit, with no consumer yet +- [Port BBR3](/quest/m1/quic/fork/bbr3.md) - the fork's corrected BBR3 and controller callbacks run on `moq-quic` as the default controller +- [Lazy stream slots](/quest/m1/quic/fork/stream-slots.md) - relay memory on `moq-quic` matches `moq-noq` +- [Switch](/quest/m1/quic/fork/switch.md) - `web-transport-moq`, moq-tokio, and moq-uring run on `moq-quic`, and `moq-noq*` is gone + +## Related + +- [Shard the endpoint](/quest/m1/quic/shard.md) - the first feature the fork makes possible +- [Remove moq-uring copies](/quest/m1/perf/uring-copies.md) - cheap copy removals on the new core +- [Redesign the QUIC I/O boundary](/quest/m2/quic-io-boundary.md) - the breaking perf rewrite, gated on a profile diff --git a/quest/m1/quic/fork/bbr3.md b/quest/m1/quic/fork/bbr3.md new file mode 100644 index 0000000000..727836ea44 --- /dev/null +++ b/quest/m1/quic/fork/bbr3.md @@ -0,0 +1,27 @@ +# [L] Port BBR3 and the controller callbacks + +## Goal + +`moq-quic` carries the BBR3 controller from moq-dev/noq, with every +correctness fix the fork has shipped, and it is the default controller. The +fork's BBR regressions pass on `moq-quic`. + +## Plan + +Move moq-dev/noq's `congestion/bbr3/` over as a unit rather than starting from +quinn#2481: the fork's copy already descends from #2481 and carries the seven +BBR fixes (#4206), the app-limited fix, and the classic ECN CE response +(moq-dev/noq#12). Port the `Controller` extensions it depends on (`PacketId` +and packet-space callbacks, app-limited, cwnd-limited, ACK frequency) by hand; +they are about 350 lines in `connection/mod.rs` and `pacing.rs`. + +The fork's roughly 580 lines of regression tests are written against noq's +test harness; rework them onto quinn's rather than porting the harness. Keep +MoQ's config naming controller families (`Loss`, `Delay`), never algorithms. + +Verify with the fork's BBR tests and the benchmark matrix against `moq-noq` +1.3.x on the same workloads; report any throughput or latency difference. + +## Required + +- [Import quinn](/quest/m1/quic/fork/import.md) diff --git a/quest/m1/quic/fork/import.md b/quest/m1/quic/fork/import.md new file mode 100644 index 0000000000..56179975f9 --- /dev/null +++ b/quest/m1/quic/fork/import.md @@ -0,0 +1,20 @@ +# [M] Import quinn as moq-quic + +## Goal + +`rs/moq-quic`, `rs/moq-quic-udp`, and `rs/moq-quic-tokio` are quinn-proto, +quinn-udp, and quinn from quinn-rs/quinn `main`, and they build and pass their +own tests in `just check` and CI. Nothing consumes them yet. + +## Plan + +Keep the first commit verbatim at a recorded quinn commit so a reviewer can +diff it against upstream; renames, workspace lints, and edition fixes go in +separate commits. Record the parent commit in `rs/moq-quic/README.md` with the +advisory triage process the [questline](/quest/m1/quic/fork/README.md) +describes. + +Keep quinn's own features (qlog, rustls providers, runtimes) as they are; +pruning waits until the [switch](/quest/m1/quic/fork/switch.md) shows what MoQ +uses. Wire the crates into release-plz like the other `rs/` crates, and check +that the added test time fits CI. diff --git a/quest/m1/quic/fork/stream-slots.md b/quest/m1/quic/fork/stream-slots.md new file mode 100644 index 0000000000..4b541b4623 --- /dev/null +++ b/quest/m1/quic/fork/stream-slots.md @@ -0,0 +1,20 @@ +# [S] Lazy stream slots on moq-quic + +## Goal + +`moq-quic` allocates stream state lazily, so relay memory on it matches +`moq-noq` on the bulk and fanout workloads. + +## Plan + +Port quinn#2601 (the same change noq took as noq#667). #3342 measured the +difference it makes: 75 vs 141 MiB relay memory on bulk and 31 vs 97 MiB on +fanout. Re-run those workloads on `moq-quic` with and without the change. + +Check the port against quinn's 2026 MAX_STREAM_DATA advisory fix: noq's lazy +stream code was not exploitable there because it checks the limit itself, so +make sure the combined code keeps exactly one check. + +## Required + +- [Import quinn](/quest/m1/quic/fork/import.md) diff --git a/quest/m1/quic/fork/switch.md b/quest/m1/quic/fork/switch.md new file mode 100644 index 0000000000..e69eedff6b --- /dev/null +++ b/quest/m1/quic/fork/switch.md @@ -0,0 +1,32 @@ +# [M] Switch MoQ onto moq-quic + +## Goal + +`web-transport-moq` lives in `rs/` on `moq-quic-tokio`, moq-tokio and +moq-uring run on `moq-quic`, and no workspace crate depends on `moq-noq*`. +Upstream `noq-proto` remains only for the `iroh` feature. + +## Plan + +Move `web-transport-moq` from moq-dev/noq. It derives from +web-transport-quinn; its noq-only parts are `PathId::ZERO` and `path_stats`. +Report bandwidth from quinn's `PathStats::bandwidth_estimate` instead of the +current cwnd/rtt guess. + +moq-uring built against both quinn-proto and noq-proto until #3811, behind +about 20 `cfg` lines (stats fields, CID generator, qlog, BBR). Rename +`rs/moq-uring/src/quic/noq` by role and use that history as the map. Keep +moq-uring's `qlog` feature working on quinn's qlog. + +Rename the `noq` cargo features by role (`quic` is the recommendation); +confirm the name with the maintainer in the PR. Update every doc and example +that names noq, and run `just test interop --all`. + +In the PR, list each carried change from moq-dev/noq's `CHANGELOG-MOQ.md` as +ported (with its quest) or not applicable (with the reason). + +## Required + +- [Import quinn](/quest/m1/quic/fork/import.md) +- [Port BBR3](/quest/m1/quic/fork/bbr3.md) +- [Lazy stream slots](/quest/m1/quic/fork/stream-slots.md) diff --git a/quest/m1/quic/peer-limits.md b/quest/m1/quic/peer-limits.md index 7c36f74bc2..2646f5a6ff 100644 --- a/quest/m1/quic/peer-limits.md +++ b/quest/m1/quic/peer-limits.md @@ -32,6 +32,7 @@ viewer session does not; the io_uring path applies the same values; a ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork - [io_uring flow control](/quest/m1/uring-flow-control-windows.md) - the io_uring workers hardcode their windows today, so the peer values have nothing to raise there until the static windows reach them diff --git a/quest/m1/quic/reliable-reset.md b/quest/m1/quic/reliable-reset.md index 64becb43ca..5be1e64a0d 100644 --- a/quest/m1/quic/reliable-reset.md +++ b/quest/m1/quic/reliable-reset.md @@ -64,6 +64,10 @@ Track the unversioned draft during implementation. The planning baseline is draft 10, with transport parameter `0x1d` and frame type `0x24`; do not freeze provisional codepoints if the document changes before release. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork + ## Related - moq-net (`rs/moq-net/src/tail.rs`) and `@moq/net` (`js/net/src/tail.ts`) wait diff --git a/quest/m1/quic/scheduler.md b/quest/m1/quic/scheduler.md index c28b33fb3a..0f564298c2 100644 --- a/quest/m1/quic/scheduler.md +++ b/quest/m1/quic/scheduler.md @@ -102,6 +102,10 @@ scheduler completion must not wait for that dependent integration. Preserve working behavior on backends not yet migrated, and remove queue code only where the new implementation makes it redundant. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - lands in `moq-quic` on `dev`, not the frozen fork + ## Closes - [#699](https://github.com/moq-dev/moq/issues/699) - close this issue when the diff --git a/quest/m1/quic/shard.md b/quest/m1/quic/shard.md new file mode 100644 index 0000000000..ac42b48802 --- /dev/null +++ b/quest/m1/quic/shard.md @@ -0,0 +1,37 @@ +# [M] Shard the QUIC endpoint across cores + +## Goal + +A `moq-quic` server endpoint shards across cores by default: the library +forms the steered `SO_REUSEPORT` group and issues connection IDs that name +their shard, and each runtime (moq-tokio and moq-uring) only drives the +shards it is handed on threads of its own. The relay's `--workers` flag +becomes an optional count override, and the two copies of the group logic +are gone. + +## Plan + +Decided in the 2026-09-30 plan: the library shards and the runtime drives. +The library does not own threads, so embedders (libmoq, moq-ffi) keep their +own threading model. + +- `moq-quic-udp` absorbs `moq-sock::shard` (the group, claims, and steering + filter); `moq-sock` keeps binding and CPU helpers. +- `moq-quic` issues shard-prefixed connection IDs itself, replacing the + custom CID generators in moq-tokio and moq-uring. +- moq-tokio's `worker/group.rs` shrinks to spawning one pinned + `current_thread` runtime per shard; moq-uring's `udp::Bound` group code is + replaced by the library's. +- A server endpoint defaults to one shard per core. Measure the default + against a single socket with `just bench` before flipping it, since the + relay's default changes. + +Update `doc/bin/relay/` for the flag change. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the stack has to be in-tree before the library can own sharding + +## Related + +- [UDP demux](/quest/m2/one-port/udp-demux.md) - demuxes each shard's socket; keep the demux over whatever the library hands out diff --git a/quest/m1/quic/upstream.md b/quest/m1/quic/upstream.md deleted file mode 100644 index 796f087fd9..0000000000 --- a/quest/m1/quic/upstream.md +++ /dev/null @@ -1,72 +0,0 @@ -# [M] Upstream the fork - -## Goal - -Every general change the fork carries has been offered to n0-computer/noq as -a reviewable pull request, and the fork's carried set is only what upstream -declined or what is MoQ-specific. The noq maintainers have seen this roadmap -before the first proposal lands, so nothing arrives as a surprise. - -## Plan - -Fork now, upstream opportunistically: this quest is the "opportunistically" -half, and it completes the line rather than gating it. Each feature quest -lands in the fork on MoQ's schedule; once a feature has shipped in a MoQ -release and its shape has stopped moving, split it into an upstream PR with -the tests it landed with. - -Offer the seven BBR correctness fixes (moq-noq 1.3.1, #4206) and -their [loss](/quest/m2/quic-bbr-loss-parity.md) and -[starvation](/quest/m1/quic/bbr-app-limited-edges.md) follow-ups with their -regressions before promoting BBR as the default. Reuse existing -upstream work, particularly [PR #802](https://github.com/n0-computer/noq/pull/802), -and preserve attribution; the fork found that #802's `has_sample()` check -fires one ACK late under noq's callback order and that its 1-ms floor keeps -sub-millisecond paths at the placeholder rate, so offer both back there. The -additive `PacketId` callbacks answer upstream's TODO on the `Controller` -impl. Fixes can be offered as their shapes settle; upstream acceptance never -gates the fork's corrected release. - -Then the feature proposal order, each linked to its producing quest: - -0. BBR3 as `TransportConfig`'s default controller (noq-proto 1.2.0 defaults - to Cubic, `config/transport.rs:586`, while MoQ has run BBR3 by default - since it adopted noq): a one-line change whose point is that every noq - and iroh user, the maintainers included, runs the controller MoQ depends - on, so its regressions are found upstream and not only here; -1. per-stream acknowledgment progress ([ACK progress](/quest/m2/quic-ack-progress.md)); -2. `RESET_STREAM_AT` ([reliable reset](/quest/m1/quic/reliable-reset.md)); -3. hierarchical send groups ([scheduler](/quest/m1/quic/scheduler.md)); -4. per-stream deadlines ([deadlines](/quest/m2/quic-deadline.md)); -5. the qmux crate over the shared stream state machine ([qmux](/quest/m2/quic-qmux.md)). - -Items 1, 4, and 5 moved to m2 in the 2026-09-30 audit; like the other m2 -features they are offered when they land and do not gate this quest. The -other m2 features (keep-alive by deadline, careful resume as a `Controller` -wrapper, ECT(1) marking, the media-headroom mechanism) are offered when they -land, but do not gate this quest: an m1 quest must not wait on m2 work. The -next experiments (receive timestamps, GCC, the egress profile, the -natural-drain check) join the list only with a -positive verdict. - -Record in this quest what upstream accepted, what it asked to see as an -extension crate, and what it declined; a declined change stays in the fork -with the link beside it. The quest completes when the m1 items above have -been offered and answered. - -## Required - -- [Mark BBR starvation wherever the source runs dry](/quest/m1/quic/bbr-app-limited-edges.md) -- [Reliable stream reset](/quest/m1/quic/reliable-reset.md) -- [Hierarchical stream scheduling](/quest/m1/quic/scheduler.md) - -## Related - -- [Keep-alive by deadline](/quest/m2/quic-keep-alive.md), - [Careful resume on reconnect](/quest/m2/quic-careful-resume.md), - [L4S on the backbone](/quest/m2/quic-ecn.md), - [Discover media headroom](/quest/m2/quic-probe.md) - m2 features offered - upstream when they land -- [BBR media study](/quest/m2/quic-bbr-natural-drain.md), - [Receive timestamps](/quest/m3/quic-receive-ts.md), [GCC](/quest/m3/quic-gcc.md) - - experiments that join the list with a positive verdict diff --git a/quest/m2/README.md b/quest/m2/README.md index f79539a25f..8d695ae09c 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -88,7 +88,7 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [vcpkg registry](/quest/m2/cpp-vcpkg.md) - a registry we own serves the prebuilt package to `vcpkg` manifests - [Binary delta stats](/quest/m2/stats-delta.md) - an on-demand varint delta flavor of every stats track, if relay encode CPU still matters after the JSON fixes - [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) - moqsink: a flushing restart after EOS opens a new publication generation -- [Multipath spike](/quest/m2/multipath-spike.md) - whether bonded contribution over multipath QUIC is worth building, given it needs noq on both ends +- [QUIC I/O boundary](/quest/m2/quic-io-boundary.md) - moq-uring receives from the buffer ring and transmits into registered buffers with no copy, once a profile says where - [BBR media study](/quest/m2/quic-bbr-natural-drain.md) - whether bounded drain credit avoids ProbeRTT deadline interference, and where our BBR differs from Google's - [Discover media headroom](/quest/m2/quic-probe.md) - test useful-media pacing before adding redundant probe traffic - [L4S on the backbone](/quest/m2/quic-ecn.md) - an ECT(1) option in the fork, an `ecn` config knob, and a dualpi2 measurement diff --git a/quest/m2/multipath-spike.md b/quest/m2/multipath-spike.md deleted file mode 100644 index 6e7646f68b..0000000000 --- a/quest/m2/multipath-spike.md +++ /dev/null @@ -1,64 +0,0 @@ -# [M] Multipath spike - -## Goal - -A verdict on whether bonded contribution over multipath QUIC is worth -building, backed by two paths actually carrying one session. Bonding means -sending a broadcast over several links at once (cellular plus cellular plus -wifi) so a field encoder survives any one of them degrading, which is the -business LiveU and Peplink built on proprietary tech and SRT has no answer -for. - -## Plan - -The protocol work is done and unused. `noq` implements draft-ietf-quic-multipath -in full (`open_path`, `path_events`, `path_stats`, a `Path` type, and a -substantial test suite), but `apply_transport` never calls -`max_concurrent_multipath_paths`, whose default is `None`, so multipath is not -negotiated on any MoQ connection. - -The spike is small: call that setter, add a max-paths knob beside the other -transport settings, and open a second path from the connect side. It rides -entirely below the `web_transport_trait` line, because -`web_transport_moq::Session` derefs to `noq::Connection` and the trait requires -`Clone`, so the backend can hand one clone to moq-net and keep another to -drive paths on. No trait change, no moq-net change, and none belongs there: a -browser WebTransport backend could never implement it. - -**What the spike is really for is the constraint nobody has written down.** -`max_concurrent_multipath_paths` maps to the `initial_max_path_id` transport -parameter, which both peers negotiate, so a multipath session needs noq on -*both* ends. noq is the only native QUIC backend, while browser and qmux -transports have no such thing. So bonded contribution is not an opt-in client feature; it -constrains every relay and client on the path to run noq. Establish what that -actually costs before anyone builds a fan-out socket. - -Prove it on a multi-homed host, which is enough to negotiate and open a second -path and is not yet phone bonding. - -Then record what the rest would take, so a go verdict has something to plan -from and a no-go says why: - -- **One endpoint is one UDP socket.** `FourTuple` carries `local_ip` only, - because "we can only specify the `src_ip`, not the source port", and setting - a source IP does not choose an egress interface. Android needs - `Network.bindSocket` and iOS needs an interface bind to reach the cellular - radio at all. The escape hatch is `Endpoint::new_with_abstract_socket` with a - fan-out `AsyncUdpSocket` owning one interface-bound socket per radio, which - then has to answer for `local_addr`, and take the conservative minimum for - `max_receive_segments` and `may_fragment`. -- **Stats are single-path shaped.** `ConnectionStats` carries one `rtt` and one - `estimated_send_rate`, and per-path scheduling for contribution would need - both that and `web_transport_trait::Stats` reshaped. -- **Scheduling policy is unchosen**: duplicate every path for redundancy, or - stripe across them for capacity. - -Naming trap: `Path` in moq-net is the broadcast namespace path, and -`Client::with_path` is the MoQ SETUP resource path. A network path needs a -different name. - -Target `main`: `apply_transport` lives in `rs/moq-tokio/src/noq.rs` there -too, and the max-paths knob is additive. - -Public API: an additive transport setting in `moq-tokio`. Wire: none; multipath -is negotiated by QUIC transport parameters, below MoQ. diff --git a/quest/m2/quic-bbr-loss-parity.md b/quest/m2/quic-bbr-loss-parity.md index 5a2cb439a6..3b3c877d5e 100644 --- a/quest/m2/quic-bbr-loss-parity.md +++ b/quest/m2/quic-bbr-loss-parity.md @@ -34,7 +34,3 @@ no `Controller` or wire change. Decided in the 2026-09-30 audit: moved to m2. The seven correctness fixes already shipped, and this remaining gap has no observed impact on MoQ traffic yet. - -## Related - -- [Upstream the fork](/quest/m1/quic/upstream.md) - offers this fix alongside the seven diff --git a/quest/m2/quic-bbr-natural-drain.md b/quest/m2/quic-bbr-natural-drain.md index 5ca6de5d3b..77bdd93c03 100644 --- a/quest/m2/quic-bbr-natural-drain.md +++ b/quest/m2/quic-bbr-natural-drain.md @@ -77,5 +77,4 @@ is not an end-to-end network measurement. ## Related -- [Upstream the fork](/quest/m1/quic/upstream.md) - share useful findings with upstream - [Discover media headroom](/quest/m2/quic-probe.md) - preserving an estimate and discovering spare capacity are separate problems diff --git a/quest/m2/quic-deadline.md b/quest/m2/quic-deadline.md index 008fcbf9c2..d17acb6f9c 100644 --- a/quest/m2/quic-deadline.md +++ b/quest/m2/quic-deadline.md @@ -42,9 +42,7 @@ retransmitted past their deadline (must be zero), spurious resets under reordering, and probe overhead versus the default PTO. A proactive probe that raises loss or latency under any profile stays off by default. -Decided in the 2026-09-30 audit: moved to m2. No m1 quest consumes it, and -[Upstream the fork](/quest/m1/quic/upstream.md) offers it when it lands -rather than waiting on it. +Decided in the 2026-09-30 audit: moved to m2. No m1 quest consumes it. ## Required diff --git a/quest/m2/quic-io-boundary.md b/quest/m2/quic-io-boundary.md new file mode 100644 index 0000000000..69b8beef50 --- /dev/null +++ b/quest/m2/quic-io-boundary.md @@ -0,0 +1,34 @@ +# [L] Redesign the QUIC I/O boundary around io_uring + +## Goal + +`moq-quic`'s sans-IO boundary lets moq-uring receive from the kernel's +buffer ring and transmit into registered buffers, with no copy between the +kernel and the crypto. The API may break; it is in-tree. + +## Plan + +Decided in the 2026-09-30 plan: commit to the redesign, but start only after +a noq or `moq-quic` profile shows where ingress and egress spend their time. +No current profile is from noq, and every other copy is removed without an +API change by [Remove moq-uring copies](/quest/m1/perf/uring-copies.md). + +Candidates, to be ranked by the profile: + +- `Endpoint::handle` accepts a foreign-owned buffer (a buffer-ring slot), and + stream frames keep references to it instead of requiring an owned + `BytesMut`. +- `poll_transmit` writes into a caller slice or registered buffer as a + contract, not by relying on `Vec` capacity. +- One `handle` call takes a whole GRO batch. + +Registered TX buffers ([#3204](/quest/m3/3204-moq-uring-register-tx-pool-buffers-for-zero-copy-sends.md)) +and `SENDMSG_ZC` ([#3201](/quest/m3/3201-moq-uring-use-sendmsg-zc-for-large-udp-gso-trains.md)) +build on the transmit contract. Report relay CPU per Gbps before and after on +the same workloads. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the redesign breaks `moq-quic`'s API +- [Relay profiles](/quest/m1/performance-profiles.md) - the redesign targets measured costs +- [Remove moq-uring copies](/quest/m1/perf/uring-copies.md) - the cheap wins land first, so the profile shows what is left diff --git a/quest/m2/quic-keep-alive.md b/quest/m2/quic-keep-alive.md index 7c7fc08512..f9943f8ccb 100644 --- a/quest/m2/quic-keep-alive.md +++ b/quest/m2/quic-keep-alive.md @@ -24,7 +24,6 @@ timeout by hand. Replace it in the fork: - Keep `keep_alive_interval` as a maximum: `None` means "as late as possible"; a value means "no later than this", for a NAT binding with a shorter life than the idle timeout. That is the only knob. -- The multipath per-path keep-alive follows the same rule per path. moq-tokio's `quic.keep_alive` (`rs/moq-tokio/src/quic.rs`) in the `[quic]` sections, CLI flags, and env vars becomes that optional maximum, default none, From 0dffe49e61146502519d25e4d2fbc4232f56e26a Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 30 Sep 2026 12:09:51 -0700 Subject: [PATCH 2/3] quest(quic): retarget fork-release quests in-tree and scope the shard default Co-Authored-By: Claude Opus 5.5 --- quest/m1/quic/shard.md | 16 ++++++++++++---- quest/m2/README.md | 4 ++-- quest/m2/noq-socket-close.md | 13 ++++++++----- quest/m2/quic-ack-hook.md | 18 ++++++------------ quest/m2/quic-ack-progress.md | 20 ++++++++++---------- quest/m2/quic-bbr-loss-parity.md | 4 ++++ quest/m2/quic-bbr-natural-drain.md | 4 ++++ quest/m2/quic-careful-resume.md | 4 ++++ quest/m2/quic-deadline.md | 1 + quest/m2/quic-ecn-measure.md | 4 ++++ quest/m2/quic-ecn.md | 1 + quest/m2/quic-egress-profile.md | 4 ++++ quest/m2/quic-keep-alive.md | 4 ++++ quest/m2/quic-probe.md | 4 ++++ quest/m2/quic-qmux.md | 17 +++++++---------- quest/m3/quic-gcc.md | 1 + quest/m3/quic-receive-ts.md | 1 + 17 files changed, 77 insertions(+), 43 deletions(-) diff --git a/quest/m1/quic/shard.md b/quest/m1/quic/shard.md index ac42b48802..b62bd34522 100644 --- a/quest/m1/quic/shard.md +++ b/quest/m1/quic/shard.md @@ -2,7 +2,8 @@ ## Goal -A `moq-quic` server endpoint shards across cores by default: the library +A `moq-quic` server endpoint shards across cores by default where the +platform can steer (Linux): the library forms the steered `SO_REUSEPORT` group and issues connection IDs that name their shard, and each runtime (moq-tokio and moq-uring) only drives the shards it is handed on threads of its own. The relay's `--workers` flag @@ -22,9 +23,16 @@ own threading model. - moq-tokio's `worker/group.rs` shrinks to spawning one pinned `current_thread` runtime per shard; moq-uring's `udp::Bound` group code is replaced by the library's. -- A server endpoint defaults to one shard per core. Measure the default - against a single socket with `just bench` before flipping it, since the - relay's default changes. +- A server endpoint defaults to one shard per core only where + `SO_REUSEPORT` steering works, which today is Linux: `moq-sock` returns + `Unsupported` for both load balancing and steering elsewhere. macOS and + Windows keep a genuine single-socket endpoint, and an explicit count above + one is refused there rather than silently ignored. +- Before flipping the default, add a benchmark sweeping shard count against + concurrent session load (publishers and subscribers), so steering, + dispatch, and per-shard overhead show up as slopes. A single + default-versus-one-socket run cannot show a cost that grows with either + axis. Update `doc/bin/relay/` for the flag change. diff --git a/quest/m2/README.md b/quest/m2/README.md index 8d695ae09c..4de4234a25 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -46,7 +46,7 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [Starvation at frame granularity](/quest/m2/starvation-frames.md) - the acknowledged frontier moves at every frame boundary through `poll_acked`, with a delivery-delay histogram for jitter -- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - the fork reports +- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - `moq-quic` reports how far a send stream has been acknowledged and when - [poll_acked in web-transport](/quest/m2/quic-ack-hook.md) - the backend-neutral hook that awaits an acknowledged stream offset, implemented @@ -94,7 +94,7 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [L4S on the backbone](/quest/m2/quic-ecn.md) - an ECT(1) option in the fork, an `ecn` config knob, and a dualpi2 measurement - [Careful resume on reconnect](/quest/m2/quic-careful-resume.md) - a redial starts at the previous connection's rate - [Keep-alive by deadline](/quest/m2/quic-keep-alive.md) - a PING only when the idle deadline nears, no fixed timer -- [noq socket close](/quest/m2/noq-socket-close.md) - noq releases an endpoint's socket on close, so moq-tokio drops its wrapper +- [Socket close](/quest/m2/noq-socket-close.md) - `moq-quic` releases an endpoint's socket on close, so moq-tokio drops its wrapper - [GOP overhead](/quest/m2/gop-overhead.md) - price the I-frames a short GOP pays for, deciding whether a long GOP plus a keyframe request is worth designing - [Per-program SI](/quest/m2/ts-program-si.md) - a selected TS program's broadcast carries only its own service's SDT and EIT - [TS import health](/quest/m2/ts-import-health.md) - `moq import ts` counts the TR 101 290 errors of the feed it receives, PCR and PTS graded on its own values diff --git a/quest/m2/noq-socket-close.md b/quest/m2/noq-socket-close.md index 3fa7a9621b..a442d2121b 100644 --- a/quest/m2/noq-socket-close.md +++ b/quest/m2/noq-socket-close.md @@ -1,21 +1,24 @@ -# [M] noq releases an endpoint's socket on close +# [M] moq-quic releases an endpoint's socket on close ## Goal -A noq endpoint can close its UDP socket, and report when it has, without +A `moq-quic` endpoint can close its UDP socket, and report when it has, without waiting for every connection handle to drop. moq-tokio then deletes the closable socket wrapper that #4087 added to make `Listener::close` release the port. ## Plan -In moq-dev/noq, add an endpoint operation that closes the endpoint, waits +In `moq-quic-tokio`, add an endpoint operation that closes the endpoint, waits until each connection has sent its close, and then releases the socket. Later -sends are dropped and receives end. Test it there. Cut a noq release, bump the -pin, and replace moq-tokio's wrapper with the upstream call. The Go +sends are dropped and receives end. Test it there and replace moq-tokio's wrapper with it in the same PR. The Go `TestReconnectAcrossRelayRestart` and moq-tokio's `close_releases_quic_socket` stay green. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the endpoint is in-tree + ## Related - [Listener close](https://github.com/moq-dev/moq/pull/4087) - the local wrapper this replaces diff --git a/quest/m2/quic-ack-hook.md b/quest/m2/quic-ack-hook.md index 2879ba8da3..9dbb125ee1 100644 --- a/quest/m2/quic-ack-hook.md +++ b/quest/m2/quic-ack-hook.md @@ -3,18 +3,14 @@ ## Goal A released `web-transport-trait` lets a sender await acknowledgment of a -named byte offset on a send stream, a released `web-transport-moq` implements +named byte offset on a send stream, `web-transport-moq` implements it, and every other backend reports that it cannot rather than returning a guess. ## Plan The trait method lives in `moq-dev/web-transport`. The `web-transport-moq` -adapter lives in the moq-dev/noq fork and is released from it -(Cargo.toml:236-237), so its implementation folds into -[ACK progress](/quest/m2/quic-ack-progress.md)'s release rather than a -separate one. This quest exists so the release it produces is one condition -dependents wait on; it supersedes the design in moq-dev/web-transport#368, +adapter is in-tree after [the fork](/quest/m1/quic/fork/README.md). This quest supersedes the design in moq-dev/web-transport#368, whose snapshot-counter shape loses the wakeups a latency sample needs. Add one method to `SendStream`, in the trait's poll style: @@ -33,7 +29,7 @@ use. A default body cannot construct a backend's own `Self::Error`, so unsupported has to live in the return type rather than the error, and a consumer must treat `None` as unknown, never as delivered. -Implement it in `web-transport-moq`, in the fork, over the fork's accessor. Leave +Implement it in `web-transport-moq` over `moq-quic`'s accessor. Leave `web-transport-wasm` on the default; the browser's `WebTransportSendStream.getStats()` is unimplemented in shipping Chrome and its `bytesAcknowledged` is at risk in the W3C draft. qmux over a reliable transport may treat serialization as @@ -45,17 +41,15 @@ the middle of an in-flight frame, several waiters on one stream in offset order, a waiter whose offset lies beyond the final size, reset by sender, STOP_SENDING by the receiver, and session close. -Cut a `web-transport-trait` release, and ship the adapter in the fork -release that carries ACK progress. The quest completes when both are on -crates.io. +Cut a `web-transport-trait` release; the adapter lands in-tree against it. Decided in the 2026-09-30 audit: moved to m2 with its only consumer, [frame-granularity starvation](/quest/m2/starvation-frames.md). ## Required -- [Per-stream ACK progress in noq](/quest/m2/quic-ack-progress.md) - the - noq-proto accessor the adapter reads, released together +- [Per-stream ACK progress](/quest/m2/quic-ack-progress.md) - the + `moq-quic` accessor the adapter reads ## Related diff --git a/quest/m2/quic-ack-progress.md b/quest/m2/quic-ack-progress.md index 839ad1b145..31f15a52b7 100644 --- a/quest/m2/quic-ack-progress.md +++ b/quest/m2/quic-ack-progress.md @@ -1,17 +1,16 @@ -# [S] noq-proto reports per-stream acknowledgment progress +# [S] moq-quic reports per-stream acknowledgment progress ## Goal -A released noq-proto lets a sender ask how much of a send stream the peer has +`moq-quic` lets a sender ask how much of a send stream the peer has acknowledged and when each acknowledgment was received, corrected for the peer's reported ACK delay. Nothing MoQ builds on top needs private state. ## Plan -The work lives in the moq-dev/noq fork. This quest exists so the release it -produces is one condition dependents wait on. +The work lives in `moq-quic`, in-tree after [the fork](/quest/m1/quic/fork/README.md). -noq-proto already tracks everything needed in `SendBuffer`: the acknowledged +noq-proto (and so quinn-proto; check on the fork) already tracks everything needed in `SendBuffer`: the acknowledged range set, `fully_acked_offset()`, and the unacked length. `Connection` also computes the peer's `ack_delay` per ACK frame when it updates the RTT estimator. Expose that state through the public `SendStream` handle without @@ -33,15 +32,16 @@ stream teardown: the accessor must not return a prefix that includes bytes the peer will never acknowledge, and a waiter for an offset beyond the final size must resolve with the reset instead of hanging. -Land it in the fork and offer it upstream once it is stable. The -`web-transport-moq` half of [the ACK hook](/quest/m2/quic-ack-hook.md) lives -in the same fork repository and ships in the same release. The quest -completes when a `moq-noq-proto` release carries the accessor and -`Cargo.lock` here can name it. +The `web-transport-moq` half of [the ACK hook](/quest/m2/quic-ack-hook.md) +is in-tree too and can land in the same PR. Decided in the 2026-09-30 audit: moved to m2 with its consumers, the ACK hook and [frame-granularity starvation](/quest/m2/starvation-frames.md). +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the accessor lands in `moq-quic` + ## Related - [poll_acked in web-transport](/quest/m2/quic-ack-hook.md) - the first diff --git a/quest/m2/quic-bbr-loss-parity.md b/quest/m2/quic-bbr-loss-parity.md index 3b3c877d5e..1b6d118264 100644 --- a/quest/m2/quic-bbr-loss-parity.md +++ b/quest/m2/quic-bbr-loss-parity.md @@ -34,3 +34,7 @@ no `Controller` or wire change. Decided in the 2026-09-30 audit: moved to m2. The seven correctness fixes already shipped, and this remaining gap has no observed impact on MoQ traffic yet. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork diff --git a/quest/m2/quic-bbr-natural-drain.md b/quest/m2/quic-bbr-natural-drain.md index 77bdd93c03..a6dd11bde9 100644 --- a/quest/m2/quic-bbr-natural-drain.md +++ b/quest/m2/quic-bbr-natural-drain.md @@ -75,6 +75,10 @@ pinned code and configurations. Keep transport differences and QUICHE flags explicit and compare each algorithm change separately. A simulation result is not an end-to-end network measurement. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork + ## Related - [Discover media headroom](/quest/m2/quic-probe.md) - preserving an estimate and discovering spare capacity are separate problems diff --git a/quest/m2/quic-careful-resume.md b/quest/m2/quic-careful-resume.md index 30d906eb0a..81a09db0d3 100644 --- a/quest/m2/quic-careful-resume.md +++ b/quest/m2/quic-careful-resume.md @@ -30,6 +30,10 @@ Measure time to the encoder's target rate after a reconnect on the impaired path profile, plus loss and latency during the jump. Ship it on by default only when the jump never makes the first second worse than slow start. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork + ## Related - [Transport upgrade](/quest/m1/transport-upgrade/README.md) - one of the diff --git a/quest/m2/quic-deadline.md b/quest/m2/quic-deadline.md index d17acb6f9c..634e382126 100644 --- a/quest/m2/quic-deadline.md +++ b/quest/m2/quic-deadline.md @@ -46,6 +46,7 @@ Decided in the 2026-09-30 audit: moved to m2. No m1 quest consumes it. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - [Hierarchical stream scheduling](/quest/m1/quic/scheduler.md) - the scheduler decides which stream's data a probe carries diff --git a/quest/m2/quic-ecn-measure.md b/quest/m2/quic-ecn-measure.md index b04caf4481..32657baf08 100644 --- a/quest/m2/quic-ecn-measure.md +++ b/quest/m2/quic-ecn-measure.md @@ -39,3 +39,7 @@ from 1.3.1 is a defective baseline, not evidence that classic ECN cannot help. Decided in the 2026-09-30 audit: moved to m2 with its only consumer, [L4S on the backbone](/quest/m2/quic-ecn.md). + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork diff --git a/quest/m2/quic-ecn.md b/quest/m2/quic-ecn.md index a2779e7c03..980ffefb2c 100644 --- a/quest/m2/quic-ecn.md +++ b/quest/m2/quic-ecn.md @@ -35,5 +35,6 @@ need the fork. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - [Measure ECN on the backbone](/quest/m2/quic-ecn-measure.md) - the provider verdict this quest acts on diff --git a/quest/m2/quic-egress-profile.md b/quest/m2/quic-egress-profile.md index 6a30e7127f..71c75e9d17 100644 --- a/quest/m2/quic-egress-profile.md +++ b/quest/m2/quic-egress-profile.md @@ -48,6 +48,10 @@ Report CPU per Gbps, RSS, loss, and p99 latency via `just bench BASE` on Linux. The verdict names which levers ship and which qdisc, if any, relay hosts need. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork + ## Related - [#3201](/quest/m3/3201-moq-uring-use-sendmsg-zc-for-large-udp-gso-trains.md) - diff --git a/quest/m2/quic-keep-alive.md b/quest/m2/quic-keep-alive.md index f9943f8ccb..6fb98dd04b 100644 --- a/quest/m2/quic-keep-alive.md +++ b/quest/m2/quic-keep-alive.md @@ -33,3 +33,7 @@ documented in `doc/bin/relay/config.md` for NAT-sensitive deployments. The qmux Tests: an idle connection survives an idle timeout with exactly one PING per period; a busy connection sends none; a lost PING is probed before the deadline; the maximum knob shortens the period. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork diff --git a/quest/m2/quic-probe.md b/quest/m2/quic-probe.md index 9f9f4827d1..c053143f61 100644 --- a/quest/m2/quic-probe.md +++ b/quest/m2/quic-probe.md @@ -44,6 +44,10 @@ is not proof of full path capacity. Persist regressions in CI and broader network scenarios at least nightly. A measured no-go is a valid outcome; retain the baseline and record why before exposing an ineffective option. +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork + ## Related - [Natural media drains](/quest/m2/quic-bbr-natural-drain.md) - separate ProbeRTT policy experiment diff --git a/quest/m2/quic-qmux.md b/quest/m2/quic-qmux.md index 30bde05913..c7cb7af6ff 100644 --- a/quest/m2/quic-qmux.md +++ b/quest/m2/quic-qmux.md @@ -41,21 +41,18 @@ prohibited QUIC frames, params-first setup, record-size validation, close and reset semantics (the first recorded close wins, as close codes #4262 settled on dev), keep-alive behavior, and bounded flow-control tests. -The crate lives in the fork's workspace as `moq-noq-qmux`, so the stream -state machine internals it drives stay crate-private there; `moq-dev/web-transport`'s +The crate lives in-tree as `moq-quic-qmux` beside `moq-quic`, so the +stream state machine internals it drives stay private to that family; `moq-dev/web-transport`'s `qmux` becomes a thin re-export or is retired. There must be one stream state machine in the dependency graph, and the accessors qmux needs are -`pub(crate)` to that workspace, never part of `moq-noq-proto`'s public API. +never part of `moq-quic`'s documented public API. A separate crate cannot +see `pub(crate)`, so choose between a `moq-quic` module behind a `qmux` +feature and a hidden seam in this quest; the module is the simpler default. -Release it from the fork like the other fork crates: tag `moq-noq-qmux` with -the parent noq commit and the carried patches (each with its upstream PR or -the reason there is none), pin the crates.io version in this workspace, and -verify minimal, default, and all-feature builds so enabling iroh, qmux, or -the uring runtime never unifies two copies of the stream state. The quest -completes when that release is on crates.io and `Cargo.lock` names it. +Verify minimal, default, and all-feature builds so enabling iroh, qmux, or +the uring runtime never unifies two copies of the stream state. Decided in the 2026-09-30 audit: moved to m2, since no m1 quest consumes it. -The separate QUIC release quest was deleted and its qmux step folded here. ## Required diff --git a/quest/m3/quic-gcc.md b/quest/m3/quic-gcc.md index 268726aa01..60fdaf3f5b 100644 --- a/quest/m3/quic-gcc.md +++ b/quest/m3/quic-gcc.md @@ -33,5 +33,6 @@ exists. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - [Receive timestamps](/quest/m3/quic-receive-ts.md) - the per-packet arrival times the delay filter runs on diff --git a/quest/m3/quic-receive-ts.md b/quest/m3/quic-receive-ts.md index ad94b72070..7cf17d9bd8 100644 --- a/quest/m3/quic-receive-ts.md +++ b/quest/m3/quic-receive-ts.md @@ -32,6 +32,7 @@ GCC until such a consumer exists. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - A native-to-native consumer of receive timestamps (browsers never see them) ## Related From 952fc23e9f288642d2de4c6b2f88e25661d81520 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 30 Sep 2026 12:33:59 -0700 Subject: [PATCH 3/3] quest(quic): plan qmux in-tree, carry the quinn PRs, and link the noq issues Co-Authored-By: Claude Opus 5.5 --- quest/m0/noq-reassembly-cap.md | 7 +++- quest/m1/quic/README.md | 1 + quest/m1/quic/fork/bbr3.md | 10 ++++++ quest/m1/quic/fork/import.md | 11 ++++++ quest/m1/quic/fork/stream-slots.md | 8 ++++- quest/m1/quic/reliable-reset.md | 1 + quest/m1/quic/scheduler.md | 2 ++ quest/m1/quic/send-errors.md | 29 ++++++++++++++++ quest/m2/quic-ack-progress.md | 1 + quest/m2/quic-careful-resume.md | 1 + quest/m2/quic-deadline.md | 1 + quest/m2/quic-ecn.md | 4 +++ quest/m2/quic-keep-alive.md | 4 +++ quest/m2/quic-probe.md | 2 ++ quest/m2/quic-qmux.md | 56 +++++++++++++++++++++--------- quest/m3/quic-gcc.md | 4 +++ 16 files changed, 123 insertions(+), 19 deletions(-) create mode 100644 quest/m1/quic/send-errors.md diff --git a/quest/m0/noq-reassembly-cap.md b/quest/m0/noq-reassembly-cap.md index 160d9c08f7..cc4a200a02 100644 --- a/quest/m0/noq-reassembly-cap.md +++ b/quest/m0/noq-reassembly-cap.md @@ -20,7 +20,8 @@ the crates are renamed. and #22: fold the port into those releases or take the next patch numbers, then bump the pins here. - Open the same port as a PR on n0-computer/noq (approved 2026-09-29; the - advisory is public). iroh builds stay on the unfixed upstream crate until + advisory is public; opened as + [noq#828](https://github.com/n0-computer/noq/pull/828)). iroh builds stay on the unfixed upstream crate until n0 releases it; bump when they do. - Give `moq-tokio` a finite default connection `receive_window` instead of the backend's `VarInt::MAX` (`rs/moq-tokio/src/noq.rs` `apply_windows`, @@ -32,6 +33,10 @@ the crates are renamed. - `rs/moq-uring` depends on `moq-noq-proto` too, so the same pin bump covers the io_uring workers. +The in-tree [fork](/quest/m1/quic/fork/README.md) inherits the cap from +quinn main, so after it lands only the receive-window default and iroh's noq +pin remain relevant on `dev`. + Public API: none. Wire: a peer that exceeds the chunk cap is closed. ## Related diff --git a/quest/m1/quic/README.md b/quest/m1/quic/README.md index 785326dc89..856696ec7b 100644 --- a/quest/m1/quic/README.md +++ b/quest/m1/quic/README.md @@ -66,6 +66,7 @@ consumes them. and MAX_DATA are raised after SETUP identifies a cluster peer - [Shard the endpoint](/quest/m1/quic/shard.md) - the library shards a server endpoint across cores by default, replacing the `--workers` group +- [Surface UDP send errors](/quest/m1/quic/send-errors.md) - a client drops a QUIC attempt the moment its address family is unreachable ## Related diff --git a/quest/m1/quic/fork/bbr3.md b/quest/m1/quic/fork/bbr3.md index 727836ea44..45d81cbcc1 100644 --- a/quest/m1/quic/fork/bbr3.md +++ b/quest/m1/quic/fork/bbr3.md @@ -19,9 +19,19 @@ The fork's roughly 580 lines of regression tests are written against noq's test harness; rework them onto quinn's rather than porting the harness. Keep MoQ's config naming controller families (`Loss`, `Delay`), never algorithms. +BBR3 must fill in `ControllerMetrics::bandwidth_estimate` (quinn#2802, merged), +which the [switch](/quest/m1/quic/fork/switch.md) reads for +`web-transport-moq`'s bandwidth stat. + Verify with the fork's BBR tests and the benchmark matrix against `moq-noq` 1.3.x on the same workloads; report any throughput or latency difference. ## Required - [Import quinn](/quest/m1/quic/fork/import.md) + +## Related + +- [noq#819](https://github.com/n0-computer/noq/issues/819) - BBR3 as the default, proposed to n0; their sims often favor Cubic +- [noq#475](https://github.com/n0-computer/noq/issues/475) - the original BBR ask +- [noq#768](https://github.com/n0-computer/noq/issues/768) - the `inflight_at_loss` underflow the fork fixed with `saturating_sub`; keep that regression diff --git a/quest/m1/quic/fork/import.md b/quest/m1/quic/fork/import.md index 56179975f9..b0f6c03b3d 100644 --- a/quest/m1/quic/fork/import.md +++ b/quest/m1/quic/fork/import.md @@ -14,6 +14,17 @@ separate commits. Record the parent commit in `rs/moq-quic/README.md` with the advisory triage process the [questline](/quest/m1/quic/fork/README.md) describes. +Carry [quinn#2724](https://github.com/quinn-rs/quinn/pull/2724) as its own +cherry-pick commit (the same fix as noq#746, decided 2026-09-30): when the +kernel rejects a GSO batch with EIO or EINVAL, quinn-udp disables GSO and +drops the batch, so an Android client's Initial waits a full RTO and the +WebSocket fallback wins the race. Resend the batch instead. List it as carried +in `rs/moq-quic/README.md`; drop it if upstream lands +[quinn#2748](https://github.com/quinn-rs/quinn/pull/2748) and we cherry-pick +that. moq-uring's own UDP path (`rs/moq-uring/src/udp.rs`) submits GSO +trains itself, so check whether it drops a rejected train the same way, and +fix it here if so. + Keep quinn's own features (qlog, rustls providers, runtimes) as they are; pruning waits until the [switch](/quest/m1/quic/fork/switch.md) shows what MoQ uses. Wire the crates into release-plz like the other `rs/` crates, and check diff --git a/quest/m1/quic/fork/stream-slots.md b/quest/m1/quic/fork/stream-slots.md index 4b541b4623..4956c768bd 100644 --- a/quest/m1/quic/fork/stream-slots.md +++ b/quest/m1/quic/fork/stream-slots.md @@ -7,7 +7,8 @@ ## Plan -Port quinn#2601 (the same change noq took as noq#667). #3342 measured the +Cherry-pick the approved commits of quinn#2601 (the same change noq took as +noq#667); decided 2026-09-30 not to rebase the upstream PR first. #3342 measured the difference it makes: 75 vs 141 MiB relay memory on bulk and 31 vs 97 MiB on fanout. Re-run those workloads on `moq-quic` with and without the change. @@ -18,3 +19,8 @@ make sure the combined code keeps exactly one check. ## Required - [Import quinn](/quest/m1/quic/fork/import.md) + +## Related + +- [quinn#2601](https://github.com/quinn-rs/quinn/pull/2601) - the approved upstream PR this cherry-picks +- [noq#667](https://github.com/n0-computer/noq/pull/667) - the same change merged into noq diff --git a/quest/m1/quic/reliable-reset.md b/quest/m1/quic/reliable-reset.md index 5be1e64a0d..eca1feab38 100644 --- a/quest/m1/quic/reliable-reset.md +++ b/quest/m1/quic/reliable-reset.md @@ -76,3 +76,4 @@ provisional codepoints if the document changes before release. known browser-compliance gap. - [qmux on the QUIC stream state machine](/quest/m2/quic-qmux.md) - consumes the same reset state without a parallel implementation +- [noq#809](https://github.com/n0-computer/noq/issues/809) - the RESET_STREAM_AT ask to n0 diff --git a/quest/m1/quic/scheduler.md b/quest/m1/quic/scheduler.md index 0f564298c2..e39292b84a 100644 --- a/quest/m1/quic/scheduler.md +++ b/quest/m1/quic/scheduler.md @@ -123,3 +123,5 @@ where the new implementation makes it redundant. scope - [Signed priority](/quest/m2/signed-priority.md) - changes the priority type this orders on; keep the ordering, not just the type +- [noq#816](https://github.com/n0-computer/noq/issues/816) - the send-group proposal to n0; this quest supersedes it +- [noq#817](https://github.com/n0-computer/noq/issues/817) - the 64-bit priority ask; send groups replace the scalar, but the per-stream order within a group must still hold a 64-bit group sequence diff --git a/quest/m1/quic/send-errors.md b/quest/m1/quic/send-errors.md new file mode 100644 index 0000000000..429ad578ac --- /dev/null +++ b/quest/m1/quic/send-errors.md @@ -0,0 +1,29 @@ +# [S] Surface UDP send errors so the client can fall back + +## Goal + +When a UDP send fails because the address family is unreachable (for +example an IPv6 target on a host without IPv6 routing), `moq-quic-tokio` +reports the error instead of swallowing it, and moq-tokio's dial race drops +that QUIC attempt at once rather than waiting out the handshake timeout. + +## Plan + +Rebase [kixelated/quinn#3](https://github.com/kixelated/quinn/pull/3) onto +`moq-quic-udp` and `moq-quic-tokio`. It is 154 commits behind quinn main and +never went upstream; it touches only the udp and tokio crates, not the +sans-IO core. Decided 2026-09-30: an m1 quest after the switch, since the +client connect path is user-facing. + +Only fatal, destination-specific errors end the attempt; transient ones +(`ENOBUFS`, `EAGAIN`) stay handled as packet loss, as today. Test the fallback with a dial +against an unroutable family, and check that moq-uring maps the same errors +through its ring. + +## Required + +- [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic-udp` and `moq-quic-tokio` + +## Related + +- [quinn#2766](https://github.com/quinn-rs/quinn/issues/2766) - the upstream issue diff --git a/quest/m2/quic-ack-progress.md b/quest/m2/quic-ack-progress.md index 31f15a52b7..cca026beb7 100644 --- a/quest/m2/quic-ack-progress.md +++ b/quest/m2/quic-ack-progress.md @@ -46,3 +46,4 @@ hook and [frame-granularity starvation](/quest/m2/starvation-frames.md). - [poll_acked in web-transport](/quest/m2/quic-ack-hook.md) - the first consumer of the accessor +- [noq#808](https://github.com/n0-computer/noq/issues/808) - the acked-offset ask to n0 diff --git a/quest/m2/quic-careful-resume.md b/quest/m2/quic-careful-resume.md index 81a09db0d3..d00d185ebd 100644 --- a/quest/m2/quic-careful-resume.md +++ b/quest/m2/quic-careful-resume.md @@ -39,3 +39,4 @@ only when the jump never makes the first second worse than slow start. - [Transport upgrade](/quest/m1/transport-upgrade/README.md) - one of the reconnects this speeds up - [Drain](/quest/m1/drain/README.md) - GOAWAY redials are the other +- [noq#815](https://github.com/n0-computer/noq/issues/815) - the careful-resume proposal to n0 diff --git a/quest/m2/quic-deadline.md b/quest/m2/quic-deadline.md index 634e382126..a644a87043 100644 --- a/quest/m2/quic-deadline.md +++ b/quest/m2/quic-deadline.md @@ -56,3 +56,4 @@ Decided in the 2026-09-30 audit: moved to m2. No m1 quest consumes it. delay replaces the half-RTT estimate - [Discover media headroom](/quest/m2/quic-probe.md) - can reuse retransmission machinery if redundant capacity probes prove worthwhile +- [noq#813](https://github.com/n0-computer/noq/issues/813) - the per-stream deadline proposal to n0 diff --git a/quest/m2/quic-ecn.md b/quest/m2/quic-ecn.md index 980ffefb2c..8e6db1d0df 100644 --- a/quest/m2/quic-ecn.md +++ b/quest/m2/quic-ecn.md @@ -38,3 +38,7 @@ need the fork. - [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - [Measure ECN on the backbone](/quest/m2/quic-ecn-measure.md) - the provider verdict this quest acts on + +## Related + +- [noq#814](https://github.com/n0-computer/noq/issues/814) - the L4S proposal to n0 diff --git a/quest/m2/quic-keep-alive.md b/quest/m2/quic-keep-alive.md index 6fb98dd04b..609172872f 100644 --- a/quest/m2/quic-keep-alive.md +++ b/quest/m2/quic-keep-alive.md @@ -37,3 +37,7 @@ deadline; the maximum knob shortens the period. ## Required - [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork + +## Related + +- [noq#810](https://github.com/n0-computer/noq/issues/810) - the proposal to n0; flub and matheus23 asked to keep a cap for NAT bindings, which the optional maximum covers diff --git a/quest/m2/quic-probe.md b/quest/m2/quic-probe.md index c053143f61..ffe83c091c 100644 --- a/quest/m2/quic-probe.md +++ b/quest/m2/quic-probe.md @@ -52,3 +52,5 @@ retain the baseline and record why before exposing an ineffective option. - [Natural media drains](/quest/m2/quic-bbr-natural-drain.md) - separate ProbeRTT policy experiment - [GCC egress experiment](/quest/m3/quic-gcc.md) - delay control changes what headroom means +- [noq#811](https://github.com/n0-computer/noq/issues/811) - probing while app-limited, proposed to n0 +- [noq#476](https://github.com/n0-computer/noq/issues/476) - the earlier bitrate-probing ask diff --git a/quest/m2/quic-qmux.md b/quest/m2/quic-qmux.md index c7cb7af6ff..a131cdfb01 100644 --- a/quest/m2/quic-qmux.md +++ b/quest/m2/quic-qmux.md @@ -1,17 +1,39 @@ -# [L] Run qmux on the QUIC stream state machine +# [XL] Run qmux on the QUIC stream state machine ## Goal -qmux uses noq-proto for stream lifecycle, flow -control, datagram buffering, priority scheduling, and transport parameters. -It owns only record framing and the semantics that differ from QUIC, instead -of maintaining a parallel stream implementation. +qmux lives in the monorepo and uses `moq-quic`'s stream state machine for +stream lifecycle, flow control, datagram buffering, priority scheduling, and +transport parameters. It owns only record framing, its transport adapters, +and the semantics that differ from QUIC, instead of maintaining a parallel +stream implementation. moq-tokio and moq-relay depend on the in-tree crate, +and moq-dev/web-transport no longer carries a Rust qmux. ## Plan +Decided in the 2026-09-30 plan: + +- **Scope is the Rust core plus adapters.** Port the prototype core and + moq-dev/web-transport's TCP, TLS, WebSocket, and UDS adapters, ALPN + negotiation, byte-stream framing, and socket stats (about 2.3k lines) onto + it, along with the `web-transport-trait` impl. The TypeScript `@moq/qmux` + and `@moq/web-socket-stream` stay in moq-dev/web-transport with their own + state machine and become the interop reference. +- **All four wire versions stay**: draft-02, draft-01, draft-00, and the + legacy `webtransport` format, since published clients (and the TS peer) + negotiate them. They differ only in record framing, so they share the one + state machine. +- **Shape**: the sans-IO part is `moq_quic::mux`, a module behind a feature, + so it reaches the stream state without widening `moq-quic`'s public API. + The tokio session, adapters, and trait impl are the `qmux` crate, published + from the monorepo under the existing crates.io name as its next breaking + version. moq-dev/web-transport removes its copy; the external dependents + (atmoq-moq-native, web-transport-ws) can move to the new release on their own schedule. + Start from the green -[kixelated/quinn#2](https://github.com/kixelated/quinn/pull/2) prototype. Rebase -it onto noq and preserve its central invariant: drive the +[kixelated/quinn#2](https://github.com/kixelated/quinn/pull/2) prototype, +which is already quinn-based. Rebase it onto `moq-quic` and preserve its +central invariant: drive the existing stream state machine through the same receive and write entry points as QUIC, then treat serialization to the reliable underlying transport as the acknowledgment. Do not copy the stream maps, flow-control accounting, reset @@ -34,29 +56,29 @@ reusable acceptance fixtures, including byte fairness, strict priority, newest/oldest group ordering, cancellation, and blocked-stream behavior. The native scheduler does not wait for this dependent proof. -Add the missing wire evidence before release: golden draft-02 vectors, -bidirectional interoperability against the published `qmux` 0.5.x crate, and +Add the missing wire evidence before release: golden vectors for each of the four versions, +bidirectional interoperability against the last web-transport `qmux` release (0.6.x), and the TypeScript qmux/WebSocket peer used by `js/net`. Preserve rejection of prohibited QUIC frames, params-first setup, record-size validation, close and reset semantics (the first recorded close wins, as close codes #4262 settled on dev), keep-alive behavior, and bounded flow-control tests. -The crate lives in-tree as `moq-quic-qmux` beside `moq-quic`, so the -stream state machine internals it drives stay private to that family; `moq-dev/web-transport`'s -`qmux` becomes a thin re-export or is retired. There must be one stream -state machine in the dependency graph, and the accessors qmux needs are -never part of `moq-quic`'s documented public API. A separate crate cannot -see `pub(crate)`, so choose between a `moq-quic` module behind a `qmux` -feature and a hidden seam in this quest; the module is the simpler default. +There must be one stream state machine in the dependency graph. Verify minimal, default, and all-feature builds so enabling iroh, qmux, or the uring runtime never unifies two copies of the stream state. -Decided in the 2026-09-30 audit: moved to m2, since no m1 quest consumes it. +Decided in the 2026-09-30 audit: moved to m2, since no m1 quest consumes it; +the fork plan the same day kept it there. ## Required +- [Hard fork](/quest/m1/quic/fork/README.md) - qmux drives `moq-quic` - [Reliable stream reset](/quest/m1/quic/reliable-reset.md) - qmux reuses the extension's stream state rather than implementing reset locally - [Hierarchical stream scheduling](/quest/m1/quic/scheduler.md) - qmux must expose the same scheduling contract as native QUIC + +## Related + +- [noq#812](https://github.com/n0-computer/noq/issues/812) - the qmux proposal to n0 diff --git a/quest/m3/quic-gcc.md b/quest/m3/quic-gcc.md index 60fdaf3f5b..ff482bdd4d 100644 --- a/quest/m3/quic-gcc.md +++ b/quest/m3/quic-gcc.md @@ -36,3 +36,7 @@ exists. - [Hard fork](/quest/m1/quic/fork/README.md) - the change lands in `moq-quic`, not the frozen fork - [Receive timestamps](/quest/m3/quic-receive-ts.md) - the per-packet arrival times the delay filter runs on + +## Related + +- [noq#818](https://github.com/n0-computer/noq/issues/818) - the GCC proposal to n0; matheus23 asked for a non-breaking `Controller` trait