Repository navigation
quest(quic): hard-fork quinn in-tree as moq-quic #4617
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| # [S] Remove moq-uring's avoidable copies | ||
|
|
||
| ## Goal | ||
|
|
||
| moq-uring stops copying and allocating on paths where the QUIC core already | ||
| supports zero-copy, with a measured before/after. | ||
|
|
||
| ## Plan | ||
|
|
||
| Found in the 2026-09-30 fork planning; none needs a QUIC API change: | ||
|
|
||
| - Egress: `quic/noq/connection.rs` copies each GSO train from `scratch` into | ||
| the TX buffer. Hand the TX pool's `Vec` to `poll_transmit` directly; it | ||
| encrypts in place and only reserves when capacity is short. | ||
| - Stream send: moq-net's `write_chunk(Bytes)` falls back to the trait's | ||
| default `poll_write_buf`, which copies through `poll_write(&[u8])`. Route | ||
| it to the core's `write_chunks` so large chunks are kept by reference. | ||
| - Stream receive: moq-uring uses the trait's default `poll_read_chunk`, which | ||
| allocates an 8 KiB `BytesMut` and copies each read. Return the core's | ||
| `Bytes` directly. | ||
| - Datagram receive: `endpoint.rs` allocates a `BytesMut` per GRO segment. | ||
| Copy once per completion and split it into segments. | ||
|
|
||
| The clock-read fix stays in | ||
| [#3122](/quest/m1/perf/3122-moq-uring-2-5-of-relay-cpu-is-vdso-clock-reads-the-drive.md). | ||
| Measure with the echo benches and `just bench`. This branches from `dev`, | ||
| where the fork lands; paths above are pre-fork names. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Hard fork](/quest/m1/quic/fork/README.md) - the copies are removed on `moq-quic`, not the frozen fork |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,48 +2,42 @@ | |
|
|
||
| ## Goal | ||
|
|
||
| MoQ owns the QUIC features it needs in an in-tree stack, `moq-quic`, hard | ||
| forked from quinn (see [the fork](/quest/m1/quic/fork/README.md) for why | ||
| quinn and not noq). One core serves the tokio backend, the thread-per-core | ||
| `moq-uring` backend, and qmux; iroh stays on upstream noq. The features are | ||
| BBR correctness, reliable stream resets, hierarchical stream scheduling with | ||
| per-broadcast fairness, wider limits for relay peers, and endpoint sharding. | ||
| Per-stream acknowledgment progress, per-stream deadlines, qmux on the shared | ||
| stream state machine, and the experiments (GCC, receive timestamps, the egress | ||
| profile, media probing, L4S, careful resume, deadline keep-alive) live in | ||
| [m2](/quest/m2/README.md) and do not gate this line. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided in the 2026-09-30 plan: the moq-dev/noq soft fork is replaced by a | ||
| hard fork of quinn in `rs/`, so every QUIC change here waits for | ||
| [the fork](/quest/m1/quic/fork/README.md) and lands on `dev`. Retarget this | ||
| line's PR (#3975) to `dev` when the fork starts. moq-dev/noq is frozen to | ||
| security patches for `main`. | ||
|
|
||
| Older quests say "the fork" or "noq"; read that as `moq-quic`. Their steps to | ||
| publish a fork release, pin it, or offer a change upstream are superseded: | ||
| a change lands in-tree with its consumer, and upstreaming is optional. | ||
|
Comment on lines
+24
to
+26
There was a problem hiding this comment.
When the remaining m2 QUIC quests are started, this blanket reinterpretation still leaves their actual goals and completion criteria pointing at the repository that is now frozen to security fixes. For example, AGENTS.md reference: AGENTS.md:L27-L28 Useful? React with 👍 / 👎.
Collaborator
Author
There was a problem hiding this comment. Agreed. Fixed in 0dffe49: ACK progress, the ACK hook, qmux, and socket close no longer end in a moq-noq release, and every m2/m3 quest that changes the stack now lists the fork under Required. (Written by Claude Opus 5.5) |
||
|
|
||
| The seven BBR correctness fixes shipped in moq-noq 1.3.1 (#4206) and move to | ||
| `moq-quic` with the [BBR3 port](/quest/m1/quic/fork/bbr3.md). The remaining | ||
| BBR quests here and [BBR ACK cleanup](/quest/m1/bbr-ack-cleanup.md) all edit | ||
| `bbr3/mod.rs`, so one owner should work there at a time. Controller-level | ||
| regressions extend the shared test `Sim` in `bbr3/mod.rs` with only what each | ||
| needs, rather than adding another simulation loop; a fix at the transport | ||
| boundary still needs a transport test through the real callbacks. The | ||
| comparison against Google's BBR is part of the separate | ||
| [natural drain study](/quest/m2/quic-bbr-natural-drain.md). | ||
|
|
||
| MoQ's config names congestion families (`Loss`, `Delay`, and `RealTime` once | ||
| GCC ships), never algorithms; the `Controller` trait is the seam experiments | ||
| plug into, and MoQ owns which algorithm each family means. | ||
|
|
||
| The scheduling contract has three levels: strict subscription priority, | ||
| byte-fair service between send groups at the same priority, then the | ||
|
|
@@ -54,12 +48,11 @@ This is a transport API change, not a MoQ wire change. | |
|
|
||
| Decided in the 2026-09-30 audit: deadlines, qmux, BBR loss parity, ECN | ||
| measurement, ACK progress, and the ACK hook moved to m2, since no m1 quest | ||
| consumes them. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Hard fork](/quest/m1/quic/fork/README.md) - quinn in-tree as `moq-quic`, with BBR3 and lazy stream slots, and MoQ switched onto it | ||
| - [BBR idle burst](/quest/m1/quic/bbr-app-limited.md) - a fork regression proves a burst after a long idle is paced at the learned bandwidth, closing #4219 | ||
| - [Mark BBR starvation wherever the source runs dry](/quest/m1/quic/bbr-app-limited-edges.md) - partial polls count, local send caps do not, receiver credit is pinned | ||
| - [Deliver the application close before io_uring teardown](/quest/m1/quic/uring-close.md) - | ||
|
|
@@ -71,17 +64,16 @@ and the qmux crate release folds into [qmux](/quest/m2/quic-qmux.md). | |
| the lossy scalar; retransmits follow the same order | ||
| - [Relay peers get wider limits](/quest/m1/quic/peer-limits.md) - MAX_STREAMS | ||
| and MAX_DATA are raised after SETUP identifies a cluster peer | ||
| - [Shard the endpoint](/quest/m1/quic/shard.md) - the library shards a | ||
| server endpoint across cores by default, replacing the `--workers` group | ||
| - [Surface UDP send errors](/quest/m1/quic/send-errors.md) - a client drops a QUIC attempt the moment its address family is unreachable | ||
|
|
||
| ## Related | ||
|
|
||
| - [Scope track priority](/quest/m1/track-priority-scope.md) - the | ||
| per-broadcast fairness policy on cluster sessions | ||
| - [Starvation](/quest/m1/qos/starvation.md) - the first consumer of ACK | ||
| progress: how far behind viewers are, from the relay's point of view | ||
| - [Discover media headroom](/quest/m2/quic-probe.md) - test useful-media pacing before adding redundant probe traffic | ||
| - [L4S on the backbone](/quest/m2/quic-ecn.md) - an ECT(1) option in the fork, an `ecn` config knob, and a dualpi2 measurement | ||
| - [Careful resume on reconnect](/quest/m2/quic-careful-resume.md) - a redial starts at the previous connection's rate | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # Hard-fork quinn into the monorepo as moq-quic | ||
|
|
||
| ## Goal | ||
|
|
||
| MoQ's QUIC stack lives in `rs/` as `moq-quic` (the sans-IO core), | ||
| `moq-quic-udp`, `moq-quic-tokio`, and `web-transport-moq`, forked from | ||
| quinn-rs/quinn `main`. Every MoQ QUIC path (moq-tokio and moq-uring) runs on | ||
| it, the `moq-noq*` dependencies are gone, and a QUIC change lands in the same | ||
| PR as the MoQ code that needs it. | ||
|
|
||
| Out of scope: iroh keeps upstream noq (the `iroh` feature still compiles it for | ||
| iroh's controller factory), and multipath, NAT traversal, and QAD are not | ||
| carried. | ||
|
|
||
| ## Plan | ||
|
|
||
| Decided in the 2026-09-30 plan, with reasons, so later sessions don't reopen | ||
| them: | ||
|
|
||
| - **Hard fork, no merges.** The moq-dev/noq soft fork never merged upstream | ||
| (its weekly sync job failed on a workflow-permission error), n0-computer/noq | ||
| slowed to 5-7 commits a month in Aug-Sep, and n0's strict semver limits | ||
| what we can upstream, not what a fork can change. Upstream fixes are | ||
| cherry-picked by hand; offering our changes upstream is optional courtesy, | ||
| not tracked work. | ||
| - **Base on quinn, not noq.** quinn-proto is 32.9k lines against noq-proto's | ||
| 58.5k, about 15k of which is multipath, NAT traversal, and QAD that MoQ | ||
| never negotiates. Stripping those from noq was estimated at 2-3 weeks with | ||
| loss-recovery risk, and would leave every quinn fix a hand port. quinn ships | ||
| security fixes the day the advisory publishes (11 in 2026), and staying | ||
| close to its code keeps those cherry-picks cheap. The cost is re-porting | ||
| BBR3, the controller callbacks, and lazy stream slots, which is why MoQ | ||
| moved to noq in the first place (#1706, #3342, #3789). | ||
| - **Multipath is deleted**, and with it the multipath spike. | ||
| - **In-tree**, so `just check` and CI cover the stack, one PR spans the core | ||
| and its consumers, and `main` and `dev` each carry their own copy instead of | ||
| double-landing fixes on the 1.3 and 2.0 fork lines. | ||
| - **Named by role**: `moq-quic`, `moq-quic-udp`, `moq-quic-tokio`; | ||
| `web-transport-moq` keeps its name. Each crate's README and license credit | ||
| quinn and noq. | ||
| - **Lands on `dev`.** moq-tokio exposes `noq::Endpoint`, | ||
| `noq::TransportConfig`, and `noq::Incoming` publicly, so the crate swap is | ||
| a break. | ||
| - **Security folds into the fork.** Rebasing on quinn picks up the 2026 | ||
| advisories noq is missing; `main` stays on `moq-noq` until `dev` releases. | ||
| [noq reassembly cap](/quest/m0/noq-reassembly-cap.md) still finishes on the | ||
| old fork for `main`. | ||
| - **moq-dev/noq is frozen**: security patches for `main` only, archived once | ||
| no released MoQ crate depends on it. The other | ||
| [QUIC quests](/quest/m1/quic/README.md) wait for this line and land | ||
| in-tree on `dev`. | ||
|
|
||
| This README's own work: delete moq-dev/noq's `moq-sync.yml`, mark its README | ||
| frozen, and document advisory triage in `rs/moq-quic/README.md`. `cargo audit` | ||
| cannot match renamed crates, so the triage is: watch quinn-rs/quinn's security | ||
| advisories and releases, check each fix against `moq-quic`, and port it with | ||
| its regression test. Every carried change in moq-dev/noq's `CHANGELOG-MOQ.md` | ||
| is either ported by a child quest or recorded as not applicable in the | ||
| [switch](/quest/m1/quic/fork/switch.md) PR. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Import quinn](/quest/m1/quic/fork/import.md) - quinn's three crates build and test in-tree as `moq-quic*`, verbatim at a recorded commit, with no consumer yet | ||
| - [Port BBR3](/quest/m1/quic/fork/bbr3.md) - the fork's corrected BBR3 and controller callbacks run on `moq-quic` as the default controller | ||
| - [Lazy stream slots](/quest/m1/quic/fork/stream-slots.md) - relay memory on `moq-quic` matches `moq-noq` | ||
| - [Switch](/quest/m1/quic/fork/switch.md) - `web-transport-moq`, moq-tokio, and moq-uring run on `moq-quic`, and `moq-noq*` is gone | ||
|
|
||
| ## Related | ||
|
|
||
| - [Shard the endpoint](/quest/m1/quic/shard.md) - the first feature the fork makes possible | ||
| - [Remove moq-uring copies](/quest/m1/perf/uring-copies.md) - cheap copy removals on the new core | ||
| - [Redesign the QUIC I/O boundary](/quest/m2/quic-io-boundary.md) - the breaking perf rewrite, gated on a profile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| # [L] Port BBR3 and the controller callbacks | ||
|
|
||
| ## Goal | ||
|
|
||
| `moq-quic` carries the BBR3 controller from moq-dev/noq, with every | ||
| correctness fix the fork has shipped, and it is the default controller. The | ||
| fork's BBR regressions pass on `moq-quic`. | ||
|
|
||
| ## Plan | ||
|
|
||
| Move moq-dev/noq's `congestion/bbr3/` over as a unit rather than starting from | ||
| quinn#2481: the fork's copy already descends from #2481 and carries the seven | ||
| BBR fixes (#4206), the app-limited fix, and the classic ECN CE response | ||
| (moq-dev/noq#12). Port the `Controller` extensions it depends on (`PacketId` | ||
| and packet-space callbacks, app-limited, cwnd-limited, ACK frequency) by hand; | ||
| they are about 350 lines in `connection/mod.rs` and `pacing.rs`. | ||
|
|
||
| The fork's roughly 580 lines of regression tests are written against noq's | ||
| test harness; rework them onto quinn's rather than porting the harness. Keep | ||
| MoQ's config naming controller families (`Loss`, `Delay`), never algorithms. | ||
|
|
||
| BBR3 must fill in `ControllerMetrics::bandwidth_estimate` (quinn#2802, merged), | ||
| which the [switch](/quest/m1/quic/fork/switch.md) reads for | ||
| `web-transport-moq`'s bandwidth stat. | ||
|
|
||
| Verify with the fork's BBR tests and the benchmark matrix against `moq-noq` | ||
| 1.3.x on the same workloads; report any throughput or latency difference. | ||
|
|
||
| ## Required | ||
|
|
||
| - [Import quinn](/quest/m1/quic/fork/import.md) | ||
|
|
||
| ## Related | ||
|
|
||
| - [noq#819](https://github.com/n0-computer/noq/issues/819) - BBR3 as the default, proposed to n0; their sims often favor Cubic | ||
| - [noq#475](https://github.com/n0-computer/noq/issues/475) - the original BBR ask | ||
| - [noq#768](https://github.com/n0-computer/noq/issues/768) - the `inflight_at_loss` underflow the fork fixed with `saturating_sub`; keep that regression |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The newly added requirement is fresh evidence that this quest is supposed to land in
moq-quic, but its executable plan still says the fix lives inmoq-dev/noqand requires publishing and pinning an immutable fork release at lines 13 and 51-53. When this m1 quest is started, following those completion criteria sends the work to the repository this change freezes rather than the in-tree crate, so retarget or delete those steps instead of relying on the parent README's reinterpretation.AGENTS.md reference: AGENTS.md:L27-L28
Useful? React with 👍 / 👎.