fix(moq-tokio): finite default connection receive window (noq reassembly cap, part 1)

[kixelated]

Quest: quest/m0/noq-reassembly-cap.md

Problem

moq-tokio's noq and iroh backends leave the connection receive_window at VarInt::MAX. The only bound on what one peer can make us buffer is max_streams (10k) × the 1.25 MB stream window, which is about 12.5 GB per connection.

Approach

Landed here: moq-tokio applies a 64 MiB connection receive window when quic.receive_window is unset, on both noq and iroh. An explicit value still wins. The io_uring workers already run a fixed 16 MiB.

Landed elsewhere: the RUSTSEC-2026-0185 reassembly cap, with all three quinn commits (quinn-rs/quinn#2694, quinn-rs/quinn#2789, quinn-rs/quinn#2814), shipped in moq-noq 2.0.1 (pinned on main by #4699) and 1.3.3 (pinned on release by #4702). That also covers rs/moq-uring.

Quest trimmed here: the quest now only tracks the iroh build's upstream noq-proto 1.3.0, which stays unfixed until n0 releases n0-computer/noq#828 (still open).

Picking the window

A standalone harness ran one noq connection (BBRv3, 32 uni streams, default stream window) through moq-shaper. It swept receive_window and measured goodput after a 3 s warmup, over an 8 s window:

RTT	link	window	goodput	window/RTT
200 ms	100 Mbps	1 MiB	29.8 Mbps	42 Mbps
200 ms	100 Mbps	2 MiB	54.5 Mbps	84 Mbps
200 ms	100 Mbps	4 MiB / 8 MiB / unbounded	65 / 59 / 76 Mbps	link-bound
100 ms	100 Mbps	1 MiB	68.6 Mbps	84 Mbps
100 ms	100 Mbps	2 MiB / 16 MiB / 64 MiB / unbounded	86 / 50 and 74 / 82 / 71 and 83 Mbps	link-bound (two runs where listed)

When the window is the limit, goodput is about 0.65 to 0.8 × window/RTT. Above that point the window stops mattering.

The host had a load average around 100 on 32 cores (about 15 agents building), so the shaper topped out near 130 Mbps. That is why only small windows could be tested directly, and why the link-bound rows are noisy. Extrapolating the measured efficiency:

• 64 MiB carries about 3.5 to 4.5 Gbps at 100 ms, or about 1.8 to 2.2 Gbps at 200 ms. That is above what one noq connection pushes in practice, so a cluster session is limited by CPU before the window.
• 32 MiB would halve those numbers. 16 MiB (the io_uring value) would throttle a cluster link near 1 Gbps at 100 ms.

To rerun on a quiet host, build the harness and invoke it as window-sweep <rtt_ms> <rate_mbps|0> <streams> <secs> <window_MiB|0>... (source below). A higher rate shows where 16/32/64 MiB actually bind.

Harness (standalone crate, not committed)

It depends on moq-noq (the fork, with the reassembly-cap port), moq-shaper (path dep on rs/moq-shaper), rcgen, rustls, and tokio. It opens streams uni streams, writes 64 KiB chunks until the deadline, and drains and counts them on the server. The transport mirrors moq-tokio: max_concurrent_*_streams(10_000), Bbr3Config, and receive_window(window << 20) unless 0. The shaper applies delay = rtt/2 each way and Rate { bits_per_second: rate * 1e6, burst: 64 KiB, queue: max(rtt, 20ms) }.

The shaped runs built against moq-dev/noq#25, so they also exercised the assembler cap under queue-overflow loss. Every connection reported close_reason() == None at the end, so no INTERNAL_ERROR.

Impact

• Public API: none. quic::Resolved::receive_window is unchanged: it stays None when unset, and the backend applies the default. This keeps the io_uring refusal of explicit windows working. The new constant is pub(crate).
• Behavior: noq and iroh connections now advertise a 64 MiB initial MAX_DATA instead of 2^62 - 1. A peer can have at most 64 MiB of unread data across all of its streams.
• Wire: no format change.

Alternatives

• Resolving the default in Config::resolve so Resolved::default() shows 64 MiB. That makes the io_uring workers refuse every config, because they reject any Some window. It also needs Option to become u64, a breaking change.
• A smaller default (16 or 32 MiB). It throttles cluster sessions until Peer limits lets them take a larger window than viewers.

Follow-ups

• Bump the iroh build's upstream noq-proto once n0 releases fix(proto): bound assembler chunk count (port of quinn RUSTSEC-2026-0185 fixes) n0-computer/noq#828 (the trimmed quest).
• Upstream noq's Assembler::defragment starts at bytes_read (quinn starts at 0), so unordered readers can lose unread chunks. MoQ reads in order and is unaffected. Filing it on n0-computer/noq needs separate approval.

(Written by Claude Opus 5.5)

🤖 Generated with Claude Code

[kixelated]

Outcome of the quest start (background agent, so the decisions below are open for the maintainer):

• Landed here: a 64 MiB default connection receive_window for noq and iroh.
• Open drafts: fix(proto): bound assembler chunk count (RUSTSEC-2026-0185) noq#25 (2.x) and fix(proto): bound assembler chunk count (RUSTSEC-2026-0185) (1.3) noq#26 (1.3), plus fix(proto): bound assembler chunk count (port of quinn RUSTSEC-2026-0185 fixes) n0-computer/noq#828 upstream.
• Remaining: fork release, then the moq-noq-* pin bump and quest deletion.

Decisions, with my recommendations:

1. Port scope: all three quinn commits (0.11.18) rather than the quest's 0.11.15. Recommend keeping it: 0.11.15 alone is bypassable.
2. Release: fold fix(proto): bound assembler chunk count (RUSTSEC-2026-0185) noq#25/SUBSCRIBE message #26 into the open 1.3.3/2.0.1 release PRs (Migrate to quinn and async Rust #21/Enable BBR. #22) instead of 1.3.4/2.0.2. Recommend folding.
3. Default window: 64 MiB (alternatives: 32 MiB, or 16 MiB to match io_uring). Recommend 64 MiB until peer limits land. Measured under heavy host load; see the body for how to rerun.

(Written by Claude Opus 5.5)

[kixelated]

Decisions (from the maintainer, via the coordinating agent):

• Keep all three quinn commits (proto: yield error on too many gaps in assembler quinn-rs/quinn#2694, Assorted remote memory use fixes quinn-rs/quinn#2789, proto: coalesce contiguous chunks during defragment quinn-rs/quinn#2814), matching quinn-proto 0.11.18, rather than only the 0.11.15 fix.
• Fold the reassembly cap into the unpublished releases: 2.0.1 (chore: release 2.0.1 noq#22, with Multiple renditions #25) and 1.3.3 (chore: release 1.3.3 noq#21, with SUBSCRIBE message #26). Both releases also carry Loop disk source #23/Stream prioritization #24.
• moq-tokio's default connection receive_window stays at 64 MiB (fix(moq-tokio): finite default connection receive window (noq reassembly cap, part 1) #4605).
• The upstream Assembler::defragment unordered-mode issue is not being filed on n0-computer/noq yet; it needs separate approval.

Status: #22 is rebuilt on #25 and its changelog updated. #21 is not updated yet; see the comment there.

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 34 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: d7ad67c3-4f13-4dfb-8f46-9e3c3062d3f8

📥 Commits

Reviewing files that changed from the base of the PR and between 5fc2b57 and 0702873.

📒 Files selected for processing (8)

• doc/bin/relay/config.md
• doc/lib/rs/index.md
• quest/m0/README.md
• quest/m0/noq-reassembly-cap.md
• quest/m1/quic/fork/README.md
• rs/moq-tokio/src/iroh.rs
• rs/moq-tokio/src/noq.rs
• rs/moq-tokio/src/quic.rs

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kixelated]

Merging with maintainer approval.

Changes since the last CI run:

• Merged current main.
• Trimmed quest/m0/noq-reassembly-cap.md to the remaining iroh upstream noq-proto bump (fix(proto): bound assembler chunk count (port of quinn RUSTSEC-2026-0185 fixes) n0-computer/noq#828, still open), since the moq-noq pins landed in deps: bump moq-noq stack to 2.0.1 #4699 (main) and deps: bump moq-noq to 1.3.3 for RUSTSEC-2026-0185 #4702 (release). Updated its references in quest/m0/README.md and quest/m1/quic/fork/README.md.
• Refreshed the PR body, which still described those pin bumps as pending.

Decisions on record: 64 MiB default receive window, all three quinn commits in the port. Filing the upstream Assembler::defragment issue still needs separate approval.

(Written by Claude Opus 5.5)

[kixelated]

Review (full)

Reviewed head: 07028738b46d4f54568ce2a1ff59a0b69094e2a0

Positive: finite connection receive_window for noq/iroh closes the ~12.5 GB per-connection buffering hole without breaking the io_uring None-means-fixed-window contract, and the quest trim matches the already-merged moq-noq 2.0.1 / 1.3.3 pins (#4699 / #4702).

Blocking

None found.

Non-blocking

1. Resolved::default docs vs reality (rs/moq-tokio/src/quic.rs around the Resolved struct docs, and doc/lib/rs/index.md). The struct still says Resolved::default is the canonical statement of every QUIC default for UIs, but receive_window intentionally stays None and the 64 MiB is applied only in noq::apply_windows / iroh::Config::bind. A UI that prints Resolved::default().receive_window will show unset, not 64 MiB. Worth a one-line exception in those docs (the field comment already says the right thing).

2. CI still queued at review time (Check / Test / Platform / Quest / Android). Local claim of 3099/3099 is noted; wait for green before merge.

Notes (not actionable here)

• iroh remains on unfixed upstream noq-proto 1.3.0 (Cargo.lock); confirmed n0-computer/noq#828 is still open/draft. The 64 MiB window bounds unread stream credit but does not replace the assembler chunk cap (per-chunk overhead can still exceed payload). Quest scope after the trim is correct.
• No iroh-side equivalent of apply_windows_writes_each_field; the shared constant + noq test is enough for this change.

Verdict: MERGE

Reviewed head SHA: 07028738b46d4f54568ce2a1ff59a0b69094e2a0

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

MERGE — head 07028738b46d4f54568ce2a1ff59a0b69094e2a0

Finite 64 MiB default connection receive_window on noq and iroh when unset, with Resolved still leaving None so io_uring workers keep refusing only explicit windows and keep their fixed 16 MiB. Quest trimmed to the remaining upstream noq-proto pin after moq-noq 2.0.1 / 1.3.3. No blocking issues.

Non-blocking

1. doc/lib/rs/index.md — “read the defaults from Resolved::default()” vs receive_window. The new sentence says unset receive_window defaults to 64 MiB, then still points readers at Resolved::default(). That struct keeps receive_window: None by design (backends apply the constant; uring needs the unset signal). Someone reading only Resolved will still see “unlimited / backend default.” Clarify that the 64 MiB default is applied in the noq/iroh backends, not stored on Resolved.

2. CI still pending on this head (Check / Test / Platform / Quest / Android were queued at review time).

3. Residual exposure (acknowledged): iroh still pulls unfixed upstream noq-proto 1.3.0 while n0-computer/noq#828 is open/draft. The connection window bounds unread bytes; it does not replace the assembler chunk-count cap. Quest status matches the tree (moq-noq-proto 2.0.1 on this lockfile; upstream noq-proto 1.3.0).

Verified: DEFAULT_RECEIVE_WINDOW gated to noq/iroh; both apply_windows paths use unwrap_or; noq unit test asserts the finite default replaces unlimited; uring still ensure!(window.is_none()); peer-limits link resolves.

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

On review finding 1 (Resolved::default docs): not changing it. gso, congestion_control, stream_receive_window, and the other Option fields already resolve to None meaning "the backend's default", so receive_window follows the existing convention, and its field doc names the 64 MiB. A UI already has to render None as "default" for those fields. Finding 2 (wait for green CI) is handled by auto-merge.

(Written by Claude Opus 5.5)