From 3367412a23cea6ddc3586665cdcb22e0d7710068 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 17:22:32 -0700 Subject: [PATCH 1/7] quest: claim impaired-handshake Co-Authored-By: Claude Opus 5.5 From af7f1d219c8e270e1824954be30d04f8fb866481 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 18:35:18 -0700 Subject: [PATCH 2/7] fix(moq-quic): reset the PTO backoff when discarding Initial or Handshake keys RFC 9002 A.11 resets pto_count on key discard. A client never resets it on an Initial ACK, so its Initial backoff carried into the Handshake space and delayed the retransmission of a lost Finished past the peer's idle timeout. The impaired drills hit this as 'reset by peer'; moq-dev/noq#32 carries the same fix for the stack they run on. The quest records the second cause, two Initial flights per 3x-PTO idle window, which needs a decision. Co-Authored-By: Claude Opus 5.5 --- quest/m1/test-flakes-2/README.md | 2 +- quest/m1/test-flakes-2/impaired-handshake.md | 67 ++++++++++++++------ rs/moq-quic/README.md | 1 + rs/moq-quic/src/connection/mod.rs | 4 ++ rs/moq-quic/src/tests/mod.rs | 39 ++++++++++++ 5 files changed, 91 insertions(+), 22 deletions(-) diff --git a/quest/m1/test-flakes-2/README.md b/quest/m1/test-flakes-2/README.md index 406f511667..a95ac1e5c9 100644 --- a/quest/m1/test-flakes-2/README.md +++ b/quest/m1/test-flakes-2/README.md @@ -34,6 +34,6 @@ Public API: none. Wire: none. - [Media late join](/quest/m1/test-flakes-2/media-late-join.md) - a late joiner shows video promptly and catches up to live, and the check asserts that under load - [Relay restart rebind](/quest/m1/test-flakes-2/relay-restart-rebind.md) - the crash drill restarts on its original UDP address under concurrent load -- [Impaired handshake](/quest/m1/test-flakes-2/impaired-handshake.md) - the impaired cluster drills' clients never time out while connecting +- [Impaired handshake](/quest/m1/test-flakes-2/impaired-handshake.md) - the impaired drills' clients never time out while connecting - [Cluster burst overrun](/quest/m1/test-flakes-2/cluster-burst-overrun.md) - the impaired cluster burst drill always overruns its bottleneck, so it always exercises it - [Media audio tone](/quest/m1/media-audio-tone.md) - the `just test media` audio-tone check passes under load, fixed at its cause diff --git a/quest/m1/test-flakes-2/impaired-handshake.md b/quest/m1/test-flakes-2/impaired-handshake.md index 736e2d0b70..76e5724a88 100644 --- a/quest/m1/test-flakes-2/impaired-handshake.md +++ b/quest/m1/test-flakes-2/impaired-handshake.md @@ -2,30 +2,55 @@ ## Goal -The impaired cluster drills (`bursts_cross_a_cluster::impaired`, -`bursts_cross_a_flapping_peer::impaired` in `rs/moq-relay/tests/drills.rs`) -never fail while connecting their publisher or subscriber. +The impaired drills (`bursts_cross_a_cluster::impaired`, +`bursts_cross_a_flapping_peer::impaired`, and +`interrupted_publisher_republishes_new_content::impaired` in +`rs/moq-relay/tests/drills.rs`) never fail while connecting their publisher or +subscriber. ## Plan -Measured 2026-10-07 by running the built drill binary in a loop (four tests -in parallel): - -- `main` at `fe0113ff2`: 1 of 40 runs failed `publisher connect failed: - Noq(Connection("timed out"))`, seed 3495212678159036442. -- #4972, with both relays back on the default 10s idle timeout: 3 of 40 runs, - `subscriber connect failed: Noq(Connection("timed out"))` twice and once - `failed to exchange h3 settings: ... reset by peer`. - -So the relays' idle timeout is not the cause. The likely cause is the drills' -2s client idle timeout (`quic()`) against a handshake over 5% loss and a -100 kbit/s bottleneck: noq arms the idle timer at `max(idle, 3 * PTO)` with -the base PTO, while a backed-off PTO can leave the client silent longer than -that. Confirm with a qlog of a failing seed, then fix it at the cause, without -raising a timeout or retrying the connect. - -Public API: none. Wire: none. +Qlogs of failing runs (2026-10-09) show two causes. Both come from the +drills' 2s client idle timeout meeting a backed-off PTO during the handshake. + +1. **A lost Finished waits on the Initial backoff.** RFC 9002 A.11 resets + `pto_count` when Initial or Handshake keys are discarded. noq and quinn do + not, and a client never resets it on an Initial ACK. After five Initial + PTOs, a client armed its Handshake PTO 2.3s out. Its Finished was lost, the + server cannot read 1-RTT without it, and the server idled out first. The + client saw `reset by peer`. Fixed in `moq-quic` and in + [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); the drills pick it + up once that ships in a `moq-noq-proto` release and the pin moves to it. +2. **Two Initial flights per idle window.** Before an RTT sample the base PTO + is 999ms, so the idle timer is armed at `max(2s, 3 * 999ms)` = 2997ms. The + probes go at 0 and 999ms. The second probe is due at 2997ms plus timer slop, + so it always lands a millisecond or two after the idle deadline. If neither + flight draws a reply, the client fails `Noq(Connection("timed out"))`. On + `bursty()` the second ClientHello datagram (it spans two, with the ML-KEM + key share) always overflows the 50ms queue, so each flight relies on the + server's ACK getting through. This was 3 of 300 cluster runs and 1 of 480 + republish runs, and it is the quest's original failure. + +Cause 2 needs a decision, since it changes QUIC stack behavior: + +- A handshake idle timeout separate from the idle timeout, as msquic + (`HandshakeIdleTimeoutMs`, 10s) and Chromium do. Dead-peer detection on an + established connection is unchanged. This needs a new `TransportConfig` field + in `moq-quic`, and in moq-dev/noq too if the switch is not close. +- Cap the handshake probe interval so several probes fit in the idle window, + as picoquic does. This needs no API, but it changes the PTO deadline + computation, which has to stay monotonic in `pto_count`. +- Floor the idle timer at 3x the backed-off PTO. This slows dead-peer + detection whenever probes are outstanding, which works against the drills' + killed-relay budget. + +Measure with a loop of the impaired drills under load. A temporary +`quic.qlog` from an env var in `quic()` and `relay_config` captures a failing +run's qlog (needs the `qlog` feature). A missing server qlog means no client +Initial ever arrived. + +Public API: none so far. Wire: none. ## Related -- [Switch](/quest/m1/quic/fork/switch.md) - moves the runtimes from noq to `moq-quic`, whose idle-timer arming a fix may land in instead +- [Switch](/quest/m1/quic/fork/switch.md) - moves the runtimes from noq to `moq-quic`, which already carries the cause 1 fix and is where a cause 2 fix may land instead diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index 21b774bb8c..1632753e25 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -48,6 +48,7 @@ Changes on top of the upstream commit, besides the renames: A `WouldBlock` partway through the resend makes the caller retry the whole batch, duplicating the datagrams already sent; QUIC drops the duplicates. Drop it if upstream lands [quinn#2748](https://github.com/quinn-rs/quinn/pull/2748) and we cherry-pick that. - Apple fast-path fallback preserves the segmentation of batches prepared before the fast path was disabled, including when a private symbol is unavailable. +- Discarding Initial or Handshake keys resets the PTO backoff, as RFC 9002 A.11 does, so a client's Initial backoff no longer delays the retransmission of a lost Finished. Also [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); not offered upstream yet. Recheck the BBR3 changes when cherry-picking quinn#2481 updates, or if it merges. diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index ba3ff92752..275d9da76c 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -2360,6 +2360,10 @@ impl Connection { for packet in sent_packets.into_values() { self.remove_in_flight(&packet); } + // Discarding keys is progress, so the backoff from probing the old space + // must not delay the next one (RFC 9002 A.11). A client never resets it on + // an Initial ACK, so this is where its handshake backoff ends. + self.pto_count = 0; self.set_loss_detection_timer(now) } diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 2542ce9c83..6a3b4d2ba1 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -3225,6 +3225,45 @@ fn handshake_anti_deadlock_probe() { ); } +/// A client's lost Finished is probed at the Handshake space's own PTO, not one +/// still backed off from Initials lost before it (RFC 9002 A.11). +#[test] +fn lost_finished_after_initial_backoff() { + let _guard = subscribe(); + let mut pair = Pair::default(); + // Without latency the RTT is zero and every PTO is the timer granularity. + pair.latency = Duration::from_millis(50); + + let client_ch = pair.begin_connect(client_config()); + // The client's first Initial and two probes are lost, backing its PTO off three times. + for _ in 0..3 { + pair.drive_client(); + pair.server.inbound.clear(); + pair.time = pair.client.next_wakeup().unwrap(); + } + + // The next probe gets through, until the client's Finished, which is lost. + loop { + pair.drive_client(); + if !pair.client_conn_mut(client_ch).is_handshaking() { + break; + } + pair.drive_server(); + pair.time = min_opt(pair.client.next_wakeup(), pair.server.next_wakeup()).unwrap(); + } + pair.server.inbound.clear(); + let lost = pair.time; + + let server_ch = pair.server.assert_accept(); + while pair.server_conn_mut(server_ch).is_handshaking() { + assert!(pair.step()); + } + // One RTT is 100ms, so a retransmitted Finished arrives within about 400ms; + // still backed off, it takes 2.5s. + let elapsed = pair.time - lost; + assert!(elapsed < Duration::from_secs(1), "{elapsed:?}"); +} + /// Ensures that the server can respond with 3 initial packets during the handshake /// before the anti-amplification limit kicks in when MTUs are similar. #[test] From 4db5acf072037ce88fe2e8901ea01d7c871e4262 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 19:34:23 -0700 Subject: [PATCH 3/7] feat(moq-quic): add a handshake idle timeout Until the handshake completes, the idle timer uses TransportConfig::handshake_idle_timeout (default 10s, as msquic's HandshakeIdleTimeoutMs), then the negotiated idle timeout. A 2s idle timeout otherwise expires at 3x the initial PTO, just before the second Initial probe, leaving a client two flights: the impaired drills' 'timed out' connects. moq-dev/noq#34 carries the same field for the stack the drills run on; the quest now tracks the pin bump. Co-Authored-By: Claude Opus 5.5 --- quest/m1/test-flakes-2/impaired-handshake.md | 75 +++++++++----------- rs/moq-quic/README.md | 2 + rs/moq-quic/src/config/transport.rs | 15 ++++ rs/moq-quic/src/connection/mod.rs | 12 ++-- rs/moq-quic/src/tests/mod.rs | 42 +++++++++++ 5 files changed, 100 insertions(+), 46 deletions(-) diff --git a/quest/m1/test-flakes-2/impaired-handshake.md b/quest/m1/test-flakes-2/impaired-handshake.md index 76e5724a88..7276528e0a 100644 --- a/quest/m1/test-flakes-2/impaired-handshake.md +++ b/quest/m1/test-flakes-2/impaired-handshake.md @@ -1,4 +1,4 @@ -# [S] Impaired drill clients connect reliably +# [XS] Impaired drill clients connect reliably ## Goal @@ -10,47 +10,38 @@ subscriber. ## Plan -Qlogs of failing runs (2026-10-09) show two causes. Both come from the -drills' 2s client idle timeout meeting a backed-off PTO during the handshake. - -1. **A lost Finished waits on the Initial backoff.** RFC 9002 A.11 resets - `pto_count` when Initial or Handshake keys are discarded. noq and quinn do - not, and a client never resets it on an Initial ACK. After five Initial - PTOs, a client armed its Handshake PTO 2.3s out. Its Finished was lost, the - server cannot read 1-RTT without it, and the server idled out first. The - client saw `reset by peer`. Fixed in `moq-quic` and in - [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); the drills pick it - up once that ships in a `moq-noq-proto` release and the pin moves to it. -2. **Two Initial flights per idle window.** Before an RTT sample the base PTO - is 999ms, so the idle timer is armed at `max(2s, 3 * 999ms)` = 2997ms. The - probes go at 0 and 999ms. The second probe is due at 2997ms plus timer slop, - so it always lands a millisecond or two after the idle deadline. If neither - flight draws a reply, the client fails `Noq(Connection("timed out"))`. On - `bursty()` the second ClientHello datagram (it spans two, with the ML-KEM - key share) always overflows the 50ms queue, so each flight relies on the - server's ACK getting through. This was 3 of 300 cluster runs and 1 of 480 - republish runs, and it is the quest's original failure. - -Cause 2 needs a decision, since it changes QUIC stack behavior: - -- A handshake idle timeout separate from the idle timeout, as msquic - (`HandshakeIdleTimeoutMs`, 10s) and Chromium do. Dead-peer detection on an - established connection is unchanged. This needs a new `TransportConfig` field - in `moq-quic`, and in moq-dev/noq too if the switch is not close. -- Cap the handshake probe interval so several probes fit in the idle window, - as picoquic does. This needs no API, but it changes the PTO deadline - computation, which has to stay monotonic in `pto_count`. -- Floor the idle timer at 3x the backed-off PTO. This slows dead-peer - detection whenever probes are outstanding, which works against the drills' - killed-relay budget. - -Measure with a loop of the impaired drills under load. A temporary -`quic.qlog` from an env var in `quic()` and `relay_config` captures a failing -run's qlog (needs the `qlog` feature). A missing server qlog means no client -Initial ever arrived. - -Public API: none so far. Wire: none. +Both causes are fixed in the QUIC stack, in `moq-quic` (#5163) and in +moq-dev/noq ([#32](https://github.com/moq-dev/noq/pull/32) and +[#34](https://github.com/moq-dev/noq/pull/34), stacked). The drills still run +on `moq-noq`, and [the switch](/quest/m1/quic/fork/switch.md) is not started, +so what remains is to bump the `moq-noq-proto` pin to a release carrying both +noq PRs. Then loop the impaired drills under load (four to eight in parallel, +a few hundred runs each) and confirm that no connect fails. + +The causes, from qlogs of failing runs (2026-10-09). Both are the drills' 2s +client idle timeout meeting a backed-off PTO during the handshake. + +1. **A lost Finished waited on the Initial backoff** (`reset by peer`). + RFC 9002 A.11 resets `pto_count` when Initial or Handshake keys are + discarded; noq and quinn did not. The server, which cannot read 1-RTT + before the Finished, idled out first. Fix: reset `pto_count` in + `discard_space`. +2. **Two Initial flights per idle window** (`timed out`). The idle timer was + armed at `max(2s, 3 * 999ms)` = 2997ms, and the second probe is due at + 2997ms plus timer slop. Decided 2026-10-09: add a separate + `handshake_idle_timeout` (default 10s, as msquic does) that governs until + the handshake completes. Dead-peer detection on established connections is + unchanged. The rejected options were capping the handshake probe interval + (it touches the PTO deadline's monotonic invariant) and flooring the idle + timer at the backed-off PTO (it slows dead-peer detection). + +With the bump, a handshake to an unreachable relay waits 10s instead of about +3s. Check that the crash drill's reconnect loop (`dial()`'s 5s backoff +timeout) still holds. + +Public API: none in this repo; `moq-quic` and `moq-noq-proto` gain +`TransportConfig::handshake_idle_timeout`. Wire: none. ## Related -- [Switch](/quest/m1/quic/fork/switch.md) - moves the runtimes from noq to `moq-quic`, which already carries the cause 1 fix and is where a cause 2 fix may land instead +- [Switch](/quest/m1/quic/fork/switch.md) - moves the runtimes from noq to `moq-quic`, which already carries both fixes diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index 1632753e25..6e7419d90a 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -49,6 +49,8 @@ Changes on top of the upstream commit, besides the renames: Drop it if upstream lands [quinn#2748](https://github.com/quinn-rs/quinn/pull/2748) and we cherry-pick that. - Apple fast-path fallback preserves the segmentation of batches prepared before the fast path was disabled, including when a private symbol is unavailable. - Discarding Initial or Handshake keys resets the PTO backoff, as RFC 9002 A.11 does, so a client's Initial backoff no longer delays the retransmission of a lost Finished. Also [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); not offered upstream yet. +- `TransportConfig::handshake_idle_timeout` (default 10s, as msquic's `HandshakeIdleTimeoutMs`) governs the idle timer until the handshake completes, and the negotiated idle timeout takes over once established. A 2s idle timeout otherwise expires 3x the initial PTO in, just before the second Initial probe, leaving a client two flights. Also [moq-dev/noq#34](https://github.com/moq-dev/noq/pull/34). + quinn's `handshake_timeout` test in its async layer expects the handshake to end at the idle timeout, so the switch's import has to set both timeouts there. Recheck the BBR3 changes when cherry-picking quinn#2481 updates, or if it merges. diff --git a/rs/moq-quic/src/config/transport.rs b/rs/moq-quic/src/config/transport.rs index 5c580e7866..04b609dd1d 100644 --- a/rs/moq-quic/src/config/transport.rs +++ b/rs/moq-quic/src/config/transport.rs @@ -28,6 +28,7 @@ pub struct TransportConfig { pub(crate) max_concurrent_bidi_streams: VarInt, pub(crate) max_concurrent_uni_streams: VarInt, pub(crate) max_idle_timeout: Option, + pub(crate) handshake_idle_timeout: Duration, pub(crate) stream_receive_window: VarInt, pub(crate) receive_window: VarInt, pub(crate) send_window: u64, @@ -104,6 +105,16 @@ impl TransportConfig { self } + /// Maximum duration of inactivity to accept before abandoning a handshake. + /// + /// Used in place of [`max_idle_timeout`](Self::max_idle_timeout) until the handshake + /// completes, so a short idle timeout meant to detect a dead peer quickly does not also cut + /// a handshake off after a single probe. Defaults to 10 seconds. + pub fn handshake_idle_timeout(&mut self, value: Duration) -> &mut Self { + self.handshake_idle_timeout = value; + self + } + /// Maximum number of bytes the peer may transmit without acknowledgement on any one stream /// before becoming blocked. /// @@ -383,6 +394,8 @@ impl Default for TransportConfig { max_concurrent_uni_streams: 100u32.into(), // 30 second default recommended by RFC 9308 § 3.2 max_idle_timeout: Some(VarInt(30_000)), + // msquic's HandshakeIdleTimeoutMs default + handshake_idle_timeout: Duration::from_secs(10), stream_receive_window: STREAM_RWND.into(), receive_window: VarInt::MAX, send_window: (8 * STREAM_RWND).into(), @@ -422,6 +435,7 @@ impl fmt::Debug for TransportConfig { max_concurrent_bidi_streams, max_concurrent_uni_streams, max_idle_timeout, + handshake_idle_timeout, stream_receive_window, receive_window, send_window, @@ -452,6 +466,7 @@ impl fmt::Debug for TransportConfig { s.field("max_concurrent_bidi_streams", max_concurrent_bidi_streams) .field("max_concurrent_uni_streams", max_concurrent_uni_streams) .field("max_idle_timeout", max_idle_timeout) + .field("handshake_idle_timeout", handshake_idle_timeout) .field("stream_receive_window", stream_receive_window) .field("receive_window", receive_window) .field("send_window", send_window) diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index 275d9da76c..130243ba92 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -2116,13 +2116,15 @@ impl Connection { } fn reset_idle_timeout(&mut self, now: Instant, space: SpaceId) { - let Some(timeout) = self.idle_timeout else { - return; + let timeout = match &self.state { + state if state.is_closed() => None, + State::Handshake(_) => Some(self.config.handshake_idle_timeout), + _ => self.idle_timeout, }; - if self.state.is_closed() { + let Some(timeout) = timeout else { self.timers.stop(Timer::Idle); return; - } + }; let dt = cmp::max(timeout, 3 * self.pto(space)); self.timers.set(Timer::Idle, now + dt); } @@ -2821,6 +2823,8 @@ impl Connection { self.events.push_back(Event::Connected); self.state = State::Established; + // Swap the handshake idle timeout for the negotiated one. + self.reset_idle_timeout(now, SpaceId::Data); trace!("established"); Ok(()) } diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 6a3b4d2ba1..5a5c4a6da6 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -3264,6 +3264,48 @@ fn lost_finished_after_initial_backoff() { assert!(elapsed < Duration::from_secs(1), "{elapsed:?}"); } +/// A short idle timeout does not cut the handshake to two Initial flights. +/// +/// Before an RTT sample the PTO is 999ms, so a 2s idle timeout is armed at 2997ms, while +/// the second probe is due 999ms + 1998ms after the first send, plus however late the first +/// probe went out. The handshake idle timeout governs until the handshake completes. +#[test] +fn handshake_outlasts_short_idle_timeout() { + let _guard = subscribe(); + let mut pair = Pair::default(); + let mut transport = cubic_transport(); + transport.max_idle_timeout(Some(Duration::from_secs(2).try_into().unwrap())); + let mut client = client_config(); + client.transport_config(Arc::new(transport)); + + let client_ch = pair.begin_connect(client); + let start = pair.time; + // The first flight is lost, and so is the first probe, which goes out 2ms late. + pair.drive_client(); + pair.server.inbound.clear(); + pair.time = pair.client.next_wakeup().unwrap() + Duration::from_millis(2); + loop { + pair.drive_client(); + pair.server.inbound.clear(); + pair.time = pair.client.next_wakeup().unwrap(); + // Past any pacing: the second probe, due 2999ms after the first send, beyond the + // old 2997ms idle deadline. + if pair.time - start > Duration::from_secs(2) { + break; + } + } + + pair.drive(); + assert_matches!( + pair.client_conn_mut(client_ch).poll(), + Some(Event::HandshakeDataReady) + ); + assert_matches!( + pair.client_conn_mut(client_ch).poll(), + Some(Event::Connected) + ); +} + /// Ensures that the server can respond with 3 initial packets during the handshake /// before the anti-amplification limit kicks in when MTUs are similar. #[test] From 6d228d0b69421ae947da21264c912c8170b6679f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 20:24:34 -0700 Subject: [PATCH 4/7] test(moq-quic): an established connection idles out at the negotiated timeout Also document that the handshake idle timeout applies whatever the idle timeout or the peer's advertised one is. Co-Authored-By: Claude Opus 5.5 --- rs/moq-quic/src/config/transport.rs | 6 +++-- rs/moq-quic/src/tests/mod.rs | 35 +++++++++++++++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diff --git a/rs/moq-quic/src/config/transport.rs b/rs/moq-quic/src/config/transport.rs index 04b609dd1d..eb72d81385 100644 --- a/rs/moq-quic/src/config/transport.rs +++ b/rs/moq-quic/src/config/transport.rs @@ -109,7 +109,10 @@ impl TransportConfig { /// /// Used in place of [`max_idle_timeout`](Self::max_idle_timeout) until the handshake /// completes, so a short idle timeout meant to detect a dead peer quickly does not also cut - /// a handshake off after a single probe. Defaults to 10 seconds. + /// a handshake off after a single probe. It applies whether the idle timeout is shorter, + /// longer, or disabled, and whatever idle timeout the peer advertises; the negotiated idle + /// timeout takes over once the connection is established. Defaults to 10 seconds, as + /// msquic's `HandshakeIdleTimeoutMs`. pub fn handshake_idle_timeout(&mut self, value: Duration) -> &mut Self { self.handshake_idle_timeout = value; self @@ -394,7 +397,6 @@ impl Default for TransportConfig { max_concurrent_uni_streams: 100u32.into(), // 30 second default recommended by RFC 9308 § 3.2 max_idle_timeout: Some(VarInt(30_000)), - // msquic's HandshakeIdleTimeoutMs default handshake_idle_timeout: Duration::from_secs(10), stream_receive_window: STREAM_RWND.into(), receive_window: VarInt::MAX, diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 5a5c4a6da6..31d9512b7a 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -3306,6 +3306,41 @@ fn handshake_outlasts_short_idle_timeout() { ); } +/// Once established, a connection whose peer goes silent idles out at the negotiated idle +/// timeout, not the longer handshake idle timeout. +#[test] +fn established_idles_out_at_negotiated_timeout() { + let _guard = subscribe(); + let mut pair = Pair::default(); + let mut transport = cubic_transport(); + transport.max_idle_timeout(Some(Duration::from_secs(2).try_into().unwrap())); + let mut client = client_config(); + client.transport_config(Arc::new(transport)); + + let client_ch = pair.begin_connect(client); + loop { + pair.drive_client(); + if !pair.client_conn_mut(client_ch).is_handshaking() { + break; + } + pair.drive_server(); + pair.time = min_opt(pair.client.next_wakeup(), pair.server.next_wakeup()).unwrap(); + } + let established = pair.time; + + // Nothing reaches the client once it is established. + while !pair.client_conn_mut(client_ch).is_closed() { + pair.client.inbound.clear(); + if !pair.step() + && let Some(t) = pair.client.next_wakeup() + { + pair.time = t; + } + } + let elapsed = pair.time - established; + assert!(elapsed < Duration::from_secs(3), "{elapsed:?}"); +} + /// Ensures that the server can respond with 3 initial packets during the handshake /// before the anti-amplification limit kicks in when MTUs are similar. #[test] From 5e4ce04dd4501ffd77c5ad77a75031f965225ddc Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 21:31:17 -0700 Subject: [PATCH 5/7] fix(moq-quic): a deferred accept gets the handshake idle timeout The stale-Initial check in Endpoint::accept still used max_idle_timeout, so a server that deferred accepting past a short idle timeout refused a handshake the handshake idle timeout should still allow. Also bound the established idle test from both sides. Co-Authored-By: Claude Opus 5.5 --- rs/moq-quic/src/endpoint.rs | 13 ++++------- rs/moq-quic/src/tests/mod.rs | 45 ++++++++++++++++++++++++++++++++---- 2 files changed, 44 insertions(+), 14 deletions(-) diff --git a/rs/moq-quic/src/endpoint.rs b/rs/moq-quic/src/endpoint.rs index abc7686cd0..9a53066cbb 100644 --- a/rs/moq-quic/src/endpoint.rs +++ b/rs/moq-quic/src/endpoint.rs @@ -18,8 +18,8 @@ use thiserror::Error; use tracing::{debug, error, trace, warn}; use crate::{ - Duration, INITIAL_MTU, Instant, MAX_CID_SIZE, MIN_INITIAL_SIZE, RESET_TOKEN_SIZE, ResetToken, - Side, Transmit, TransportConfig, TransportError, + INITIAL_MTU, Instant, MAX_CID_SIZE, MIN_INITIAL_SIZE, RESET_TOKEN_SIZE, ResetToken, Side, + Transmit, TransportConfig, TransportError, cid_generator::ConnectionIdGenerator, coding::BufMutExt, config::{ClientConfig, EndpointConfig, ServerConfig}, @@ -599,13 +599,8 @@ impl Endpoint { .clone() }); - if server_config - .transport - .max_idle_timeout - .is_some_and(|timeout| { - incoming.received_at + Duration::from_millis(timeout.into()) <= now - }) - { + // Still handshaking, so the handshake idle timeout applies, not the idle timeout. + if incoming.received_at + server_config.transport.handshake_idle_timeout <= now { debug!("abandoning accept of stale initial"); self.ignore(incoming); return Err(Box::new(AcceptError { diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 31d9512b7a..1f8048e82e 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -1127,6 +1127,37 @@ fn zero_rtt_incoming_buffer_size_total() { }); } +/// A server that defers accepting an Initial has the handshake idle timeout to do so, not the +/// shorter idle timeout. +#[test] +fn deferred_accept_uses_handshake_idle_timeout() { + let _guard = subscribe(); + let mut transport = cubic_transport(); + transport.max_idle_timeout(Some(Duration::from_secs(2).try_into().unwrap())); + let mut server = server_config(); + server.transport_config(Arc::new(transport)); + let mut pair = Pair::new(Default::default(), server); + pair.server.handle_incoming = Box::new(|_| IncomingConnectionBehavior::Wait); + + pair.begin_connect(client_config()); + pair.begin_connect(client_config()); + pair.drive_client(); + pair.drive_server(); + let start = pair.time; + let late = pair.server.waiting_incoming.pop().unwrap(); + let delayed = pair.server.waiting_incoming.pop().unwrap(); + + // Past the idle timeout, but within the handshake idle timeout. + pair.time = start + Duration::from_secs(3); + assert!(pair.server.try_accept(delayed, pair.time).is_ok()); + + pair.time = start + Duration::from_secs(10); + assert_matches!( + pair.server.try_accept(late, pair.time), + Err(ConnectionError::TimedOut) + ); +} + /// Verify that datagrams arriving while a connection is in the `Accepting` state (between /// `start_accept` and `finish_accept`) are buffered in `incoming_buffers` and replayed into the /// connection after `finish_accept`. Drives through the full handshake and clean shutdown to @@ -3331,14 +3362,18 @@ fn established_idles_out_at_negotiated_timeout() { // Nothing reaches the client once it is established. while !pair.client_conn_mut(client_ch).is_closed() { pair.client.inbound.clear(); - if !pair.step() - && let Some(t) = pair.client.next_wakeup() - { - pair.time = t; + if !pair.step() && !pair.client_conn_mut(client_ch).is_closed() { + pair.time = pair + .client + .next_wakeup() + .expect("an open connection has an idle timer"); } } let elapsed = pair.time - established; - assert!(elapsed < Duration::from_secs(3), "{elapsed:?}"); + assert!( + (Duration::from_secs(2)..Duration::from_secs(3)).contains(&elapsed), + "{elapsed:?}" + ); } /// Ensures that the server can respond with 3 initial packets during the handshake From 37be2e800d9b33bd2ee8e9b10b975d9cd4203711 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 21:49:16 -0700 Subject: [PATCH 6/7] fix(moq-quic): an unrepresentable idle deadline never fires Ports moq-dev/noq#34's review fixes: a Duration::MAX handshake idle timeout no longer panics when added to an Instant, in the idle timer or the stale-Initial check. noq#34's third fix, the handshake idle timeout picking the PTO interval cap, has no counterpart: quinn's PTO backoff has no idle-based cap. Co-Authored-By: Claude Opus 5.5 --- rs/moq-quic/README.md | 2 +- rs/moq-quic/src/connection/mod.rs | 13 +++++++------ rs/moq-quic/src/endpoint.rs | 6 +++++- rs/moq-quic/src/tests/mod.rs | 16 ++++++++++++++++ 4 files changed, 29 insertions(+), 8 deletions(-) diff --git a/rs/moq-quic/README.md b/rs/moq-quic/README.md index 6e7419d90a..988e958e14 100644 --- a/rs/moq-quic/README.md +++ b/rs/moq-quic/README.md @@ -49,7 +49,7 @@ Changes on top of the upstream commit, besides the renames: Drop it if upstream lands [quinn#2748](https://github.com/quinn-rs/quinn/pull/2748) and we cherry-pick that. - Apple fast-path fallback preserves the segmentation of batches prepared before the fast path was disabled, including when a private symbol is unavailable. - Discarding Initial or Handshake keys resets the PTO backoff, as RFC 9002 A.11 does, so a client's Initial backoff no longer delays the retransmission of a lost Finished. Also [moq-dev/noq#32](https://github.com/moq-dev/noq/pull/32); not offered upstream yet. -- `TransportConfig::handshake_idle_timeout` (default 10s, as msquic's `HandshakeIdleTimeoutMs`) governs the idle timer until the handshake completes, and the negotiated idle timeout takes over once established. A 2s idle timeout otherwise expires 3x the initial PTO in, just before the second Initial probe, leaving a client two flights. Also [moq-dev/noq#34](https://github.com/moq-dev/noq/pull/34). +- `TransportConfig::handshake_idle_timeout` (default 10s, as msquic's `HandshakeIdleTimeoutMs`) governs the idle timer until the handshake completes, and the negotiated idle timeout takes over once established. A 2s idle timeout otherwise expires 3x the initial PTO in, just before the second Initial probe, leaving a client two flights. A server also drops an Initial left waiting for `accept` past it, and a deadline too far out to represent never fires instead of panicking. Also [moq-dev/noq#34](https://github.com/moq-dev/noq/pull/34), whose third change (the handshake idle timeout picks noq's idle-based PTO interval cap) has nothing to apply to here, since quinn's PTO backoff has no such cap. quinn's `handshake_timeout` test in its async layer expects the handshake to end at the idle timeout, so the switch's import has to set both timeouts there. Recheck the BBR3 changes when cherry-picking quinn#2481 updates, or if it merges. diff --git a/rs/moq-quic/src/connection/mod.rs b/rs/moq-quic/src/connection/mod.rs index 130243ba92..82c82d934a 100644 --- a/rs/moq-quic/src/connection/mod.rs +++ b/rs/moq-quic/src/connection/mod.rs @@ -2121,12 +2121,13 @@ impl Connection { State::Handshake(_) => Some(self.config.handshake_idle_timeout), _ => self.idle_timeout, }; - let Some(timeout) = timeout else { - self.timers.stop(Timer::Idle); - return; - }; - let dt = cmp::max(timeout, 3 * self.pto(space)); - self.timers.set(Timer::Idle, now + dt); + // A deadline too far out to represent never fires. + let deadline = + timeout.and_then(|timeout| now.checked_add(cmp::max(timeout, 3 * self.pto(space)))); + match deadline { + Some(deadline) => self.timers.set(Timer::Idle, deadline), + None => self.timers.stop(Timer::Idle), + } } fn reset_keep_alive(&mut self, now: Instant) { diff --git a/rs/moq-quic/src/endpoint.rs b/rs/moq-quic/src/endpoint.rs index 9a53066cbb..616afffd82 100644 --- a/rs/moq-quic/src/endpoint.rs +++ b/rs/moq-quic/src/endpoint.rs @@ -600,7 +600,11 @@ impl Endpoint { }); // Still handshaking, so the handshake idle timeout applies, not the idle timeout. - if incoming.received_at + server_config.transport.handshake_idle_timeout <= now { + if incoming + .received_at + .checked_add(server_config.transport.handshake_idle_timeout) + .is_some_and(|deadline| deadline <= now) + { debug!("abandoning accept of stale initial"); self.ignore(incoming); return Err(Box::new(AcceptError { diff --git a/rs/moq-quic/src/tests/mod.rs b/rs/moq-quic/src/tests/mod.rs index 1f8048e82e..925ab83b94 100644 --- a/rs/moq-quic/src/tests/mod.rs +++ b/rs/moq-quic/src/tests/mod.rs @@ -3337,6 +3337,22 @@ fn handshake_outlasts_short_idle_timeout() { ); } +/// A handshake idle timeout too long to add to an `Instant` never fires, instead of panicking. +#[test] +fn unbounded_handshake_idle_timeout() { + let _guard = subscribe(); + let mut transport = cubic_transport(); + transport.handshake_idle_timeout(Duration::MAX); + let transport = Arc::new(transport); + let mut server = server_config(); + server.transport = transport.clone(); + let mut client = client_config(); + client.transport_config(transport); + + let mut pair = Pair::new(Default::default(), server); + pair.connect_with(client); +} + /// Once established, a connection whose peer goes silent idles out at the negotiated idle /// timeout, not the longer handshake idle timeout. #[test] From d51546708c8b6ec34de7b1c9279eb2c781a0bf28 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 9 Oct 2026 22:05:30 -0700 Subject: [PATCH 7/7] quest: drop two Required blockers that already merged main's quest check fails since #5141 and #5138 deleted the quests these name. Same change as #5173, so the two merge cleanly. Co-Authored-By: Claude Opus 5.5 --- quest/m1/js-restart-keeps-request.md | 4 ---- quest/m1/webgpu-safari.md | 4 ---- 2 files changed, 8 deletions(-) diff --git a/quest/m1/js-restart-keeps-request.md b/quest/m1/js-restart-keeps-request.md index 2587b3a409..6992b22e9d 100644 --- a/quest/m1/js-restart-keeps-request.md +++ b/quest/m1/js-restart-keeps-request.md @@ -26,7 +26,3 @@ both languages already cover (`a_newer_epoch_leaves_the_broadcast_in_flight`). Add the test to `js/net/src/origin.test.ts`, failing before the fix. Public API: behavior only. Wire: none. - -## Required - -- [Dynamic epoch](/quest/m0/broadcast-epoch/dynamic-epoch.md) - #5141, which corrects the Rust `Dynamic::update` doc this quest aligns JS to diff --git a/quest/m1/webgpu-safari.md b/quest/m1/webgpu-safari.md index a37c812c4c..0089eeb11b 100644 --- a/quest/m1/webgpu-safari.md +++ b/quest/m1/webgpu-safari.md @@ -27,10 +27,6 @@ instead of guessing. Public API: possibly a new `RendererError` value. Wire: none. -## Required - -- [WebGPU renderer](/quest/m1/webgpu-renderer.md) - the renderer and `@moq/video` package (#5138) this verifies - ## Related - [WebGPU HDR](/quest/m1/webgpu-hdr.md) - builds on the same renderer