From 056a1c2792d8f99ac40fb9653ff44672ad13dab4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 10:41:21 +0200 Subject: [PATCH 1/2] Normalize WASI deadline-event accounting (I02) The WASI backends counted their own deadline wake as native work, so a timer-only wake reported zero zero-event OS waits and DESIGN section 10 rule 4a's no-spin bound could not see it. WASI 0.2 appends its deadline pollable after the owned handles, so its readiness has no owners entry: count only socket and DNS indices. WASI 0.3 joins a deadline subtask to the wait set: recognize an event whose waitable is that subtask, skip the operation scan for it and report the step as empty. Real socket, DNS and notifier events in the same call still count. Shared contract quiet_deadline_accounting proves, for the three design delays, that each of sixty expiries costs exactly one blocking wait that observed no native event, identically idle and with a registered-but-idle read, and that a call carrying real bytes is not counted empty. The quiet-deadline allocation gate now runs on both WASI targets and covers the registered-but-idle phase. --- CONTRIBUTING.md | 5 +- DESIGN.md | 2 +- crates/turnloop-contract/src/lib.rs | 162 ++++++++++++++++++ crates/turnloop-contract/tests/allocations.rs | 63 +++---- crates/turnloop-contract/tests/wasi.rs | 6 + crates/turnloop-contract/tests/windows.rs | 1 + crates/turnloop/src/backend/mod.rs | 13 +- crates/turnloop/src/backend/wasi_p2.rs | 13 +- crates/turnloop/src/backend/wasi_p3.rs | 14 +- docs/BACKEND_REVISION_2.md | 6 +- 10 files changed, 237 insertions(+), 48 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 4896763..cf9d96a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -120,7 +120,10 @@ work test `os_waits == 0` and `discovery_polls <= 1` (exactly 1 where the backen no cached work); require both zero without native operations. For all-call totals and bounds use their sum. Preserve the raw `zero_event_waits` counter across both categories and every §10 rule 4a no-spin bound. Callback/Event-helper queue draining -is neither kind of native call. +is neither kind of native call. A wake produced only by the backend's own deadline +source (timerfd, IOCP deadline packet, WASI deadline pollable/subtask) is a +zero-event call, not native work; simultaneous I/O or notifier events keep it +non-empty. See [Backend revision 2](docs/BACKEND_REVISION_2.md#blocking-waits-and-nonblocking-discovery-tl-i01b). ## Workspace discovery and test metadata diff --git a/DESIGN.md b/DESIGN.md index a449088..066e847 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -499,7 +499,7 @@ No other Perry change is needed for P0. *Queued work* means completions already queued before the native step: posts, blocking-pool and external-wait results, synchronous and terminal completions, including a timer's `Cancelled`/`Closed`; timer start, reset, cancel and close are therefore pure churn. A timer expiry that is due when the turn starts is not queued work: the effective wait is zero, so the turn may spend its one call on a zero-timeout poll. *Native operations* are operations submitted on native handles (sockets, pipes, files, processes, signals, web fetch/WebSocket), plus requests a backend accepts natively (WASI 0.2 DNS, and WASI filesystem requests, including path requests). Typed file requests on the native blocking pool are not native operations: their results arrive as queued work, like host blocking jobs. `TurnInfo::os_waits` counts blocking waits and `TurnInfo::discovery_polls` counts zero-timeout discovery polls; their sum is at most one per turn. Web callback draining and Windows Event-helper queue draining make neither kind of call. **Contract tests:** a queued post beside an idle UDP receive delivers with `os_waits == 0`, `discovery_polls == 1`, then byte-verified I/O; queued posts and timer `Cancelled`/`Closed` with no native operation record zero of both; a sustained producer keeps fresh I/O progressing within two seconds. 4. **Ticks:** no fixed-interval ticks and no minimum wait floor. -4a. **No spin.** A turn with nothing ready and a future deadline blocks until that deadline, at the precision in §7.6. It never returns immediately and never degrades into a zero-timeout poll loop. Hosts must pass exact deadlines (`Instant`, not truncated milliseconds). The Linux A/B measured what happens otherwise: 37,607 turns for 50 timers, ~70× user instructions. **Contract test:** with an idle registered socket and a 0.5 ms / 2 ms / 10 ms timer, turns per expiry ≤ 2 and zero-event OS waits ≤ 1 per expiry, on every backend. +4a. **No spin.** A turn with nothing ready and a future deadline blocks until that deadline, at the precision in §7.6. It never returns immediately and never degrades into a zero-timeout poll loop. Hosts must pass exact deadlines (`Instant`, not truncated milliseconds). The Linux A/B measured what happens otherwise: 37,607 turns for 50 timers, ~70× user instructions. A backend whose timeout is implemented by a private wakeup source — epoll's timerfd, the IOCP deadline packet, the WASI 0.2 deadline pollable, the WASI 0.3 deadline subtask — reports that wake as a **zero-event** wait, exactly as a timed OS wait that returned nothing; real I/O or notifier events arriving in the same call still make it non-empty. **Contract tests:** with an idle registered socket and a 0.5 ms / 2 ms / 10 ms timer, turns per expiry ≤ 2 and zero-event OS waits ≤ 1 per expiry, on every backend; and (tl-i02) each of the sixty expiries at those delays costs exactly one blocking wait that observed no native event, identically with the loop idle and with a registered-but-idle native operation, with no allocation, while a call that carries real bytes is not counted empty. 5. **Instruction budgets per operation** (Linux, cgu=1, `perf stat -e instructions:u,instructions:k`): TCP read / write / accept, timer start + cancel, notify + turn round trip, blocking job round trip, idle turn. **Values to be set from the attribution run of today's tokio bridge**, with a target below the tokio-bridge cost and within X % of a hand-written epoll loop. The CI gate compares against a committed baseline, with a control probe that must not move. **Rule 3 rationale (tl-i01b, spec-owner decision, 2026-09-15).** The original rule 3 ("none when completions are already queued") forbade even a zero-timeout poll. Backend revision 2 has no separate no-wait discovery primitive: `poll(Duration::ZERO)` is the only way to learn about fresh readiness or completions, and on Unix, after cached readiness reaches `EAGAIN`, only the poller marks a resource ready again. libuv and Node make the same trade: `uv_run` computes `uv_backend_timeout()`, which is zero while pending, idle or closing work exists, and still calls `uv__io_poll` with that zero timeout (see the libuv [loop API](https://docs.libuv.org/en/v1.x/loop.html) and [`src/unix/core.c`](https://github.com/libuv/libuv/blob/v1.x/src/unix/core.c)). The [tl-i01 probes](docs/lanes/tl-i01.md#evidence--specification-decision) showed that skipping discovery breaks the unchanged fairness contract: skipping the native step whenever work was queued failed the timer/I/O/post fairness test after two seconds, and a guard that only drained cached work delivered 64 posts with zero reads and failed the same fairness test. A separate no-wait collection API on all six backends is not justified before measurement. Rule 4a and its raw zero-event accounting are unchanged: an empty discovery poll still counts toward the same no-spin bound. diff --git a/crates/turnloop-contract/src/lib.rs b/crates/turnloop-contract/src/lib.rs index ecd4970..69e7a73 100644 --- a/crates/turnloop-contract/src/lib.rs +++ b/crates/turnloop-contract/src/lib.rs @@ -235,6 +235,10 @@ mod native { no_spin::(); } #[test] + fn quiet_deadlines_account_identically_idle_and_registered() { + quiet_deadline_accounting::(); + } + #[test] fn timer_bounds() { timer_precision::(); } @@ -1292,6 +1296,164 @@ pub fn io_and_posts_progress_with_repeating_timers() { assert_eq!((read, wrote), (64, 1)); } +/// Run the three design delays (0.5 ms, 2 ms, 10 ms) twenty times each against +/// `l`, reusing the caller's storage, and return +/// `[os_waits, discovery_polls, zero_event_waits, expiries]`. +/// +/// DESIGN §10 rules 3, 4 and 4a. A deadline with nothing else ready is *one* +/// native call that observes *no* native event, and the timer is the only +/// completion. A backend that reported its own private timeout source as native +/// work would record `zero_event_waits == 0` here: epoll normalizes its timerfd, +/// IOCP its deadline packet, WASI 0.2 its deadline pollable and WASI 0.3 its +/// deadline subtask. Each of the sixty measured expiries is a real blocking +/// wait; a round whose own setup outran the delay is already due at turn entry, +/// which DESIGN rule 3 lets spend its one call on a zero-timeout poll instead, +/// so it is asserted in that shape and re-run rather than measured (a cold or +/// preempted process does that a few times; sixteen consecutive failures mean +/// the delay itself can no longer be measured). `native_pending` states whether a +/// native operation is registered, the only thing allowed to differ afterwards: +/// the queued `Closed` turn may then spend its one call on a discovery poll. +pub fn quiet_deadlines( + l: &mut Driver, + out: &mut Completions, + native_pending: bool, +) -> [u32; 4] { + let mut totals = [0; 4]; + let mut late = Duration::ZERO; + for delay in [ + Duration::from_micros(500), + Duration::from_millis(2), + Duration::from_millis(10), + ] { + for round in 0..20 { + let token = Token(500 + delay.as_micros() as u64 + round); + let mut measured = false; + let mut attempt = 0; + while !measured { + attempt += 1; + assert!( + attempt <= 16, + "{delay:?}: the timer setup never finished inside its own delay" + ); + let at = l.now() + delay; + let h = l.timer(at, None, token).expect("timer"); + let op = l.timer_op(h).expect("timer operation"); + let info = l.turn(Timeout::Until(at), out).expect("quiet wait"); + // One native call, and it observed no native event at all. + assert_eq!( + (info.os_waits + info.discovery_polls, info.zero_event_waits), + (1, 1), + "{delay:?} deadline: one native call observing no native event" + ); + // A blocking wait is the measurable case. A preempted setup that + // outran its own delay leaves the expiry already due at turn + // entry, which DESIGN rule 3 lets spend the one call on a + // zero-timeout poll; that round is re-run, never counted, so a + // backend that only ever polls can never reach sixty expiries. + let quiet = info.os_waits == 1; + // Exactly one turn per expiry: the completion is already here. + assert_eq!(out.len(), 1, "{delay:?} deadline needed a second turn"); + assert_eq!( + (out[0].handle, out[0].op, out[0].token), + (Some(h), Some(op), token) + ); + assert!(matches!(out[0].result, OpResult::Timer)); + // The host deadline is honoured exactly, with no wait floor + // rounding it up: sub-millisecond delays expire like the others. + assert!(l.now() >= at, "{delay:?} deadline fired early"); + late = late.max(l.now().saturating_duration_since(at)); + assert!( + l.now().saturating_duration_since(at) < Duration::from_millis(100), + "{delay:?} deadline overshot its wait" + ); + if quiet { + totals[0] += info.os_waits; + totals[1] += info.discovery_polls; + totals[2] += info.zero_event_waits; + totals[3] += 1; + measured = true; + } + l.close(h, token).expect("close timer"); + let info = l.turn(Timeout::Now, out).expect("queued close"); + assert_eq!(info.os_waits, 0, "a queued close must not block"); + assert!( + info.discovery_polls <= u32::from(native_pending), + "queued close polled without a pending native operation" + ); + assert!(info.zero_event_waits <= info.discovery_polls); + assert_eq!(out.len(), 1); + assert!(matches!(out[0].result, OpResult::Closed)); + } + } + } + // Never print here: allocation gates call this with their counter armed. + assert!(late < Duration::from_millis(100)); + totals +} + +/// DESIGN §10 rule 4a and the `PollInfo` contract: quiet deadline accounting is +/// the same whether the loop is idle or has registered-but-idle native services. +pub fn quiet_deadline_accounting() { + let mut l = Driver::::new(Config::default()).expect("loop"); + let mut out = Completions::with_capacity(1); + let idle = quiet_deadlines(&mut l, &mut out, false); + let (_, sender, receiver) = pair(&mut l); + let read = l + .read(receiver, ReadBuf::Pooled, Token(2)) + .expect("idle read"); + // Submit the read before measuring: the registered phase must differ from the + // idle one by a pending native operation only, not by a first-submission turn. + let info = l.turn(Timeout::Now, &mut out).expect("arm the idle read"); + assert!(out.is_empty(), "the registered read must stay idle"); + assert_eq!(info.os_waits, 0, "a zero timeout never blocks"); + let registered = quiet_deadlines(&mut l, &mut out, true); + assert_eq!( + idle, registered, + "registered-but-idle services changed quiet-deadline accounting" + ); + assert_eq!( + idle, + [60, 0, 60, 60], + "sixty expiries, each one blocking wait that saw no native event" + ); + // The same registered read now takes real bytes inside a deadline-bounded + // wait. Normalizing the private deadline must not empty a call that found + // native work: this is the control the accounting above needs. + l.write(sender, WriteBuf::Owned(vec![0x2a]), Token(3)) + .expect("write"); + let until = l.now() + Duration::from_secs(5); + let (mut calls, mut empty, mut bytes, mut wrote) = (0, 0, 0, 0); + while bytes == 0 || wrote == 0 { + assert!(l.now() < until, "the loopback exchange stalled"); + let info = l.turn(Timeout::Until(until), &mut out).expect("exchange"); + assert!(info.os_waits + info.discovery_polls <= 1); + calls += info.os_waits + info.discovery_polls; + empty += info.zero_event_waits; + for c in out.drain() { + match c.result { + OpResult::Wrote(1) => wrote += 1, + OpResult::Read { + n: 1, + lease: Some(b), + } => { + assert_eq!(c.op, Some(read), "the idle read survived every expiry"); + assert_eq!(b.as_slice(), [0x2a]); + bytes += 1; + } + other => panic!("unexpected {other:?}"), + } + } + } + assert!(calls > 0, "the exchange must make a native call"); + assert_eq!( + empty, 0, + "a native call that carried real I/O is not a zero-event wait" + ); + eprintln!( + "quiet deadlines idle={idle:?} registered={registered:?}; exchange calls={calls} empty={empty}" + ); +} + /// DESIGN §10 rule 4a: an idle registered stream must not turn timers into polling. pub fn no_spin() { let mut driver = Driver::::new(Config::default()).expect("loop"); diff --git a/crates/turnloop-contract/tests/allocations.rs b/crates/turnloop-contract/tests/allocations.rs index c4143f1..e4b273b 100644 --- a/crates/turnloop-contract/tests/allocations.rs +++ b/crates/turnloop-contract/tests/allocations.rs @@ -1107,54 +1107,43 @@ fn signal_exit_and_external_notification_delivery_allocate_nothing() { l.signal_stop(signal, Token(4)).expect("stop"); } -#[cfg(not(target_os = "wasi"))] #[test] fn quiet_deadline_waits_have_identical_accounting_without_allocations() { let mut l = Loop::new(Config::default()).expect("loop"); let mut out = Completions::with_capacity(1); - let mut expiries = 0; ALLOCS.with(|v| v.set(0)); ACTIVE.with(|v| v.set(true)); - for duration in [ - Duration::from_micros(500), - Duration::from_millis(2), - Duration::from_millis(10), - ] { - for _ in 0..20 { - let at = l.now() + duration; - let token = Token(expiries); - let h = l.timer(at, None, token).expect("timer"); - let op = l.timer_op(h).expect("timer operation"); - let info = l.turn(Timeout::Until(at), &mut out).expect("quiet wait"); - assert_eq!( - (info.os_waits, info.discovery_polls, info.zero_event_waits), - (1, 0, 1) - ); - assert!(l.now() >= at); - assert_eq!(out.len(), 1); - assert_eq!( - (out[0].handle, out[0].op, out[0].token), - (Some(h), Some(op), token) - ); - assert!(matches!(out[0].result, OpResult::Timer)); - l.close(h, token).expect("close timer"); - let info = l.turn(Timeout::Now, &mut out).expect("queued close"); - assert_eq!( - (info.os_waits, info.discovery_polls, info.zero_event_waits), - (0, 0, 0) - ); - assert_eq!(out.len(), 1); - assert!(matches!(out[0].result, OpResult::Closed)); - expiries += 1; - } - } + let idle = turnloop_contract::quiet_deadlines(&mut l, &mut out, false); + ACTIVE.with(|v| v.set(false)); + assert_eq!( + ALLOCS.with(Cell::get), + 0, + "idle deadline waits reuse reserved storage" + ); + // Registering an idle native operation must not change the accounting, and + // must not make the same deadline waits allocate. + let (_, _sender, receiver) = turnloop_contract::pair(&mut l); + let read = l + .read(receiver, ReadBuf::Pooled, Token(2)) + .expect("idle read"); + l.turn(Timeout::Now, &mut out).expect("arm the idle read"); + assert!(out.is_empty(), "the registered read must stay idle"); + ALLOCS.with(|v| v.set(0)); + ACTIVE.with(|v| v.set(true)); + let registered = turnloop_contract::quiet_deadlines(&mut l, &mut out, true); ACTIVE.with(|v| v.set(false)); - assert_eq!(expiries, 60); + assert_eq!( + idle, registered, + "registered-but-idle services changed quiet-deadline accounting" + ); + assert_eq!(idle, [60, 0, 60, 60]); + eprintln!("quiet deadline allocations: idle={idle:?} registered={registered:?}"); assert_eq!( ALLOCS.with(Cell::get), 0, - "deadline waits reuse reserved storage" + "deadline waits beside an idle native operation reuse reserved storage" ); + assert!(l.cancel(read), "the registered read stayed pending"); } #[cfg(target_os = "wasi")] diff --git a/crates/turnloop-contract/tests/wasi.rs b/crates/turnloop-contract/tests/wasi.rs index 9447326..999f818 100644 --- a/crates/turnloop-contract/tests/wasi.rs +++ b/crates/turnloop-contract/tests/wasi.rs @@ -141,6 +141,12 @@ fn timers_io_posts() { fn no_spin() { contract::no_spin::(); } +/// DESIGN §10.4a and the `PollInfo` contract: the private WASI deadline +/// pollable (0.2) / subtask (0.3) is this wait's timeout, not native work. +#[test] +fn quiet_deadline_accounting() { + contract::quiet_deadline_accounting::(); +} #[test] fn repeated_eof_shutdown_and_empty_datagram() { diff --git a/crates/turnloop-contract/tests/windows.rs b/crates/turnloop-contract/tests/windows.rs index 2b1b998..384a0de 100644 --- a/crates/turnloop-contract/tests/windows.rs +++ b/crates/turnloop-contract/tests/windows.rs @@ -17,6 +17,7 @@ contract!( refused_connect_once, ref_unref, no_spin, + quiet_deadline_accounting, timer_precision, udp_round_trip, detach_inflight, diff --git a/crates/turnloop/src/backend/mod.rs b/crates/turnloop/src/backend/mod.rs index 1f86321..5c386f1 100644 --- a/crates/turnloop/src/backend/mod.rs +++ b/crates/turnloop/src/backend/mod.rs @@ -49,8 +49,12 @@ //! counts positive-timeout/infinite waits; `discovery_polls` counts zero-time //! native polls. Their sum is at most one. `zero_event_waits` //! counts raw empty calls across both counters, including EINTR. -//! Private timeout events (e.g. timerfd expiry) count as zero-event waits, just -//! like a timed OS wait returning zero; never infer this from user completions. +//! Private timeout events count as zero-event waits, just like a timed OS wait +//! returning zero; never infer this from user completions. Every backend that +//! implements its timeout with a private wakeup source normalizes it: epoll's +//! timerfd, the IOCP deadline packet, the WASI 0.2 deadline pollable and the +//! WASI 0.3 deadline subtask. Real I/O or notifier events arriving in the same +//! call still make it non-empty. //! * Cached readiness ending in EAGAIN with no completion must retain the original //! timeout for the one permitted OS wait. It must not force a zero-timeout turn. //! * `has_work` covers queued completions and cached runnable I/O. `wake` is called @@ -253,7 +257,10 @@ pub struct PollInfo { pub discovery_polls: u32, /// Native wait/discovery invocations with no I/O or notifier events, including EINTR. /// Bounded by `waits + discovery_polls`; retains revision-2 raw empty-poll accounting. - /// Private timeout events, such as timerfd expiry, are not native work. + /// Private timeout events are not native work: epoll's timerfd, the IOCP + /// deadline packet, the WASI 0.2 deadline pollable and the WASI 0.3 deadline + /// subtask all count as zero-event calls, while simultaneous I/O or notifier + /// events keep the call non-empty. pub zero_event_waits: u32, } diff --git a/crates/turnloop/src/backend/wasi_p2.rs b/crates/turnloop/src/backend/wasi_p2.rs index 0eec990..4a6b3a5 100644 --- a/crates/turnloop/src/backend/wasi_p2.rs +++ b/crates/turnloop/src/backend/wasi_p2.rs @@ -675,6 +675,13 @@ unsafe impl Backend for WasiP2 { return Err(Error::new(ErrorKind::Unsupported)); } abi::poll(&self.handles, &mut self.poll_storage, &mut self.indices); + // The private deadline pollable implements this wait's timeout, so its + // readiness has the meaning of the poll returning nothing (see `PollInfo` + // in backend/mod.rs, epoll's timerfd and the IOCP deadline packet): it is + // not native work. It is the only handle without an `owners` entry, so + // socket and DNS readiness alone counts, including when they arrive in + // the same result as the expiry. + let mut native_events = 0; for i in 0..self.indices.len() { match self.owners.get(self.indices[i]).copied() { Some(PollOwner::Socket(h, d)) => { @@ -683,16 +690,18 @@ unsafe impl Backend for WasiP2 { .expect("live poll owner") .ready[d] = true; self.schedule(h); + native_events += 1; } Some(PollOwner::Dns(i)) => { - self.lookups[i].as_mut().expect("live lookup").ready = true + self.lookups[i].as_mut().expect("live lookup").ready = true; + native_events += 1; } None => {} } } self.run_ready(events); self.run_dns(events); - Ok(PollInfo::native(timeout, self.indices.is_empty())) + Ok(PollInfo::native(timeout, native_events == 0)) } fn release(&mut self, h: Handle) { self.files.release(h); diff --git a/crates/turnloop/src/backend/wasi_p3.rs b/crates/turnloop/src/backend/wasi_p3.rs index bdb7f83..5e24d98 100644 --- a/crates/turnloop/src/backend/wasi_p3.rs +++ b/crates/turnloop/src/backend/wasi_p3.rs @@ -595,11 +595,19 @@ unsafe impl Backend for WasiP3 { } } let (kind, waitable, code) = self.wait_set.step(blocking); + // The private deadline subtask implements this wait's timeout, so its + // completion has the meaning of the step returning no event (see + // `PollInfo` in backend/mod.rs, epoll's timerfd and the IOCP deadline + // packet): it is not native work. Socket subtask events still count, and + // a step that returns one leaves any simultaneous expiry for the timer + // wheel, which reads the clock rather than this event. + let deadline_event = + kind != 0 && deadline.is_some_and(|(task, _)| task != 0 && waitable == task); if let Some((task, initial)) = deadline && task != 0 { self.wait_set.remove(task); - if initial < 2 && !(kind != 0 && waitable == task && code >= 2) { + if initial < 2 && !(deadline_event && code >= 2) { // SAFETY: synchronously cancel this turn-owned deadline subtask before dropping it. unsafe { wait_set::subtask_cancel(task); @@ -610,7 +618,7 @@ unsafe impl Backend for WasiP3 { wait_set::subtask_drop(task); } } - if kind != 0 { + if kind != 0 && !deadline_event { for i in 0..self.ops.len() { let Some(p) = &mut self.ops[i] else { continue; @@ -628,7 +636,7 @@ unsafe impl Backend for WasiP3 { self.run_ready(events); Ok(PollInfo::native( if blocking { None } else { Some(Duration::ZERO) }, - kind == 0, + kind == 0 || deadline_event, )) } fn release(&mut self, h: Handle) { diff --git a/docs/BACKEND_REVISION_2.md b/docs/BACKEND_REVISION_2.md index 612fea8..8dfaff4 100644 --- a/docs/BACKEND_REVISION_2.md +++ b/docs/BACKEND_REVISION_2.md @@ -71,7 +71,11 @@ all previous numerical bounds and count both invocation categories. Epoll (including timerfd), kqueue, direct IOCP and WASI p2 classify the effective native timeout. WASI p3 classifies its actual wait-set step (wait versus poll), -including a deadline already completed during setup. Its existing cooperative +including a deadline already completed during setup. A backend's own deadline +source is never native work (tl-i02): epoll drops its timerfd readiness, IOCP its +deadline packet, WASI p2 the deadline pollable it appends after the owned handles +and WASI p3 the deadline subtask it joins to the wait set, so a timeout-only wake +is a zero-event call. Socket, DNS and notifier events in the same call still count. Its existing cooperative host yield remains part of discovery; the documented experimental scheduler limitations remain. Web callback draining and IOCP Event-helper queue draining report zero in both counters: neither performs a native wait/discovery call on From dd93fa96dd572ae79419d32e65bf3eb6e342b694 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Tue, 15 Sep 2026 11:02:03 +0200 Subject: [PATCH 2/2] tl-i02: lane report with root causes, mutation evidence and verification ledger --- docs/lanes/tl-i02.md | 208 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 docs/lanes/tl-i02.md diff --git a/docs/lanes/tl-i02.md b/docs/lanes/tl-i02.md new file mode 100644 index 0000000..0edfb8d --- /dev/null +++ b/docs/lanes/tl-i02.md @@ -0,0 +1,208 @@ +# tl-i02 — WASI deadline-event accounting + +Status: **implemented; macOS arm64 and Wasmtime 46 (p2 debug/release, p3 +debug/release) verified locally. Linux, Windows and web runtime await the +integrator's CI.** Base `3cce439` (main), branch `lane/tl-i02`, commit `056a1c2`. +Root `LANE_REPORT.md` and the audit documents are byte-identical to `3cce439`. + +This is an accounting defect, as the audit states. No WASI test spun before the +fix; the counter simply could not report a timeout-only wake, so DESIGN §10 rule +4a's "zero-event OS waits ≤ 1 per expiry" bound was vacuous on both WASI +backends: the quantity it bounds was always zero. + +## Root cause per backend + +**WASI 0.2** (`crates/turnloop/src/backend/wasi_p2.rs`). `poll` builds a parallel +pair of vectors: `handles` (the pollables passed to `wasi:io/poll`) and `owners` +(one `PollOwner` per *owned* handle — a socket direction or a DNS lookup). The +private deadline pollable is pushed onto `handles` **after** the owned ones and +deliberately gets no `owners` entry, so the dispatch loop ignores it via +`self.owners.get(index) => None`. The `PollInfo` argument, however, was +`self.indices.is_empty()`: every returned index counted, the deadline's included. +A quiet expiry returned exactly one index — the deadline — so the wait reported +`zero_event_waits == 0`, i.e. "this wait found native work". + +**WASI 0.3** (`crates/turnloop/src/backend/wasi_p3.rs`). `poll` lowers +`wasi:clocks/monotonic-clock.wait-until` into a subtask and, when it does not +complete during setup, joins that subtask to the wait set. The step's +`empty` argument was `kind == 0`, i.e. "the wait set returned no event at all". +A quiet expiry returns exactly one event whose waitable *is* the private deadline +subtask, so `kind != 0` and the wait again reported `zero_event_waits == 0`. The +code already knew how to recognize that event — it used +`kind != 0 && waitable == task && code >= 2` to decide whether the subtask still +needed cancelling — but never fed that knowledge to `PollInfo`. + +Native IOCP (`backend/iocp/mod.rs:1437`) and epoll's timerfd branch +(`backend/epoll.rs:241`) already normalize their own deadline source; kqueue's +timeout is the `timespec`, which never produces an event. Only the two WASI +backends were out of line with the `PollInfo` contract in `backend/mod.rs`. + +## The fix + +- **p2** (`wasi_p2.rs:684,693,697,704`): count native events in the existing + dispatch loop — one per index that resolves to a `PollOwner::Socket` or + `PollOwner::Dns` — and pass `native_events == 0` as `empty`. The deadline is + the only handle without an owner, so it is excluded by construction, and a + socket or DNS index arriving *in the same result* as the expiry still makes the + call non-empty. No extra pass, no allocation, no change to dispatch. +- **p3** (`wasi_p3.rs:604,610,621,639`): hoist the existing recognition into + `deadline_event` (`kind != 0` and the event's waitable is this turn's deadline + subtask), reuse it for the unchanged cancellation decision, skip the operation + scan for it (the deadline matches no operation, so this is equivalent and + cheaper), and pass `kind == 0 || deadline_event` as `empty`. A step that + returns a socket subtask event is still native work; any simultaneous expiry is + left to the timer wheel, which reads the clock rather than this event. +- Contract text updated where it is stated: `backend/mod.rs` module rustdoc and + the `PollInfo::zero_event_waits` doc now name all four private deadline sources + (timerfd, IOCP deadline packet, WASI 0.2 pollable, WASI 0.3 subtask) and say + that simultaneous I/O or notifier events keep the call non-empty. The same + sentence is restated in `DESIGN.md` §10 rule 4a, `docs/BACKEND_REVISION_2.md` + and `CONTRIBUTING.md`. + +Nothing in the driver changed. `PollInfo::native`'s `waits`/`discovery_polls` +classification from tl-i01b is untouched: a private deadline wake is an empty +*blocking* wait, which is exactly what the split already expresses. + +## Tests + +**Shared contract** `turnloop_contract::quiet_deadlines` (`src/lib.rs:1316`) and +`quiet_deadline_accounting` (`src/lib.rs:1396`). + +`quiet_deadlines(l, out, native_pending)` runs the three design delays +(0.5 ms / 2 ms / 10 ms) twenty times each against the caller's loop and +completion storage and returns `[os_waits, discovery_polls, zero_event_waits, +expiries]`. Per round it requires: + +- `os_waits + discovery_polls == 1` and `zero_event_waits == 1` — exactly one + native call, and it observed no native event. **This is the I02 assertion**; + both unfixed backends report `zero_event_waits == 0` here. +- exactly one turn per expiry (`out.len() == 1` immediately after the wait), with + the timer's handle, operation id and token, and `OpResult::Timer`; +- the deadline honoured exactly — `now >= at`, with lateness under 100 ms (the + bound `timer_precision` already uses), so no wait floor rounds a 0.5 ms delay + up and no wait returns early; +- the queued `Closed` turn makes no blocking wait, and a discovery poll only when + a native operation is pending (`discovery_polls <= native_pending`), with + `zero_event_waits <= discovery_polls`. + +A round whose own setup is preempted past its delay leaves the expiry already due +at turn entry, which DESIGN §10 rule 3 lets spend the turn's one call on a +zero-timeout poll. Such a round still asserts the full I02 property (one call, +zero-event) but is **re-run instead of counted**, bounded at 16 consecutive +attempts. That is why the totals can be required exactly rather than as a `≤` +bound: a backend that only ever polls can never reach sixty blocking waits, and +16 consecutive failures at one delay fail the test outright. This matters in +practice — under an 8-way CPU load the p3 release binary hits a due-at-entry +round about 20 % of the time at 0.5 ms; a pre-turn `now < at` prediction was +raced and discarded (measured: 2/10 runs failed with `(1, 1, 0)`). + +`quiet_deadline_accounting` runs that helper twice on one loop — idle, then with +a registered-but-idle pooled read on a loopback pair — asserts the two results +are **identical** and equal to `[60, 0, 60, 60]`, and then finishes with the +control the accounting needs: the *same* still-pending read takes real bytes +inside a deadline-bounded wait, and the exchange must make at least one native +call with **zero** zero-event waits. Both phases and the exchange are exercised +on every backend that instantiates it. + +Instantiated at: + +| Suite | Test | +|---|---| +| `crates/turnloop-contract/src/lib.rs:238` (native unit tests) | `native::quiet_deadlines_account_identically_idle_and_registered` | +| `crates/turnloop-contract/tests/wasi.rs:147` (p2 and p3, debug and release) | `quiet_deadline_accounting` | +| `crates/turnloop-contract/tests/windows.rs:20` | `quiet_deadline_accounting` | + +**Allocation gate.** `quiet_deadline_waits_have_identical_accounting_without_allocations` +(`tests/allocations.rs:1111`) lost its `#[cfg(not(target_os = "wasi"))]` and now +runs on both WASI targets as well as the three native OSes. Its body now calls +the shared helper twice under the counting allocator — idle, then with a +registered-but-idle read — and asserts identical `[60, 0, 60, 60]` results and +zero allocations in **both** phases. Its previous assertions are preserved +exactly: `(os_waits, discovery_polls, zero_event_waits) == (1, 0, 1)` per expiry +(now expressed as one native call plus `os_waits == 1` for every counted round) +and `(0, 0, 0)` for the queued close in the idle phase. + +No test was weakened. No numeric bound moved. `no_spin`'s `≤ 2 turns` / +`≤ 1 zero-event wait` per expiry is unchanged and now measures a live quantity on +WASI; so do `external_wait_deadlines_do_not_spin` (`tests/wasi.rs`), the +`turnloop-io` lingering-close bounds and the HTTP keep-alive bounds — all of them +saw `zero_event_waits == 0` from WASI before and now see the real count. + +### Both failure directions proved + +Every mutation below was applied to the committed tree, built and run; the fix +was restored immediately afterwards. + +| Mutation | Suite | Result | +|---|---|---| +| p2 `native_events == 0` → `self.indices.is_empty()` (the pre-fix expression) | `wasi` contract, wasm32-wasip2 debug | **FAIL as intended**: `500µs deadline: one native call observing no native event; left: (1, 0) right: (1, 1)` | +| same | `allocations`, wasm32-wasip2 release | **FAIL as intended**, same assertion | +| p3 `kind == 0 \|\| deadline_event` → `kind == 0` (the pre-fix expression) | `wasi` contract, wasm32-wasip3 debug | **FAIL as intended**, same assertion | +| same | `allocations`, wasm32-wasip3 release | **FAIL as intended**, same assertion | +| p2 and p3 `empty` → `true` (over-normalize: call every wait empty) | `wasi` contract, both targets | **FAIL as intended**: `a native call that carried real I/O is not a zero-event wait; left: 1 right: 0` | + +The first four prove the tests fail without the fix. The last proves they are not +satisfied by blanket-emptying every call, i.e. the backends still distinguish +simultaneous real I/O. + +## Verification + +Tree `056a1c2`, macOS arm64, `CARGO_BUILD_JOBS=4`. Plain `cargo` is +nightly-2026-08-20; WASI 0.3 uses nightly-2026-09-07. wasmtime 46.0.0 (the +`scripts/ci/tools.json` pin) was linked into this clone's `.tools/bin` from an +existing local install. + +| Command | Result | +|---|---| +| `cargo fmt --check` | **PASS** | +| `python3 scripts/ci/check-paths.py` | **PASS** (1490 tracked files, 257 references) | +| `python3 scripts/ci/feature_modes.py` | **PASS** (6 features / 18 native arms) | +| `git diff --check 3cce439..HEAD` | **PASS** | +| `cargo clippy --locked --workspace --all-targets -- -D warnings -D clippy::undocumented_unsafe_blocks` | **PASS** | +| same with `--all-features` | **PASS** | +| same strict flags, `-p turnloop -p turnloop-contract -p turnloop-io --all-targets`: x86_64-unknown-linux-gnu (default, `turnloop/epoll-timerfd`, all-features), x86_64-pc-windows-msvc (default, all-features), wasm32-wasip2 (default, all-features), wasm32-unknown-unknown (default, all-features), wasm32-wasip3 all-features | **PASS** (10 invocations) | +| wasm32-wasip3 **default** features, same package set | **FAIL, inherited**: `method read_buffer is never used` in `turnloop` (lib). Reproduced on unmodified `3cce439` in a scratch worktree; CI clippies p3 with `--all-features`, which passes | +| `cargo +stable check --locked --workspace --all-targets --all-features` | **PASS** | +| `RUSTDOCFLAGS=-Dwarnings cargo doc --locked --workspace --all-features --no-deps` | **PASS** | +| `cargo test --locked --workspace -- --test-threads=1` | **PASS**, exit 0, 285 tests | +| `python3 scripts/ci/run-tests.py native` (default, executor, all-features; workspace + every member) | **PASS**, exit 0, 1645 tests | +| `python3 scripts/ci/run-tests.py wasi --target wasm32-wasip2` | **PASS**, exit 0: core lib 11, contracts 34 debug + 34 release, allocations 12 | +| `python3 scripts/ci/run-tests.py wasi --target wasm32-wasip3` | **PASS**, exit 0: core lib 13, contracts 34 + 34, allocations 13 | +| `bash scripts/ci/no-tokio.sh` | **PASS** (all graphs, default + all features) | +| `python3 scripts/ci/soak.py` | **PASS**: 251 locked versions; inherited rustls 0.23.45 exception only, expires 2026-09-21 | +| Stability: new contract test, p2 debug wasm, 12 runs under 8-way CPU load | **PASS** 12/12 | +| Stability: allocation gate, p3 release wasm, 15 runs under 8-way CPU load | **PASS** 15/15 (2/10 before the due-at-entry re-run was added) | +| Stability: native contract test and native allocation gate, 15 runs each under load | **PASS** 30/30 | +| `python3 scripts/ci/run-tests.py native` on Linux x86_64/arm64 (all modes incl. timerfd/SIGCHLD fallbacks) | **UNRUN**, no host | +| `python3 scripts/ci/run-tests.py native` on Windows (incl. the new `quiet_deadline_accounting` arm) | **UNRUN**, no host | +| `python3 scripts/ci/run-tests.py web` / `node` | **UNRUN**: `wasm-bindgen-test-runner` not installed. The web contract lists are unchanged; cross-clippy for wasm32-unknown-unknown compiles the new code | +| `python3 scripts/ci/run-tests.py protocol` / `protocol-wasi` with real servers | **UNRUN**, not affected by the change | +| `cargo run --release -p turnloop-bench -- --portable` | **UNRUN**, not affected: the bench runs on the native backend, which this change does not touch | +| `python3 scripts/ci/check-ci.py` | **UNRUN**: requires `GH_TOKEN` | + +## Needs CI confirmation / notes for the integrator + +- **Windows:** `quiet_deadline_accounting` is new in the `contract!` list in + `tests/windows.rs` and is compiled by cross-clippy only here. IOCP already + normalizes its deadline packet, so the expected result is a pass; the arm that + needs a real look is the Event-helper mode, where a turn makes neither kind of + call and the helper's own wait is outside the turn — that mode never reaches + `quiet_deadlines`' blocking-wait branch, and if a GUI-host configuration does, + its counts are the thing to read. +- **Linux:** both epoll modes. The timerfd fallback normalizes its private timer + already (`epoll.rs:241`); `epoll_pwait2` returns zero on timeout, so both + should give `[60, 0, 60, 60]`. This is the first test that requires that + exactly, per expiry, rather than as a `≤` bound. +- **The audit documents were deliberately not edited.** `docs/DESIGN_AUDIT.md:141` + still records the defect ("WASI counters count private deadline events as + native work (`wasi_p2.rs:684`, `wasi_p3.rs:619`), contrary to + `backend/mod.rs:204`") and `docs/REMAINING_WORK.md` still carries I02 in its + audited form, following the tl-i01b precedent of leaving the audit as written. + Both rows are now stale and are the integrator's to refresh. +- **Not in scope, observed while reading p3:** when a socket subtask event and + the deadline complete in the same wait-set step, the step returns one of them. + If it returns the socket event, the expiry is still delivered in the same turn + by the timer wheel (it reads the clock, not the event), and the call correctly + counts as native work. No fairness question arises at the two-event scale, but + a deadline storm against many ready sockets is I04's territory, not measured + here.