feat(net)!: datagrams are unfetchable and bounded by the group range

[kixelated]

Completes quest/m1/datagram-unfetchable.md (deleted here), minus its "a new subscription never replays buffered datagrams" rule, which the maintainer dropped (decision 2).

Problem

A group range did not bound datagrams, and a datagram-flagged IETF fetch object failed with Unsupported. Nothing said a FETCH can never return a datagram.

Approach

• Range bounds datagrams (Rust + @moq/net). The cursor's floor and cap filter datagrams as they filter groups. A start frame above 0 drops the start group's datagram (its only frame). A datagram is judged when the reader reaches it, against the range in force then. An out-of-range one is dropped for good, never parked like a group past the cap.
• Replay is allowed, not promised. A new subscriber may still read the datagrams in the track's 64-entry send buffer, as on main. No tick, no age bound (decisions 2 and 11).
• FETCH never returns a datagram, and says so where it can.
  • On receive, a draft-16+ IETF fetch object flagged as a datagram fails only that fetch or fill with the new Error::NotFetchable, checked before any payload is read. An absolute joining FETCH, which spans whole groups, instead finishes the group before the datagram, skips every object of the datagram group (judged by its resolved Group ID), and keeps filling the groups after it. A datagram inside the head's own group is refused as NotFetchable and the partial head aborted.
  • A track with no fetch handler (an origin) answers a FETCH for a sequence still in its datagram send buffer with NotFetchable instead of NotFound. Past the buffer, or on a relay whose upstream answers, a miss stays NotFound.
  • New moq-lite-07 stream code 0x3A NOT_FETCHABLE. Earlier lite versions are sent NOT_FOUND and do not decode 0x3A. IETF keeps DOES_NOT_EXIST.
• Draft draft-lcurley-moq-lite: Datagrams section (range bound, never cached or fetched, a new subscription MAY get the send buffer), NOT_FETCHABLE row, moq-lite-07 changelog bullets. doc/concept/moq-lite.md and doc/concept/standard.md updated.

Impact

• Wire (moq-lite-07-wip only): new stream reset code 0x3A NOT_FETCHABLE, sent on a FETCH stream that reached a datagram. Lite-05/06 peers get NOT_FOUND (0x33) as before, and a received 0x3A stays StreamError::Unknown there. No change to published versions.
• Wire (IETF): none. A refused datagram FETCH is DOES_NOT_EXIST (0x10, draft-14 0x4), including a lite-07 upstream's NOT_FETCHABLE bridged by a relay. On an IETF stream reset it is INTERNAL_ERROR, like any lite-only code.
• New public Rust variants: moq_net::Error::NotFetchable ("A FETCH reached a datagram, which is never cached.") and moq_net::StreamError::NotFetchable (the received/sent lite code). A received 0x3A surfaces as Error::Stream(StreamError::NotFetchable), matching how NOT_FOUND surfaces as Error::Stream(StreamError::NotFound).
• Bindings: no new FFI/C/py/swift/kt/go/dart variant. moq-ffi folds Error::NotFetchable into MoqError::NotFound, and a received StreamError::NotFetchable reports MoqProtocolKind::NotFound (moq-c MOQ_PROTOCOL_KIND_NOT_FOUND) with the verbatim code 0x3A.
• moq-relay HTTP fetch and moq-hls export treat both forms as a miss (404).
• Behavior change (breaking for anyone relying on it): datagrams outside a subscriber's group range are skipped (Rust and @moq/net). Late-subscriber replay is unchanged from main.
• lite::Version::has_not_fetchable is pub(crate).

Decisions

Maintainer (2026-10-07):

1. ✅ A datagram-flagged fetch object fails with a new Error::NotFetchable, not NotFound or Unsupported.
2. ✅ Drop the tick: a new subscriber may receive recently buffered datagrams; strict live-only delivery is not a goal. Removed model::datagram::Tick, Consumer::subscribe_since, and the opened-at plumbing through resume. (Supersedes the earlier "keep the process-wide datagram tick".) Rejected: the process-wide tick; per-subscription buffer indices.
3. ✅ New lite stream code NOT_FETCHABLE on moq-lite-07-wip only; earlier lite versions keep NOT_FOUND; IETF keeps DOES_NOT_EXIST. (Supersedes the earlier "map onto existing codes" choice.)
4. ✅ No @moq/net NotFetchable mirror yet (deferred until JS gains an IETF fetch receive path).

Made while iterating (recommended option picked; open to challenge):

5. ✅ Code 0x3A, the next free value in moq-lite's own 48-63 range. 0x30 is also unassigned, but it was NO_CAPACITY in an earlier moq-lite-06 revision, so reusing it could be misread by a peer built against that text.
6. ✅ A received NOT_FETCHABLE decodes to Error::Stream(StreamError::NotFetchable), not the flat Error::NotFetchable, matching every other registry code (Error::from_transport nests codes received from a peer). Consumers match both forms, as they already do for NotFound.
7. ✅ Only a track with no fetch handler consults its datagram buffer. A relay's copy forwards the FETCH upstream instead: the draft lets a publisher send one group both as a datagram and on a stream, so the upstream may still serve it. Upstream's NOT_FETCHABLE (lite-07) or a datagram-flagged fetch object (IETF) then crosses the relay as NotFetchable.
8. ✅ No "datagram-only track" flag exists in the model, so the buffer is the only signal. A datagram sequence that has left the 64-entry buffer is a plain NOT_FOUND.
9. ✅ moq-ffi / moq-c report a received NOT_FETCHABLE as the NotFound protocol kind instead of adding a binding variant, keeping the bindings unchanged until a consumer needs to tell them apart.
10. ✅ JS: decoding an unknown lite code does not fail loud (fromTransport wraps it as Stream(code)), so 0x3A is not added to StreamCode here; it belongs with the JS follow-up.
11. ✅ No age bound on what a new subscriber replays from the send buffer. Alternatives: start a new cursor at the first buffered datagram within the subscription's max_delay of the newest datagram's timestamp (existing machinery, a few lines); or a wall-clock age. No in-repo consumer (hang, moq-mux) reads datagrams, each datagram carries its timestamp for the application to judge, and this is main's behavior today. A sparse track can replay up to 64 old datagrams to a late joiner; revisit with the max_delay start if one needs it.
12. ✅ Reverted moq-c's open-before-return change to moq_consume_datagrams: it closed a race only live-only delivery created.
13. ✅ Dropped the late-subscriber tests in Rust and JS: replay is allowed, not promised, so neither runtime pins it.

Tests

Model: a_datagram_group_is_not_fetchable (now NotFetchable, and a non-datagram miss stays NotFound), the_group_range_bounds_datagrams, a_mid_group_start_drops_the_start_groups_datagram, a_range_update_applies_to_buffered_datagrams. Codes: not_fetchable_is_lite_07s (lite-05/06 encode NOT_FOUND and leave 0x3A unknown; lite-07 round-trips), the lite registry round-trip and relay-hop tests, and the IETF stream/request registry sweeps (EVERY_ERROR, unregistered_conditions_are_internal, DOES_NOT_EXIST for both forms). End to end (tests/datagram.rs, a_fetched_datagram_is_refused_per_version): lite-05 and lite-06 get NOT_FOUND, lite-07 gets NOT_FETCHABLE, and through a relay lite-07 to lite-07 stays NOT_FETCHABLE while lite-07 to lite-06 becomes NOT_FOUND. IETF: a_group_fetch_refuses_a_datagram_object, an_absolute_join_skips_a_datagram_group (a two-object datagram group), an_absolute_join_refuses_a_datagram_in_the_head, a_datagram_fill_fails_only_the_fill, a_datagram_before_its_alias_is_dropped, a_standalone_fetch_of_a_datagram_group_is_refused. moq-hls: NotFetchable is a cache miss locally and over lite. JS: the group range bounds datagrams.

Checks: just check origin/main green at 03529eb (lint, clippy, 5309 Rust tests, JS, Python). After merging main a5d8d89: nextest on moq-net, moq-c, moq-ffi, moq-hls, moq-relay and clippy on moq-net green. just drafts check passes. just test interop --all passed before the tick removal, which changes no wire.

Follow-ups

• Rust and @moq/net differ on late-subscriber replay: Rust's shared track buffer hands a new subscriber up to 64 buffered datagrams, while JS's per-subscriber buffers hand it none. Both satisfy the rule; feat(js/net): carry datagrams over moq-transport as OBJECT_DATAGRAM #4979 (JS IETF datagrams) should not promise either.
• When JS gains an IETF fetch receive path, raise and export a NotFetchable there, and name 0x3A in StreamCode, to mirror Rust.
• @moq/net has no start-frame field, so the mid-group datagram rule only exists in Rust. Fold it into the JS IETF datagram quest if JS gains frame starts.
• The Rust IETF subscriber answers a model fetch_group with Unsupported, so the end-to-end datagram FETCH test cannot cover moq-transport hops; the IETF publisher tests cover DOES_NOT_EXIST instead.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

[kixelated]

Outcome: quest implemented and left as a draft. All local checks pass (just check, nextest, js/net bun test, drafts check, interop --all). The two open decisions are listed in the description: NotFound vs a new variant for a datagram-flagged fetch object, and the process-wide datagram tick. Both are recommended as implemented.

(Written by Claude Opus 5.5)

[coderabbitai]

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 7 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 907763fb-510f-4589-b300-baf7323fd4d9

📥 Commits

Reviewing files that changed from the base of the PR and between a5d8d89 and 70fb6b8.

📒 Files selected for processing (22)

• doc/concept/moq-lite.md
• doc/concept/standard.md
• drafts/draft-lcurley-moq-lite.md
• js/net/src/track.test.ts
• js/net/src/track.ts
• quest/m1/README.md
• quest/m1/datagram-unfetchable.md
• quest/m1/js-ietf-datagram.md
• rs/moq-c/src/error.rs
• rs/moq-ffi/src/consumer.rs
• rs/moq-ffi/src/error.rs
• rs/moq-hls/src/export/rendition.rs
• rs/moq-net/src/coding/codes.rs
• rs/moq-net/src/error.rs
• rs/moq-net/src/ietf/error.rs
• rs/moq-net/src/ietf/publisher.rs
• rs/moq-net/src/ietf/subscriber.rs
• rs/moq-net/src/lite/version.rs
• rs/moq-net/src/model/datagram.rs
• rs/moq-net/src/model/track.rs
• rs/moq-net/tests/datagram.rs
• rs/moq-relay/src/web.rs

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[kixelated]

Grok review of 36fc65f2 (first full review; earlier pushes were while the PR was a draft)

The live-only rule looks right in the model. Stamping each buffered datagram with a process-wide tick and opening each relay copy "since" the reader's tick closes the front/splice gap, and the per-version NOT_FETCHABLE/NOT_FOUND mapping is consistent on encode, decode, relay hops, and the IETF DOES_NOT_EXIST bridge. I couldn't find a lite path that sends a code without going through StreamCodes, so lite-05/06 peers can't receive a raw 0x3A. The moq-c change, which opens the subscription before spawning, is the right fix for the race that removing replay would otherwise expose.

Non-blocking

1. A datagram object in a multi-group absolute join aborts a group that was already complete, and drops the rest of the fill. rs/moq-net/src/ietf/subscriber.rs:3797-3804 returns Err(Error::NotFetchable) from decode_fetch_object before run_fill_objects (3242) reaches advance_fill_group. On a draft-16 to draft-19 JoiningFetch::Absolute (used whenever the subscription has a start group, line 6746), the fill spans several groups, and the previous group is only finished when the next group's first object arrives (3881). Here's the failure: the fill covers groups 5 to 9, and group 7 was published as a datagram. Group 6 has every object on the wire but is still the open head, so run_fill's error arm (3209-3213) aborts it with NotFetchable. Readers then see group 6 fail mid-read, and groups 8 and 9 never arrive. The PR's tests only cover a single-object fill (a_datagram_fill_fails_only_the_fill) and a single-group recv_group_fetch. Suggested fix: have decode_fetch_object report the datagram flag instead of failing. On an absolute join, run_fill_objects would then finish the current head, read and discard that object's payload, record its group as prior_group (later objects may inherit it), and keep going. A relative or draft-20 fill, or a group fetch, can still fail as it does now. At minimum, finish a head whose group already ended before returning the error. A test with an absolute join of [stream group, datagram object, stream group] would pin this down.

2. CI hasn't run yet. Every job is still queued on this head.

Verdict: MERGE once CI is green. Item 1 is a narrow path (a pre-draft-20 IETF upstream that puts datagram objects in a joining fetch), so it's fine as a follow-up.

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

Re the Grok review of 36fc65f2, item 1: fixed in 7b6f4691. decode_fetch_object now reports the datagram flag instead of failing. An absolute join finishes the head, skips the datagram's payload, keeps its Group ID as the delta base, and keeps filling. A relative or single-group fill and a group fetch still fail with NotFetchable. Pinned by an_absolute_join_skips_a_datagram_group (stream group 7, datagram 8, stream group 9 stitched into the live tail).

(Written by Claude Opus 5.5)

[kixelated]

Grok follow-up review of 7b6f4691 (re-review after a push; last review was on 36fc65f2)

This push is one commit, fix(net): an absolute join skips a datagram group instead of failing, touching only rs/moq-net/src/ietf/subscriber.rs.

Earlier findings

1. Fixed: a datagram object in an absolute join aborted the complete group before it and dropped the rest. decode_fetch_object now reports datagram instead of failing. On an absolute join, run_fill_objects (3254-3267) finishes the head through the new end_fill_group, skips the payload, and keeps going, and prior_group is updated before the skip so a later delta-coded Group ID still resolves. Relative and draft-20 fills, and recv_group_fetch (3711), still refuse with NotFetchable. an_absolute_join_skips_a_datagram_group covers [stream group, datagram, stream group + tail], which is the case I asked for.
2. Still open: CI. Every job is still pending on this head.

Non-blocking

1. A datagram group with more than one object still fails the whole join on draft-18+. The skip branch requires group.is_some() (3258), but group is the wire field resolved by resolve_fetch_group, which returns None when the object inherits the prior Group ID. Here's the failure: an upstream (not ours, since our model keeps one frame per datagram group) published group 8 as datagram objects 0 and 1. Object 0 carries the Group ID delta and is skipped. Object 1 omits it, so group is None and the fill returns NotFetchable. With the head already ended that only loses groups 9 and up, but that's the same outcome the fix was meant to prevent. Suggested fix: judge the datagram by the effective group, prior_group (already updated above), and fail only when that is None, meaning the stream's very first object inherits nothing.
2. A datagram object in the head's own group finishes that group as complete. On draft-16/17, where Group ID is absolute, a datagram object whose group equals the head's sequence still goes through end_fill_group, which calls producer.finish(). If the publisher sent that group partly on a stream and partly as datagrams, readers see a cleanly finished group that is missing its later objects, and the next stream object of that group then fails the fill with Unsupported (it no longer starts at object 0). Suggested fix: when the datagram's effective group equals the head's sequence, abort the head with NotFetchable instead of finishing it. Both items are narrow, third-party-upstream paths, so they're fine as a follow-up.

Verdict: MERGE once CI is green.

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

Grok follow-up review of 85d0158a (re-review after a push; last review was on 7b6f4691)

This push is one commit, refactor(net): drop the datagram tick; a new subscriber may get the send buffer, which carries out maintainer decision 2. It removes model::datagram::Tick, Consumer::subscribe_since, and the opened-at plumbing through resume. It reverts moq-c's open-before-return change in moq_consume_datagrams, drops the late-subscriber tests in Rust and JS, and rewords the docs and draft.

The revert is clean. rs/moq-net/src/model/resume.rs and rs/moq-c/src/consume.rs/api.rs no longer appear in the PR diff against main at all, so the copy-replay path (datagrams_seen dedup in poll_datagram_once) is exactly main's again. In track.rs, Cursor::new starting datagram_index at 0 is safe because poll_recv_datagram clamps with saturating_sub(datagram_offset). The range filter still judges each buffered datagram when the reader reaches it, so a late subscriber's replay is still bounded by its floor and cap. No Tick, subscribe_since, or "live-only" text remains in any added line. The moq-c revert is fine too: the race it closed only mattered when a subscription missed everything sent before it opened, and with replay back, a late open just reads the buffer. The draft's new MAY sentence ("a subscriber cannot assume a datagram was sent after it subscribed") is the right way to keep Rust's 64-entry replay and JS's no-replay both conformant.

Earlier findings

1. Still open (non-blocking): a datagram group with more than one object still fails the whole join on draft-18+. This push only touched a doc comment in rs/moq-net/src/ietf/subscriber.rs, so the skip branch at 3258 still keys on the wire Group ID instead of the effective prior_group.
2. Still open (non-blocking): a datagram object in the head's own group finishes that group as complete through end_fill_group on draft-16/17. Unchanged.
3. Still open: CI. Every job is pending on this head.

Non-blocking

1. The PR title no longer matches the change. feat(net)!: datagrams are live-only will become the squash commit and changelog line, but after this push the PR says the opposite ("Late-subscriber replay is unchanged from main"). Something like feat(net)!: the group range bounds datagrams, and FETCH never returns one would describe what ships. The ! still fits: out-of-range datagrams are now skipped, and there are new public NotFetchable variants.

Verdict: MERGE once CI is green.

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

Automated review by review (OpenAI)

Reviewed commit: 85d0158, including the latest tick-removal change.

Direction: the narrowed contract is coherent: ranges bound datagrams, FETCH does not return them, and replay remains permitted. The version-gated 0x3A mapping and binding/HTTP miss handling are consistent. No additional issue from removing the tick plumbing.

Confirmed the two existing absolute-join edge cases in #4982 (comment); both remain at this head:

• [P2] rs/moq-net/src/ietf/subscriber.rs:3258 checks the presence of the wire Group ID. A second datagram object inheriting its group therefore aborts the join and loses subsequent stream groups. Use the resolved/effective prior_group and test two datagram objects between stream groups.
• [P2] :3262 calls end_fill_group even when a draft-16/17 datagram belongs to the current head's own group. The helper at :3907 marks that partial head complete. Compare effective sequences first and abort an unrepresentable mixed group rather than caching it as complete.

No duplicate inline comments added. Verification: full original diff, current twenty-two-file patch, latest commit, relevant fill/group-resolution context, and discussion inspected. Static review only; no tests or independent CI verification. GitHub currently reports a merge conflict.

[kixelated]

Re the OpenAI review of 85d0158a: both P2 findings fixed in 70fb6b8e, which also merges main.

• Inherited Group ID: the absolute-join skip now judges a datagram by its resolved group (prior_group), not the wire field, so a second datagram object inheriting group 8 is skipped too. an_absolute_join_skips_a_datagram_group now carries group 8 as two datagram objects; it fails on the old check.
• Datagram in the head's own group: refused as NotFetchable before end_fill_group, so the partial head is aborted rather than finished. an_absolute_join_refuses_a_datagram_in_the_head pins it with the inherited form; the stitch harness is draft-20 only, so the draft-16/17 explicit-ID form shares the same check without its own test.

Merge skew from untimed tracks (#4822): the datagram fetch/alias tests now stamp their objects on a timed track.

(Written by Claude Opus 5.5)

[kixelated]

Grok follow-up review of 70fb6b8e (re-review after a push; last review was on 85d0158a)

This push merges origin/main (a5d8d893) and adds the two review fixes on the absolute-join datagram skip. Outside rs/moq-net/src/ietf/subscriber.rs, every PR file's +/- lines are identical to 85d0158a, so the append_datagram conflict resolution with untimed tracks (#4822) didn't change what this PR ships. In subscriber.rs, the only other change is that the datagram fetch tests now stamp their objects and give the track a timescale, since a timed track on main now refuses an unstamped object.

Earlier findings

1. Fixed: a datagram group with more than one object failed the whole join on draft-18+. The skip now uses the resolved prior_group (3278) instead of the wire Group ID. An object 1 that inherits the datagram group's ID is skipped too, and the fill fails only when there's no resolved group or the join isn't absolute. an_absolute_join_skips_a_datagram_group now sends group 8 as two datagram objects, the second one inheriting, which covers this case.
2. Fixed: a datagram object in the head's own group finished that group as complete. The check at 3284-3287 returns NotFetchable before end_fill_group runs, so run_fill's error arm aborts the partial head instead of finishing it. This covers both the draft-16/17 explicit-ID case and the draft-18+ inherited-ID case. an_absolute_join_refuses_a_datagram_in_the_head checks the inherited case and confirms no finished group 7 reaches the consumer.
3. Fixed: the PR title. It now reads "datagrams are unfetchable and bounded by the group range".
4. Still open: CI. All 13 jobs are pending on this head.

Non-blocking

1. The reverse mix fails with a different error. If a stream object comes right after a datagram group and inherits that group's ID (the datagram came first), head is None and group is None, so the open branch at 3297 returns Unsupported rather than NotFetchable. That's still safe: earlier groups are already finished and nothing partial is published. Only the error variant differs from the head-mix case. If the variant matters to callers, treat "inherits a datagram group" as NotFetchable too. Otherwise this is fine as it is.

Verdict: MERGE once CI is green.

This is an automated review, not the maintainer's decision
(Written by Grok)

[kixelated]

Re the Grok review of 70fb6b8e, non-blocking 1 (a stream object inheriting a datagram group's ID fails as Unsupported, not NotFetchable): leaving it. Nothing partial is published, the fill still fails loud, and matching the variant would need the join to remember the last skipped datagram group just to relabel an error no caller distinguishes today.

(Written by Claude Opus 5.5)