Skip to content

fix(net): a capped IETF stream never claims END_OF_GROUP - #5077

Merged
kixelated merged 6 commits into
mainfrom
quest/m0/ietf-end-of-track-location
Oct 9, 2026
Merged

kixelated merged 6 commits into
mainfrom
quest/m0/ietf-end-of-track-location

Conversation

@kixelated

Copy link
Copy Markdown
Collaborator

Problem

A subgroup stream cut short by the subscription's end Location set the END_OF_GROUP bit in its header, so a subscriber inferred the group ended at the cap even though objects past it exist. Found in Fastly's follow-up on #5020 (status-end-of-track and publish-done cells, d18 and d21).

Approach

  • Rust (rs/moq-net/src/ietf/publisher.rs): the subscription's group header sets has_end only when GroupSlice::until is unset.
  • JS (js/net/src/ietf/publisher.ts): the matching header sets hasEnd only when slice.until is unset.

The header goes out before we know whether the group has objects past the cap, so any capped stream clears the bit. A FIN without END_OF_GROUP claims nothing about the group, which is always true.

Tests: capped_group_does_not_claim_its_end in Rust; the JS readGroup helper now reports endOfGroup, so every slice test asserts the bit, with the absolute-filter test covering a capped tail. Both fail without the fix. just check (except moq-uring, which fails locally on a shared RLIMIT_MEMLOCK, unrelated) and just test interop --all pass.

Completes and deletes quest/m0/ietf-end-of-track-location.md.

Impact

  • Public API: none.
  • Wire: no format change; a capped subgroup stream's header type drops the END_OF_GROUP bit (e.g. 0x50 instead of 0x58 on draft-18+).

Alternatives

  • Clear the bit only when the group really runs past the cap: impossible, since the header is written before the publisher knows.
  • Move END_OF_TRACK onto the upstream's Location (5/5 on group 5's stream rather than 6/0 on a new stream): deferred to an m1 follow-up per the quest.

Follow-ups

  • END_OF_TRACK on the upstream's Location (m1, conformance polish).
  • The Rust write_end_of_track stream header uses GroupFlags::default() (END_OF_GROUP set) while JS clears it; worth aligning.
  • On a backwards range within one group, Rust opens no stream while JS opens an empty one.

🤖 Generated with Claude Code

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Quest outcome: implemented as scoped, in Rust and JS, with regression tests that fail without the fix. just test interop --all passes; just check passes apart from moq-uring tests that hit a shared local RLIMIT_MEMLOCK (environmental, unrelated). No open decisions. Left as a draft for the maintainer.

Suggested follow-up quests:

  1. END_OF_TRACK on the upstream's Location (m1), as the quest deferred. Recommended.
  2. Align the END_OF_TRACK stream header's END_OF_GROUP bit between Rust (set) and JS (clear). Recommended, XS; could fold into 1.
  3. Align the backwards-range case (Rust opens no stream, JS opens an empty one). Optional.

(Written by Claude Opus 5.5)

kixelated and others added 2 commits October 8, 2026 22:32
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A subgroup stream cut short by the subscription's end Location set the
END_OF_GROUP header bit, so a subscriber inferred the group ended at the
cap. Clear it whenever the range caps the group, in Rust and JS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the quest/m0/ietf-end-of-track-location branch from 41d78c1 to 5d9c327 Compare October 9, 2026 05:32
@kixelated
kixelated marked this pull request as ready for review October 9, 2026 05:32
@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 20 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5882bab0-d732-4a0d-a941-cb9697fcd458

📥 Commits

Reviewing files that changed from the base of the PR and between 1acc928 and 2e3f374.


📒 Files selected for processing (5)
  • js/net/src/ietf/publisher.test.ts
  • js/net/src/ietf/publisher.ts
  • quest/m0/README.md
  • quest/m0/ietf-end-of-track-location.md
  • rs/moq-net/src/ietf/publisher.rs


No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 225c5f74-5557-4f79-96bc-7403de27f7bc

📥 Commits

Reviewing files that changed from the base of the PR and between bd6f516 and 1acc928.


📒 Files selected for processing (1)
  • quest/m0/README.md

💤 Files with no reviewable changes (1)
  • quest/m0/README.md

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 0 remain after this review.



Walkthrough

The JavaScript publisher sets END_OF_GROUP only when the served slice has no upper bound. JavaScript tests decode and assert the flag for capped, uncapped, live-tail, and empty streams. Rust tests check capped and whole-group ranges. The Required-list entry and the design note about capped-stream behavior were removed.

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 1acc9

No concrete issue remains that would block merging after normal checks.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 3 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check Passed The title clearly and concisely states the main fix: capped IETF streams must not claim END_OF_GROUP.
Description check Passed The description directly explains the END_OF_GROUP bug, the Rust and JavaScript fixes, the tests, and the scope of the change.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.


✨ Finishing Touches
✨ Simplify code
  • Commit to this branch
  • Create a new PR


  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kixelated

Copy link
Copy Markdown
Collaborator Author

Grok review of 5d9c3270

Small, correct fix: the subgroup header now sets END_OF_GROUP only when the slice has no until cap, in Rust (publisher.rs:2700) and JS (publisher.ts:680). Both sides apply the same rule, and the tests check both directions: a capped stream clears the bit and a whole group keeps it.

No blocking issues.

Non-blocking:

  • A cap at or past the group's real last object (for example an end Location of object 5 in a 3-object group) also clears the bit, so the subscriber loses a true END_OF_GROUP. That's safe, since a FIN with no claim is never wrong, and you can't do better because the header is written first. Still, a one-line note in the test doc that this is intended would help.
  • The follow-ups you listed are real and worth tracking: Rust write_end_of_track uses GroupFlags::default() (END_OF_GROUP set) while JS clears it, and a backwards range within one group opens no stream in Rust but an empty one in JS (the JS test now asserts endOfGroup: false there).
  • The quest deletion and README line removal are consistent, and no other links to ietf-end-of-track-location.md remain in the diff.
  • CI was still pending when I reviewed.

Verdict: MERGE (once CI is green)

This is an automated review, not the maintainer's decision
(Written by Grok)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Grok review of bd6f5162: MERGE once CI is green

The fix is small and correct. Both publishers now set END_OF_GROUP only when the slice has no until cap (rs/moq-net/src/ietf/publisher.rs ~L2786, js/net/src/ietf/publisher.ts ~L682). Clearing the bit is always safe, because a FIN without END_OF_GROUP claims nothing about the group. The quest file and its README entry are removed together, so there's no dangling link.

Non-blocking

  • The Rust test's doc comment says that a cap at or past the group's real end also clears the bit, but no assertion checks it. Adding serve(Some(2)) (cap on the last object) and asserting !has_end would pin down the deliberate over-clearing, so nobody later "optimizes" it into a false claim.
  • The trade-off: a subscriber whose range ends exactly on a group's last object can no longer tell from the header that the group is complete. That's fine for correctness, but worth noting next to the follow-up about END_OF_TRACK on the upstream's Location.
  • The follow-up listed in the PR body still stands: Rust's write_end_of_track header keeps GroupFlags::default() (END_OF_GROUP set) while JS clears it. Interop cells comparing the two may differ until that's aligned.
  • CI is still pending on all jobs.

This is an automated review, not the maintainer's decision
(Written by Grok)

@kixelated kixelated left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review by review (OpenAI)

Reviewed the full five-file diff at bd6f516214e8f534a2c7ee392d71cd4574ac400b against base a749c1aecd2e85c6391732e953c9f0ef245fcb73.

No actionable findings introduced by this PR. The direction is sound: Rust and JS derive END_OF_GROUP from the same explicit object cap, preserving the bit for uncapped groups and avoiding a false claim even when the cap happens to reach the real end. This is the simplest safe choice before the header is sent; no public API or wire-format change is needed.

I traced the inclusive-to-exclusive bounds, flag encoding, stream completion, and the capped/uncapped regression assertions. The original false END_OF_GROUP claim is fixed. The documented END_OF_TRACK placement/header and backwards-range differences remain pre-existing follow-ups. This independently agrees with the core conclusion of the existing review; no duplicate inline comments.

Verification: GitHub-only static review; I did not run tests. At publication, Check and Test are still queued, so this is not a claim that CI passed.

(Written by OpenAI)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Merge summary:

  • Merged origin/main in cleanly; the fix and both regression tests still apply after the IETF publisher changes on main.
  • Took Grok's non-blocking note: the Rust test doc now says a cap at or past the group's real end also clears END_OF_GROUP, since the header goes out first.
  • The OpenAI review of bd6f516214e8f534a2c7ee392d71cd4574ac400b has no findings.
  • Follow-ups 1 and 2 are tracked in quest/m1/ietf-end-of-track-placement.md (landed in quest: plan the second round of 2026-10-08 spawn follow-ups #5092). The backwards-range difference (Rust opens no stream, JS opens an empty one) has no quest yet.

Enabling auto-merge on bd6f516214e8f534a2c7ee392d71cd4574ac400b.

(Written by Claude Opus 5.5)

@kixelated
kixelated enabled auto-merge (squash) October 9, 2026 17:07
@kixelated

Copy link
Copy Markdown
Collaborator Author

Merged origin/main again after #5065-era IETF changes landed. The only conflict was quest/m0/README.md, where main had already dropped the End of Group status line; both lines are now gone. The PR's own diff is unchanged, so the OpenAI review of bd6f5162 carries over. just check passes locally. The first Test run hit the known bursts_cross_a_flapping_peer::impaired connect timeout (quest/m1/test-flakes-2/impaired-handshake.md).

Auto-merge is pinned to 1acc928001b93b303ee8e86260d164774a8ec009.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Grok review of 1acc9280

This is a small, focused fix. Rust (rs/moq-net/src/ietf/publisher.rs ~L2786) and JS (js/net/src/ietf/publisher.ts ~L682) now both set END_OF_GROUP only when the slice has no until cap. It's conservative and always true, since a FIN without the bit claims nothing. Tests cover the capped and whole-group cases in Rust, and the JS readGroup now checks the bit on every slice test.

Blocking: none.

Non-blocking

  • A cap at or past the group's real end (for example end.object >= last object) also drops the bit. That's documented and correct, but it costs subscribers a group-ended signal they could have used. If GroupSlice::until is ever computed against a group that's already finished (as in the cached or fetch fill path), you could keep the bit there whenever until > last id. That's optional.
  • The JS backwards-range case still opens an empty stream, now without END_OF_GROUP, while Rust opens nothing (already listed as a follow-up). That's harmless, but the two lines differ on the wire.
  • The Rust write_end_of_track header still uses GroupFlags::default() (has_end = true) while JS clears it. The PR already lists this as a follow-up. It should get a quest file so it isn't lost now that ietf-end-of-track-location.md is deleted. The same goes for the deferred "END_OF_TRACK on upstream Location" m1 item, which I don't see recorded anywhere in the diff.
  • CI is still pending.

Verdict: MERGE (once CI is green)

This is an automated review, not the maintainer's decision
(Written by Grok)

@kixelated
kixelated merged commit 9680ca9 into main Oct 9, 2026
11 checks passed
@kixelated
kixelated deleted the quest/m0/ietf-end-of-track-location branch October 9, 2026 19:53
kixelated added a commit that referenced this pull request Oct 10, 2026
fix(net): a capped IETF stream never claims END_OF_GROUP (backport #5077)
kixelated added a commit that referenced this pull request Oct 10, 2026
Conflicts are release backports whose originals are already on main
(#4840, #4892, #4710, #5077) and release's noq 1.3.5 pins (#5170);
resolved to main's side. Ports the two release-only test additions from
the backport reviews: a_parked_read_watches_its_edge_abort, which passes
on main's reworked expiry path, and the capped-group END_OF_GROUP
asserts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shermerL pushed a commit to shermerL/moq that referenced this pull request Oct 10, 2026
Backport to release. The Rust regression test decodes the header with
release's GroupHeader::decode form and runs under tokio::test, since
release lacks coding::decode_buf and moq_net_sim. Quest edits dropped.

(cherry picked from commit 9680ca9)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant