Skip to content

feat(net)!: negotiate publisher epochs as metadata - #4817

Closed
kixelated wants to merge 7 commits into
mainfrom
quest/m0/broadcast-epoch/origin
Closed

kixelated wants to merge 7 commits into
mainfrom
quest/m0/broadcast-epoch/origin

Conversation

@kixelated

@kixelated kixelated commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

Paths alone cannot prove that two routes carry identical content. Resuming through a restarted publisher or another transcode worker can splice unrelated bytes. Encoding an epoch in the path also breaks discovery for clients subscribing to the original name.

Approach

Keep one plain broadcast path and carry the publisher epoch as negotiated metadata. TRACK, SUBSCRIBE, and FETCH echo the selected epoch so a restart between requests cannot mix instances. Only matching explicit epochs permit cross-route resume. An unidentified request stays pinned to its first serving route, never binds implicitly to an epoch, and never shares a front with an explicit-epoch request.

Local Origin broadcasts mint epochs by default; replicas can supply an explicit identity. Changing an advertised epoch sends END then START. Route repricing preserves it. Legacy Lite and moq-transport peers keep plain paths and receive no epoch metadata; unknown incoming identities are never synthesized.

Impact

  • Rust adds broadcast::Id { path, epoch }, accepted by publication and request APIs, plus epoch metadata on broadcast info and routes. Plain path call sites remain supported; they lose cross-route resume, including GOAWAY reconnects.
  • JS accepts { path, epoch } on createBroadcast, exposes epochs on broadcast handles and routes, and forwards selected epochs on all track requests. request.active continues handing over distinct broadcasts for decoder resets.
  • Lite 05+ negotiates SETUP parameter 0x6 = 1. ANNOUNCE_START (Lite-05 active ANNOUNCE), TRACK, SUBSCRIBE, and FETCH carry an optional trailing canonical UUIDv7 string only when negotiated. Absent metadata preserves existing bytes. No moq-transport wire extension.
  • Removes Rust Path::join_epoch / split_epoch and JS Path.joinEpoch / splitEpoch; paths, relative references, and authorization have no special epoch syntax or grant widening.
  • Updates the Lite and E2EE drafts, API/relay docs, and dependent quests. E2EE takes its epoch from metadata or the authenticated application channel; key derivation is unchanged.

Alternatives

Path suffixes plus bare aliases, implicit bare-to-epoch binding, and announcement-only metadata were rejected. Explicit request identity prevents the catalog-A/video-B race. Epoch timestamps do not override route costs.

Validation

  • Rust regression coverage: identity separation in both request orders, stale-epoch refusal, mid-group matching-epoch failover, legacy route termination and resubscription, and immutable announcement identity.
  • JS tests, including real mock-session TRACK/SUBSCRIBE/FETCH and stale-epoch requests, pass. The final merged net/tokio suites pass 1,972 tests.
  • just test interop --all, just drafts check, and Rust lint/docs/dependency checks pass.
  • Epoch lookup benchmark sweeps 1/1,000 paths and 1/32 routes per path: approximately 0.61–0.69 microseconds, with no material slope from unrelated paths.
  • Full just check reaches the Rust tests but this machine cannot initialize io_uring within its 8 MiB locked-memory limit, including in isolation. just rs test --workspace --exclude moq-net-fuzz --exclude moq-uring --no-fail-fast passes all 6,079 tests (11 skipped); the fuzz package is a standalone libFuzzer harness. Main's existing warm-cache fix (fix(moq-net): an IETF copy goes idle before its cancel #4918) is merged.

Follow-ups

The existing Apps/Bindings quests cover native announcement-following helpers and exposing selected identities. Native plain-path requests do not become smart followers automatically. The Wildcard line still needs to decide derived-output identity and group-start requirements.

Legacy discovery and delivery remain compatible, but third-party caches that cannot see epochs still require immutable wire names and object positions. Never-stitch behavior cannot invalidate their caches.

(Written by GPT-6)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Outcome: the origin quest is implemented in Rust and JS and just check plus just drafts check pass locally. It stays a draft until the maintainer settles the open decisions in the description: catalog bare references pin or follow, the Path::mint_epoch helper shape, reusing UNROUTABLE as the move reset, covering-route precedence, and the loose @* grant widening. The questline PR #4805 was closed because questline branches are retired, so this targets main.

(Written by Claude Opus 5.5)

@kixelated

Copy link
Copy Markdown
Collaborator Author

Maintainer decision (2026-10-05, from the quest audit in #4845), relayed by an agent: this PR must settle the epoch-pin question itself before landing. The question: does a bare-name catalog broadcast reference pin its catalog's epoch, or follow the newest? Don't hand it to catalog-track-alias. It's m0 work, and catalog-track-alias never mentions epochs.

Also note: as of #4845, quest/m0/broadcast-epoch/ holds stats-epoch, stats-aggregate-bound and unannounce-demand-release, and transcode-group-start (moving to per-worker epochs in #4812) Requires broadcast-epoch/origin.md. If this PR deletes origin.md, repoint those links.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 5, 2026 13:40
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request for a bare name no route covers resolves to the greatest
name/@<uuidv7> a route serves, in Rust and JS. The Rust front follows that
one epoch and resets its tracks with Unroutable once the name resolves
elsewhere; JS swaps the request's active broadcast. A grant admitting a
name also admits its epochs. Publishing mints no epoch by default;
Path::mint_epoch / Path.mintEpoch make it one call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…end-only epochs

- Rename Path::join_epoch / Path.joinEpoch to with_epoch / withEpoch, which
  set the final epoch (replacing any) and remove it for None.
- An epoch only ends a path: publishing or serving a prefix with an epoch
  before its last segment fails with Error::MisplacedEpoch.
- Relative references resolve against a base's name past its final epoch, so
  a catalog means the same whether fetched by name or by epoch; a reference
  follows its target's newest epoch unless it spells one.
- Specify both in the hang and lite drafts and record the decisions in the
  broadcast-epoch line README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated
kixelated force-pushed the quest/m0/broadcast-epoch/origin branch from c178f35 to 63b2ea3 Compare October 5, 2026 21:22
@kixelated

Copy link
Copy Markdown
Collaborator Author

Proposal: never stitch unepoched paths

Context: moq.pro#2165 and moq.pro#2208 hit the case this PR's decision 4 is about. A transcode worker is reached through a .transcode/** wildcard claim, by rendezvous hash. If a route move resumes a subscription on a different worker, Recover::poll_serving splices two encoders' bytes mid-group (rs/moq-net/src/model/resume.rs). Matching catalogs and group numbers don't make independently encoded groups interchangeable.

Rule

Mid-group resume across routes only happens between routes serving the same epoch path. An epoch is the publisher's opt-in that every copy is the same bytes. A path without an epoch is pinned to the route that first served it. If that route goes away, the subscription ends (or the front resets), and the client's resubscribe resolves again. Unepoched content is never stitched.

This narrows #4741, where any covering route can resume any path. front.rs already calls reusing a name for different content a bug. This makes the safe behaviour the default instead of relying on publishers to avoid it, and makes seamless failover something a publisher opts into by minting an epoch.

What follows

  • Decision 4 stays as recommended: a covering route, including a wildcard claim, wins over the name's epochs. A claim-served bare name never needs epochs.
  • Derived services stay bare. Transcode output is served at .transcode/<pid>/<source> through the claim, with no @<worker> epoch, no catalog determinism, and no group-start requirement. Stickiness comes from the relay: an existing front is never rerouted, new subscribers on that relay join it, and every relay picks the same claimant by rendezvous hash. If a worker dies, its subscribers resubscribe and cold-start on another worker. During claim-set churn two relays can briefly pick different workers. That double encode is bounded and transient, and it is never stitched.
  • Demand still works. A bare SUBSCRIBE reaches the claim and starts the worker. No new wire message or SUBSCRIBE_OK field is needed.
  • Plain moq-transport clients see bare names only. When a route dies, the subscription ends and they resubscribe. Unepoched content has no other failover, because they never opted in.
  • Overlay, voice and .dash follow the same rule and don't need epochs either. One example is .dash/<pid>/stats, which every control node announces at one path: it stops being a splice hazard.

Changes to this PR

  1. Resolution: a Recover/resume across routes requires the old and new routes to serve the same epoch path. Otherwise the front resets.
  2. Docs and the draft: state that only epoch paths are resumable across routes.
  3. A test: two claimants serve the same bare path with different bytes. Kill the serving one mid-group, then assert a reset and no mixed group.

Open decision 3 (whether lite-06 and IETF clients resubscribe after the reset) matters more under this rule, since a reset is now the only failover for unepoched paths.

(Written by Claude Opus 5.5)

kixelated added a commit that referenced this pull request Oct 5, 2026
Revert the per-worker epoch re-plan of the transcode, wildcard,
broadcast-epoch, and processor quests so it is decided with #4817, and
drop the catalog-equality assertion from the two-instance test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…across routes

- A bare request binds once, at request time, to its newest announced epoch,
  ahead of any route covering the name, and otherwise to the covering route.
  It never moves: a newer epoch is a different broadcast. Drops the follow and
  reset-on-move logic.
- A path without an epoch is pinned to the route that first served it and
  ends when that route goes, so unrelated content is never stitched. Epoch
  paths keep resuming across routes.
- JS gives epochs the same precedence; a swap of `active` is already a new
  broadcast, never a splice.
- Moves the failover tests and the handoff bench onto epoch paths, adds
  pinning tests, renames the follow bench to bind, and specifies the rules in
  the lite draft, concept docs, and the line README.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Adopted the "never stitch unepoched paths" proposal in ffa7e28, with one difference: bare-to-epoch resolution stays, in a bind-once form.

  • A path without an epoch is pinned to the route that first served it, and ends when that route goes. Epoch paths still resume across routes. The two-claimant test is a_bare_path_never_resumes_on_another_route.
  • A bare request binds once to its newest announced epoch, and an epoch now wins over a covering route (decision 4 reversed). With no epoch announced, the request goes through the covering route, pinned. It never moves to a newer epoch; smart clients follow announcements instead.
  • Resolution stays because dropping it would make every epoch-minting publisher invisible to clients that send a bare SUBSCRIBE. With the follow and reset logic gone, it costs about 40 lines.
  • Derived output served through a claim stays bare and sticky. Whether that removes the Wildcard line's group-start and mirrored-epoch requirements is left to that line.

(Written by Claude Opus 5.5)

kixelated and others added 3 commits October 6, 2026 10:01
Co-Authored-By: Codex <noreply@openai.com>
Keep broadcast paths unchanged, require explicit matching identities for route resume, and pin unidentified subscriptions. Carry epochs in negotiated Lite announcements and TRACK, SUBSCRIBE, and FETCH requests.

Co-Authored-By: Codex <noreply@openai.com>
@kixelated kixelated changed the title feat(net): bare names follow their newest epoch feat(net)!: negotiate publisher epochs as metadata Oct 6, 2026
@kixelated

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #4942. In the 2026-10-06 quest audit the maintainer picked #4942's route metadata as the one epoch carrier, over SETUP-negotiated metadata and over the @<uuidv7> path segment.

(Written by Claude Opus 5.5)

@kixelated kixelated closed this Oct 6, 2026
@kixelated
kixelated deleted the quest/m0/broadcast-epoch/origin branch October 8, 2026 18:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant