From ab9c5f18f9c615e72901d5a0323873e98dc83946 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Sat, 10 Oct 2026 06:27:47 -0700 Subject: [PATCH] quest: plan epoch correctness fixes and IETF extension Co-Authored-By: GPT-6 --- quest/m0/broadcast-epoch/README.md | 10 +++ quest/m0/broadcast-epoch/export-ts.md | 7 ++ .../m0/broadcast-epoch/js-consume-identity.md | 47 +++++++++++ .../js-restart-keeps-request.md | 4 + quest/m1/README.md | 3 +- quest/m1/claim-epochs.md | 18 +++- quest/m1/ietf-claim-epochs.md | 38 +++++++++ quest/m1/ietf-epochs.md | 82 +++++++++++++++++++ 8 files changed, 204 insertions(+), 5 deletions(-) create mode 100644 quest/m0/broadcast-epoch/js-consume-identity.md rename quest/{m1 => m0/broadcast-epoch}/js-restart-keeps-request.md (86%) create mode 100644 quest/m1/ietf-claim-epochs.md create mode 100644 quest/m1/ietf-epochs.md diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index 8652959cd7..0d2fed3e11 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -25,6 +25,14 @@ path never changes and every older version and moq-transport keeps working: their routes carry no epoch, and see a restart as an end and start at the same path. +Decided 2026-10-10: keep epochless clients supported on updated relays, with +unchanged paths and End+Start on replacement. The relay retains the explicit +identity internally; fresh requests cannot join its old cached instance. +This does not promise cache invalidation in arbitrary older IETF relays or +clients that retain objects across broadcasts. The negotiated +[IETF extension](/quest/m1/ietf-epochs.md) follows in m1, independently of +this release gate; seamless JS handover stays in m1 too. + Non-goals: pooling, which needs nothing here; a redundant pair shares an explicit epoch through `moq --epoch`. Also out of scope: trusting the publisher's clock (a far-future epoch wins @@ -97,6 +105,8 @@ maintainer's pre-release merge of `release` into `main` keeps `main`'s ## Required +- [JS consume identity](/quest/m0/broadcast-epoch/js-consume-identity.md) - requests carry the serving prefix's epoch and new winners cannot reuse an old cached broadcast +- [JS restart keeps the request](/quest/m0/broadcast-epoch/js-restart-keeps-request.md) - a resolved request remains on its old instance, matching Rust's sticky subscriptions - [Export ts](/quest/m0/broadcast-epoch/export-ts.md) - `export ts` and SRT egress linger only for the same epoch, and switch to a replacement only with `--stitch`, as a full program switch - [Publish catalog restart](/quest/m0/broadcast-epoch/publish-catalog-restart.md) - `@moq/publish` never reuses catalog group numbers under one name and epoch after a re-announce - [Bindings](/quest/m0/broadcast-epoch/bindings.md) - moq-ffi and every wrapper expose the epoch and let a publisher announce one diff --git a/quest/m0/broadcast-epoch/export-ts.md b/quest/m0/broadcast-epoch/export-ts.md index ade64d5a2e..bcb34880d7 100644 --- a/quest/m0/broadcast-epoch/export-ts.md +++ b/quest/m0/broadcast-epoch/export-ts.md @@ -35,6 +35,13 @@ publisher that stays alive holds the export on a replaced broadcast. - Drive both flags from announcements, as players do: `Start`, `Restart`, and `End` with their epochs. An export that fails while its broadcast is still announced exits 1 without lingering, as today. +- Use `origin.follow(path)` for the resolved path events, shared by CLI and + SRT. The 2026-10-10 audit of #5147 found a second prefix watcher treating + any covering prefix's End or Restart as a change even while an exact + route still wins. Regress with a live exact route while a broader prefix + starts, restarts, or ends: none changes the export. Actual winner changes + and gaps must still drive the policy above. Coordinate with the existing + follower-gap fix instead of reimplementing route resolution here. - Same epoch within `--linger`: `follow` continues the same stream. It is the same content, so the existing skip handling covers the gap (late frames drop, and a forward leap opens a new generation). Within an epoch, diff --git a/quest/m0/broadcast-epoch/js-consume-identity.md b/quest/m0/broadcast-epoch/js-consume-identity.md new file mode 100644 index 0000000000..292476c65a --- /dev/null +++ b/quest/m0/broadcast-epoch/js-consume-identity.md @@ -0,0 +1,47 @@ +# [M] JS consumes use the identity of the serving announcement + +## Goal + +`@moq/net` requests name the epoch of the announcement serving their path, +including covering prefixes. A newly winning announcement never gives a new +consumer a cached broadcast from the previous source. Existing consumers +remain on their instance until they drop it or its source ends. + +## Plan + +Reproduced in the 2026-10-10 epoch audit in `js/net/src/lite/subscriber.ts`: + +- Announce `pool` with epoch A, then consume `pool/job`: TRACK has no epoch + because the subscriber looks up only the exact path. Rust already uses + the resolved route's epoch. +- Consume `pool/job` through `pool`, then announce the more-specific + `pool/job` with epoch B: another consume returns the old cached broadcast. + Restart and End evict cached consumes, but the new Start does not. + +Bind request identity and cache reuse to the serving announcement. Reuse the +existing route-resolution rules rather than adding another interpretation of +prefix precedence. A more-specific epochless announcement must not inherit +the epoch of a broader one. Inspect the shared `BroadcastCache` and IETF +subscriber for the same stale-cache behavior; IETF has no wire epoch today, +but a different epochless source still cannot reuse the old instance. + +Add regressions for both failures, with exact-path requests as a control, +more-specific epochless routes, fallback after withdrawal, and old consumers +remaining readable while new consumers select the replacement. Same explicit +identity may reuse a broadcast; equal absent epochs do not prove identity. +Use the existing JS test lane and run cross-language interop. If routing or +invalidation work changes fanout costs, extend the relevant benchmark over +announcements and consumers instead of accepting a whole-table scan. + +Decided 2026-10-10: these correctness fixes gate m0. Seamless JS track +handover remains separate in m1; fixing request identity does not require +building its resume pump. Update stale comments and existing concept docs +inline, with no separate guide. + +Public API: corrected request and cache behavior, no new exported surface +expected. Wire: existing lite-07 epoch fields carry the resolved identity; +no framing change, and older versions stay epochless. + +## Related + +- [JS track handover](/quest/m1/js-group-handover.md) - preserves open tracks across routes of the same explicit identity diff --git a/quest/m1/js-restart-keeps-request.md b/quest/m0/broadcast-epoch/js-restart-keeps-request.md similarity index 86% rename from quest/m1/js-restart-keeps-request.md rename to quest/m0/broadcast-epoch/js-restart-keeps-request.md index 6992b22e9d..c7a67f6595 100644 --- a/quest/m1/js-restart-keeps-request.md +++ b/quest/m0/broadcast-epoch/js-restart-keeps-request.md @@ -10,6 +10,10 @@ resolve under the new epoch. ## Plan +Decided 2026-10-10 in the epoch audit: correctness fixes gate m0, while +seamless JS handover remains in m1. Move this existing Rust/JS parity fix +under the broadcast-epoch release gate without expanding its scope. + Found in #5141 (2026-10-10): on an epoch change JS ends a resolved request handle (it reports unroutable), while Rust keeps the resolved broadcast. Open track subscriptions survive in both. Decided 2026-10-10: align JS to Rust, diff --git a/quest/m1/README.md b/quest/m1/README.md index 4b44207105..911f4044d2 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -85,7 +85,9 @@ ladders. - [Catalog track identity](/quest/m1/catalog-tracks.md) - a track's codec and description never change for its name; resolution changes in band below ceilings fixed at creation, and anything else mints a new rendition or epoch - [Archive](/quest/m1/archive/README.md) - record selected tracks to any object_store and replay them over FETCH or derived HLS; the catalog entry and format may break in place, since no archives exist - [In-band auth](/quest/m1/auth/README.md) - a session tells its peer what it may publish and subscribe to, unions tokens presented in band, and fails loud on an out-of-scope publish +- [IETF epochs](/quest/m1/ietf-epochs.md) - negotiate explicit broadcast identity on drafts 17-22, retaining unchanged paths and epochless clients on updated relays - [Claim epochs](/quest/m1/claim-epochs.md) - a lite-07 claim's answer carries its broadcast's epoch, so a worker restarting an output under an unchanged claim route is a new source; moved from m0 on 2026-10-08 since lite-07 is opt-in +- [IETF claim epochs](/quest/m1/ietf-claim-epochs.md) - carry the shared per-output identity in negotiated IETF responses after the extension and lite-07 model land - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - when the maintainer cuts it, lite-07 negotiates as `moq-lite-07` and the next release ships it - [Dropped sources](/quest/m1/dropped-sources.md) - track consumers see the producer's real error on every end path, never `Dropped` - [Client settings parity](/quest/m1/obs-client-config.md) - moq-ffi offers moq-c's client knobs, and the OBS Advanced settings get back the ones the C++ migration dropped @@ -196,6 +198,5 @@ ladders. - [WebGPU HDR](/quest/m1/webgpu-hdr.md) - HDR renditions play as HDR where the browser and display can show it, and tone-map to SDR elsewhere - [Request ID order](/quest/m1/request-id-order.md) - drafts 14 to 16 refuse a reused or lower Request ID in both languages - [Import catalog drop](/quest/m1/import-catalog-drop.md) - moq-cli import never drops a catalog producer without finishing it -- [JS restart keeps the request](/quest/m1/js-restart-keeps-request.md) - a resolved `@moq/net` request survives a Restart, as in Rust - [WebGPU on Safari](/quest/m1/webgpu-safari.md) - the WebGPU renderer is verified on Safari 26 for macOS and iOS - [A/V sync across a break](/quest/m1/watch-break-av-sync.md) - `@moq/watch` never plays pre-break audio out of sync with pre-break video diff --git a/quest/m1/claim-epochs.md b/quest/m1/claim-epochs.md index 98c0edbe79..1800ba3f11 100644 --- a/quest/m1/claim-epochs.md +++ b/quest/m1/claim-epochs.md @@ -18,11 +18,19 @@ to get one restarts the first viewer: the front resolved through the claim, and the epoch route wins the path and is announced as a `Restart` (Restart). A front on a drained worker no longer needs this: under Restart a request joins a front only while -its route still wins, on every version (re-scoped 2026-10-07). lite-07 only; older versions -and moq-transport keep today's behavior. +its route still wins, on every version (re-scoped 2026-10-07). This quest owns +the shared model and lite-07 wire. The negotiated IETF response is owned by +[IETF claim epochs](/quest/m1/ietf-claim-epochs.md); older and unnegotiated +versions keep today's wire. ## Plan +Decided 2026-10-10: extend claim-served identity to lite-07 and IETF as +separate m1 follow-up work. Land the shared model and lite-07 behavior here, +then apply it to IETF in its own quest, so the lite-07 cut does not acquire an +IETF dependency. The API shapes below remain implementation proposals, not +newly settled maintainer decisions. + Decisions (2026-10-07, proposed for the maintainer): - TRACK_INFO gains an `Epoch`: the instance that answered, or empty. Each hop @@ -63,8 +71,9 @@ Decisions (2026-10-07, proposed for the maintainer): - A worker drains feed by feed by closing the outputs it wants off: their readers re-request and land on the cheapest worker as a new instance, one short cut per feed instead of withdrawing the claim and cutting everything. -- Every link from the worker to the relays that serve viewers must speak - lite-07, since the epoch is learned and named hop by hop. A viewer's own +- Every link from the worker to the relays that serve viewers must carry + the served identity: lite-07 here, or the negotiated IETF extension after + its claim follow-up lands. The epoch is learned and named hop by hop. A viewer's own version should not matter, because the join rule and the refusal live at relays and workers; confirm it, including that `@moq/net`'s track cache never hands a returning viewer the old instance's group. @@ -90,5 +99,6 @@ it. Wire: lite-07 TRACK_INFO gains `Epoch`. ## Related +- [IETF claim epochs](/quest/m1/ietf-claim-epochs.md) - the same served identity on negotiated IETF responses, without gating the lite-07 cut - [Upstream position regression](/quest/m1/largest-regression.md) - catches the same restart from the answer's largest position on lite-07 and moq-transport; lite-05 and 06 stay uncovered - [Finalize moq-lite-07](/quest/m1/lite07-finalize.md) - waits on this wire change diff --git a/quest/m1/ietf-claim-epochs.md b/quest/m1/ietf-claim-epochs.md new file mode 100644 index 0000000000..80a9fb5763 --- /dev/null +++ b/quest/m1/ietf-claim-epochs.md @@ -0,0 +1,38 @@ +# [M] IETF claim responses identify the served broadcast + +## Goal + +A relay learns the identity of a broadcast served under a prefix claim over +the IETF epoch extension, as it does over lite-07. A worker restarting one +output cannot splice its new groups into a cached old instance, and the +first viewer does not need a competing exact-path announcement. + +## Plan + +Decided 2026-10-10: claim-served epochs cover both protocols but remain a +separate m1 follow-up to announcement-based IETF epochs. The existing +claim-epochs quest owns the shared broadcast model and lite-07 wire; this +quest owns the IETF response identity and mixed-protocol verification. This +split keeps the lite-07 cut independent of the IETF extension. + +Carry the accepted broadcast's identity in the negotiated response path and +pin later requests to it. Preserve the shared claim rules for adopting a +learned epoch, refusing another instance, and keeping existing readers +sticky. Do not copy a source epoch onto different derived content, mint +epochs in relays, or infer continuity from an epochless claim. + +Mirror the claim-epochs regression scenarios over IETF and mixed lite-07 / +IETF chains: first viewer without a cut, restarted output with cached old +groups, stale learned identity refused, and a worker draining one output. +Include an epochless viewer on an updated relay that resubscribes after the +replacement. Use mocked time and existing CI lanes; run cross-language +interop. Update the extension draft and existing concept docs in the same +PR, with `just drafts check` and `just check`. + +Public API: reuse the shared broadcast identity model. Wire: negotiated IETF +responses report the served identity; unnegotiated peers keep their wire. + +## Required + +- [IETF epochs](/quest/m1/ietf-epochs.md) - negotiation and announcement/request identity +- [Claim epochs](/quest/m1/claim-epochs.md) - the shared model and lite-07 reference behavior diff --git a/quest/m1/ietf-epochs.md b/quest/m1/ietf-epochs.md new file mode 100644 index 0000000000..65925d301e --- /dev/null +++ b/quest/m1/ietf-epochs.md @@ -0,0 +1,82 @@ +# [L] Negotiate broadcast epochs on IETF connections + +## Goal + +Rust and JS carry broadcast epochs over a negotiated moq-transport extension +on drafts 17 through 22 without changing the broadcast path. Matching +explicit epochs permit stitching; a different epoch or a different +epochless source is a new broadcast. Epochless clients remain supported on +updated relays. + +## Plan + +Decided 2026-10-10 in the epoch audit interview: + +- This is m1, after the m0 identity correctness fixes. Do not make the + extension or seamless JS handover a release blocker. +- Preserve the path. The previous `@` suffix changed the name older + clients subscribe to; rejected in favor of negotiated metadata. +- Use a standalone SETUP-negotiated extension on drafts 17-22, where the + existing extension infrastructure applies, rather than draft 22 only. + Drafts 14-16 and peers that do not negotiate it retain their existing wire. + Integrate with the shared extension configuration in Rust and JS instead + of inventing another configuration mechanism. +- Reuse lite-07's epoch representation and identity semantics. A relay + preserves an explicit epoch, never invents continuity for an epochless + source, and refuses malformed or mismatched identities. +- Carry identity through announcements and the requests that resolve, + subscribe to, or fetch a broadcast. Pin a request to its chosen instance; + a replacement must not substitute new content into an old request. + Separate object and negative caches by identity, including joining FETCH + and concurrent old/new subscriptions. Track aliases and response handling + must preserve that separation too. This changes identity semantics, not + merely advisory announcement metadata. +- Epoch and cluster extensions negotiate independently. The cluster draft + already describes a shared negotiated NAMESPACE parameter block; make its + framing unambiguous with either extension, both, or neither. Reconcile its + namespace-only redundant-publisher semantics with explicit epoch opt-in. +- Support epochless clients on updated relays: preserve their paths and + wire framing, translate replacements into withdrawal and reannouncement, + and resolve fresh subscriptions to the current instance. Keep the relay's + internal epoch and caches separated. A same-epoch upstream route handoff + may remain invisible downstream. Existing subscriptions stay sticky; + announcement-aware players reset and resubscribe on End+Start. A player + that ignores these signals is not promised automatic recovery. +- Arbitrary older caching relays are out of scope. Base IETF object identity + is namespace, track, group, and object ID, and withdrawal does not invalidate + it. Do not claim that removing the epoch makes reused object names safe + through those relays or through clients retaining caches across broadcasts. + Immutable wire names plus discovery/translation would be separate work. +- Claim-served response identities are a separate follow-up, shared with + lite-07's claim-epochs work. Do not block announcement-based epochs on it. + +Add a feature-specific IETF extension draft in the implementation PR, with +negotiation, identity, cache, request, downgrade, and malformed-input rules. +Update the cluster draft and existing identity/compatibility concept docs +inline. No separate migration-guide quest. The new draft must distinguish +our legacy-client behavior from base IETF's immutable-name guarantee; an +End+Start is not a general cache invalidation mechanism. + +Verify Rust and JS with matching, different, and absent epochs; exact and +covering announcements; old subscriptions held across replacement; stale +SUBSCRIBE/FETCH refusals; and no stale catalog or media on new requests. +Exercise both directions across drafts 17-22, mixed lite-07/IETF relay +chains, and epochless clients. Cover independent epoch/cluster negotiation, +declining the extension on either side, and unchanged legacy framing. +Tests belong in existing CI lanes. Run `just check`, `just drafts check`, +and `just test interop --all`; benchmark any changed routing/cache fanout. + +Public API: expose negotiation through the existing extension configuration +and reuse the epoch model in Rust and JS. Wire: an opt-in IETF identity +extension; no changes to unnegotiated published versions or broadcast paths. + +## Required + +- [Setup extensions](/quest/m1/auth/extensions.md) - the common configuration for offering extensions + +## Related + +- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - current lite-07 identity and legacy restart behavior +- [Claim epochs](/quest/m1/claim-epochs.md) - shared broadcast identity and lite-07 response identity +- [IETF claim epochs](/quest/m1/ietf-claim-epochs.md) - learn per-output identity from an IETF claim response +- [JS track handover](/quest/m1/js-group-handover.md) - a stable JS track pump for same-epoch route changes