diff --git a/doc/bin/relay/cluster.md b/doc/bin/relay/cluster.md index 96434b45c3..eb169faf27 100644 --- a/doc/bin/relay/cluster.md +++ b/doc/bin/relay/cluster.md @@ -29,15 +29,23 @@ has usually removed the other stale routes too, and the relay sends one retraction instead of advertising each stale path in turn. Requests still follow the current best route immediately; only the announcement waits. -A path names one broadcast, whoever publishes it. When the route serving a -broadcast dies, withdraws, or is beaten by a cheaper route, each subscription -continues on the new route from the first frame its readers lack, so they see +Routes carrying the same [publisher epoch](/concept/moq-lite#publisher-epochs) +serve one broadcast. When the route serving it dies, withdraws, or is beaten by +a cheaper route with that epoch, each subscription continues on the new route +from the first frame its readers lack, so they see every frame once, mid-group included. A route that is still up finishes the groups it has open, overlapping the new one. A group neither route delivers is dropped once the readers' max age has passed it. A route through the subscribing peer itself is never used. A publisher whose groups restart, such as an encoder -restarting from group 0, must publish under a new broadcast name; resumed under -the old one, readers wait for its sequence to catch up. +restarting from group 0, publishes under a new epoch, which replaces the old +broadcast instead of resuming it. A route without an epoch, including every +route on moq-lite 06 and older or moq-transport, keeps its subscriptions until it +goes. Seamless failover needs both: a publisher that announces an epoch, and +moq-lite 07 on every link the route crosses, since epochs travel on nothing +older. A cluster that mixes moq-lite 06 and 07 links to the same content has a +second cost: the route that carries the epoch supersedes the one that does not +each time it appears, so a flapping moq-lite 07 link cuts the viewers resolved +through the older one. Failover routes must carry copies of the same broadcast. For each track, the relay requires matching timescale, retention window, publisher priority, and diff --git a/doc/concept/hang.md b/doc/concept/hang.md index 156957142c..51ddd0d346 100644 --- a/doc/concept/hang.md +++ b/doc/concept/hang.md @@ -11,9 +11,9 @@ timestamp. It is modeled on [WebCodecs](https://www.w3.org/TR/webcodecs/) so a browser can decode it directly. The spec is [draft-lcurley-moq-hang](/draft/moq-hang). Plaintext broadcast names end in `.hang` so a player knows which catalog to expect. End-to-end encrypted -broadcasts live under `/`, where `` is derived from the -credential and a semantic name such as `foo.hang`, and one epoch identifies -each publisher run. The path exposes no format or protection marker; the +broadcasts live at ``, derived from the credential and a semantic name +such as `foo.hang`, and the [epoch](/concept/moq-lite#publisher-epochs) on +their route identifies each publisher run. The path exposes no format or protection marker; the payloads follow [moq-e2ee](/draft/moq-e2ee). ## Catalog diff --git a/doc/concept/moq-lite.md b/doc/concept/moq-lite.md index a2292686fb..10e089da1b 100644 --- a/doc/concept/moq-lite.md +++ b/doc/concept/moq-lite.md @@ -97,9 +97,9 @@ always a prefix, on every wire version and on moq-transport alike; a service that serves only some of the paths beneath its prefix refuses the rest as they are requested. Each route carries the chain of relay identities it passed through, which is how forwarding loops are caught, and a cost, which is how a -subscriber picks among several routes to the same broadcast. A path names one -broadcast whoever publishes it, so a subscription moves between routes without -a seam; a publisher must not reuse a path for different content. A hop of 0 is +subscriber picks among several routes to the same broadcast. A route may also +carry a [publisher epoch](#publisher-epochs), which says which routes serve the +same bytes. A hop of 0 is the anonymous mark and travels the chain unchanged; when it is the first hop, a relay puts a random ID, fresh per connection, in front of it. A route that passed through an anonymous hop at any depth ranks below every fully identified route, whatever @@ -113,8 +113,10 @@ compresses when it helps; TypeScript decodes it but always sends literally. A broadcast exists only while it is announced, for consumers in the same process and across a session alike: one that is created but never announced can be neither discovered nor requested. A broadcast published locally -competes with remote routes to its path on cost like any other route, winning -only a tie. Retracting a route (an unannounce, or the peer's `ANNOUNCE_END`) +competes with remote routes to its path like any other route: a newer epoch +beats it whatever the cost, so a local broadcast without an epoch loses to any +remote route that carries one, and among equal epochs it wins a cost tie. +Retracting a route (an unannounce, or the peer's `ANNOUNCE_END`) stops new requests from resolving through it but leaves subscriptions already in flight alone: each track runs to its own end or failure, or until its last subscriber leaves, which cancels it upstream. On moq-lite 05 and @@ -136,22 +138,44 @@ without waiting, since it has no FIN to acknowledge. ### Publisher epochs -An application can identify each publisher instance with a shared `Epoch` from -`moq-net` or `@moq/net`. It is a lowercase hyphenated UUIDv7, ordered newest -last, with its wall-clock creation time available as `Epoch::time()` in Rust -or `Epoch.time(epoch)` in TypeScript. Minting is explicit; publishing does not -add an epoch automatically. - -`Path::join_epoch(Some(&epoch))` and `Path.joinEpoch(name, epoch)` append an -`@` segment. `Path::split_epoch()` and `Path.splitEpoch(path)` return -the name and optional epoch. Only the final segment and canonical UUIDv7 text -count: `@alice`, bare UUIDs, uppercase UUIDs, and other UUID versions remain -ordinary path segments. Existing path normalization still applies. - -The segment travels as part of the ordinary broadcast path on every supported -wire version. Pattern grants still match the full path: `room/**` covers an -epoch-qualified instance, while `room/camera` is an exact name. Epochs do not -hide a broadcast; a leading `.` in a name segment still does. +A route may carry an `Epoch`: a UUIDv7 naming the publisher instance behind it, +ordered newest last, with its creation time available as `Epoch::time()` in +Rust or `Epoch.time(epoch)` in TypeScript. A path and an epoch name one +broadcast. The path never changes, so authorization, patterns, and hidden +names work exactly as without one. + +- **Publishing**: the epoch is whatever the route names, and nothing is minted + for you. A publisher mints one per run (`Epoch::mint()` with + `Route::with_epoch`, or `announce({ epoch: Epoch.mint() })`), so a restart is + a new broadcast. Replicas of the same content announce the same one. A route + without an epoch, such as a prefix claim, names no instance. +- **Resolution** ranks the newest epoch first among routes at the same prefix, + ahead of cost, and a route without an epoch last. Newest means the latest + UUIDv7 timestamp, so ordering instances minted on different hosts trusts + their clocks: a restart on a host whose clock runs behind loses to the old + instance until that one retracts. +- **Re-pricing**: announcing again (or `update` on a dynamic route) replaces + the whole route, epoch included. Start from the current route, as in + `broadcast.route()` or `dynamic.route()`, and change the cost, or the update + names another instance. +- **Resume**: a subscription moves between routes with the same epoch without a + seam, continuing from the first frame it lacks. A route without an epoch + names no instance, so its subscriptions stay on that route (a transcoder + claim stays on the worker that first served it) and end when it goes. +- **Replacement**: when a newer epoch wins the path, subscriptions to the old + one end with `Unroutable`, even ones in flight, and announcement streams see + the old route end and the new one start. Clients re-request and get the new + broadcast instead of stalling on the old. + +The epoch travels on moq-lite 07 in `ANNOUNCE_START`, and relays fill it into +the `TRACK`, `SUBSCRIBE`, and `FETCH` they send upstream; a publisher refuses a +request naming another epoch. Older moq-lite versions and moq-transport cannot +carry it: their routes have no epoch, so they keep their subscriptions on one +route and see a replacement as an ordinary end and start at the same path. In +a mixed deployment, where the same content reaches a relay over both an older +link (no epoch) and a moq-lite 07 link (with one), the epoch-carrying route +supersedes the other each time it appears, so a flapping moq-lite 07 link cuts +the viewers resolved through the older one. ### Hidden broadcasts diff --git a/doc/lib/js/net.md b/doc/lib/js/net.md index d0515a7311..c47d5b01ee 100644 --- a/doc/lib/js/net.md +++ b/doc/lib/js/net.md @@ -9,9 +9,8 @@ description: The pub/sub layer in TypeScript The TypeScript twin of [`moq-net`](/lib/rs/moq-net): connections, origins, broadcasts, tracks, groups, and frames, negotiating moq-lite or moq-transport -at setup. `Epoch` provides the shared publisher identity, and `Path.splitEpoch` -and `Path.joinEpoch` carry it in the broadcast path; see -[publisher epochs](/concept/moq-lite#publisher-epochs). +at setup. `Epoch` provides the shared publisher identity, carried on each +route as `route.epoch`; see [publisher epochs](/concept/moq-lite#publisher-epochs). ```ts import * as Moq from "@moq/net"; @@ -93,7 +92,11 @@ Three operations, on an origin: invisible and unreachable, for local consumers and peers alike, until `broadcast.announce()`. - `broadcast.announce(route)` / `broadcast.unannounce()` own that - advertisement. Announcing again re-prices the standing route. + advertisement. Announcing again replaces the standing route as given, epoch + included, so re-price from the current one + (`broadcast.announce({ ...broadcast.route, cost })`); another epoch, or + none, announces a new broadcast. `Origin.Dynamic` has the same `route` and + `update(route)`. - `origin.dynamic(prefix, route)` claims `prefix` and every path beneath it (`""` claims everything). Hold the returned `Origin.Dynamic` while the claim should stay advertised; `close()` retracts it. A request beneath it diff --git a/doc/lib/rs/moq-net.md b/doc/lib/rs/moq-net.md index 8c030f915a..f1c4e90fb6 100644 --- a/doc/lib/rs/moq-net.md +++ b/doc/lib/rs/moq-net.md @@ -17,7 +17,7 @@ above ([hang](/lib/rs/hang)); relays and CDNs implement only this. - **Origins** scope what a session can see, and merge duplicate subscriptions so a broadcast is pulled upstream once no matter how many local readers. - **Broadcasts** are created unannounced and invisible to everyone, then announced as an exact route, or served below a prefix with `dynamic`. A consumer of the same origin sees exactly what a peer sees. Discovery accepts pattern unions; events carry the advertised prefix and captures for a complete match. -- **Epochs** identify publisher instances with a canonical UUIDv7 and explicit path helpers; see [publisher epochs](/concept/moq-lite#publisher-epochs). +- **Epochs** identify publisher instances with a canonical UUIDv7 carried on each route (`Route::epoch`): the newest wins a path, and only routes with the same epoch resume a subscription; see [publisher epochs](/concept/moq-lite#publisher-epochs). - **Patterns** (`Pattern`, `Patterns`) are re-exported from [`moq-pattern`](https://docs.rs/moq-pattern). Literal `Path` stays a coordinate. - **Tracks** carry groups with a priority, an optional publisher retention window (`Info::max_age`), and a timescale. Subscribers set their own priority and max age and can change them live. - **Groups** are written frame by frame and delivered on independent streams. Old groups are cached for fetch-by-sequence; stale groups are skipped per the subscriber's budget. @@ -137,9 +137,13 @@ Three operations, on an origin: invisible and unroutable, for local consumers and peers alike, until `broadcast.announce(route)`. - `broadcast.announce(route)` / `broadcast.unannounce()` own that - advertisement. Announcing again re-prices the standing route, which competes - on cost with remote routes at the same path (a tie goes to the local - broadcast). The route retracts on `unannounce()`, `close()`, or the last + advertisement. Announcing again replaces the standing route as given, epoch + included, so re-price from the current one + (`broadcast.announce(broadcast.route().unwrap_or_default().with_cost(c))`); + another epoch, or none, announces a new broadcast; `origin::Dynamic` has the + same `route()` and `update(route)`. The route competes with + remote routes at the same path: the newest epoch wins, then the cheapest (a + tie goes to the local broadcast). The route retracts on `unannounce()`, `close()`, or the last producer dropping; tracks already in flight carry on to their own end. - `broadcast.close()` ends the broadcast for good: it retracts, leaves local discovery, and answers every later track lookup with `Unroutable`. Tracks diff --git a/drafts/draft-lcurley-moq-e2ee.md b/drafts/draft-lcurley-moq-e2ee.md index bdb5af1326..a818e2a1fd 100644 --- a/drafts/draft-lcurley-moq-e2ee.md +++ b/drafts/draft-lcurley-moq-e2ee.md @@ -132,13 +132,13 @@ Each instance of a broadcast MUST mint an epoch that no other instance under the Two instances MUST NOT share an epoch: they would derive the same keys and collide on nonces. Its leading 48-bit timestamp makes epochs sort by creation time and its random bits make collisions negligible. -A protected broadcast is published at `/@`, where `` is the 22-character base64url segment derived from the credential and the application's semantic broadcast name according to {{derive}}, and `` is the UUID text. -The `@` marker is part of the path segment only; the HKDF `bytes(epoch)` input remains the UUID text without `@`. -The opaque derivation does not include the epoch, so every instance of the same semantic broadcast shares a discovery prefix. +A protected broadcast is published at ``, the 22-character base64url segment derived from the credential and the application's semantic broadcast name according to {{derive}}. +Each instance announces its epoch as the route's moq-lite Epoch; the HKDF `bytes(epoch)` input is the UUID text. +The opaque derivation does not include the epoch, so every instance of the same semantic broadcast shares one path. The path carries no format or protection marker; for example, the semantic name `meeting.hang` appears only as an input to the opaque derivation. A plaintext consumer that opens the protected broadcast fails because it cannot find the plaintext catalog it expects, not because of a path naming rule. -Subscribers discover instances by the `/` prefix and select the greatest epoch, where greatest is newest. -A subscriber that already knows the full path parses the epoch from its last segment after removing `@`. +Subscribers resolve `` and take the epoch from its route, where relays rank the newest first. +On a wire that cannot carry the route's epoch, the application supplies it over its own authenticated channel. The epoch and the path are not secret and are not authenticated. A relay that presents a wrong epoch causes authentication failure; a relay that withholds a newer instance denies service. @@ -146,7 +146,7 @@ Neither can cause a nonce to repeat, because only the publisher instance chooses A restart or replacement of a publisher is a new instance and mints a new epoch. Transport sequence numbers therefore restart freely without any coordination between instances. -Ended instances remain readable at their own path for as long as relays or archives retain them. +An ended instance stays readable only for as long as relays or archives retain it under its epoch; once a newer epoch wins the shared path, subscribers resolve the newer one. # Canonical Encoding {#encoding} @@ -364,8 +364,8 @@ This document requests no registrations. ## draft-lcurley-moq-e2ee-00 {:numbered="false"} -- Require canonical UUIDv7 epochs and carry them in `@` path segments, leaving HKDF UUID inputs unchanged. -- Initial `moq-e2ee-00` profile: out-of-band credential, opaque broadcast path with a publisher-minted epoch as its last segment, HKDF physical names and keys, AES-128-GCM payloads, identity bounds, typed failures, and shared primitive vectors. +- Require canonical UUIDv7 epochs, carried on the route rather than in the path, leaving HKDF UUID inputs unchanged. +- Initial `moq-e2ee-00` profile: out-of-band credential, opaque broadcast path with a publisher-minted epoch on its route, HKDF physical names and keys, AES-128-GCM payloads, identity bounds, typed failures, and shared primitive vectors. # Acknowledgments diff --git a/drafts/draft-lcurley-moq-lite.md b/drafts/draft-lcurley-moq-lite.md index 08eab96eca..e9bccd9014 100644 --- a/drafts/draft-lcurley-moq-lite.md +++ b/drafts/draft-lcurley-moq-lite.md @@ -31,6 +31,7 @@ normative: RFC6455: RFC9000: RFC9002: + RFC9562: informative: I-D.lcurley-moq-cluster: @@ -118,7 +119,9 @@ Any broadcasts and subscriptions are transparently proxied by the CDN behind the ## Broadcast A Broadcast is a collection of Tracks named by a path. This corresponds to a MoqTransport's "track namespace". -A path names one Broadcast whichever publisher serves it: a publisher MUST NOT reuse a path for different content, and publishes a new instance under a new path instead. +A publisher instance is identified by an Epoch, carried on its routes (see [ANNOUNCE_START](#announce-start)): a path and an Epoch name one Broadcast, and a publisher MUST NOT serve different content under the same pair. +A restarted publisher mints a new Epoch rather than reusing the path's old one, while replicas of the same content MAY share one. +A route without an Epoch names no instance: nothing says another route serves the same bytes. A publisher advertises what it can serve via ANNOUNCE_START messages, each carrying a path prefix: a route covering every broadcast path beneath it. A route is the only shape an advertisement takes: a publisher that serves only some of the paths beneath a prefix, such as a transcoder for any broadcast's derivative, advertises the covering prefix and refuses the requests it will not serve (see [Resolution](#resolution)); no message narrows a route. @@ -421,13 +424,17 @@ The per-subscriber winner changing travels as an ANNOUNCE_UPDATE; the last quali When serving a subscription, a publisher MUST select the source by that same exclusion; if only excluded sources remain, the subscription is unroutable. Applying one rule to both advertisement and dispatch keeps advertised paths truthful, which is what prevents subscription cycles of any length. -When resolving a path covered by several routes (across any number of streams), the subscriber SHOULD prefer the most specific covering route (see [Resolution](#resolution)), then a path that contains no 0 Hop ID over one that does, then the lowest Warm Route Cost after adding each arriving link's cost (see [Cost Parameter](#cost-parameter)), breaking ties toward the lowest Cold Route Cost, then toward the shortest path, then toward the lowest Spread Hash, and then toward the most recently received, so a reconnecting publisher is not outranked by the stale session it replaced. +When resolving a path covered by several routes (across any number of streams), the subscriber SHOULD prefer the most specific covering route (see [Resolution](#resolution)), then the newest Epoch, ranking a route without one last, then a path that contains no 0 Hop ID over one that does, then the lowest Warm Route Cost after adding each arriving link's cost (see [Cost Parameter](#cost-parameter)), breaking ties toward the lowest Cold Route Cost, then toward the shortest path, then toward the lowest Spread Hash, and then toward the most recently received, so a reconnecting publisher is not outranked by the stale session it replaced. The Spread Hash is the 64-bit FNV-1a hash, with offset basis `0x420C0DECB00B` and the standard FNV-64 prime, of the requested path's UTF-8 bytes followed by each Hop ID of the route's path, oldest first, as 8 little-endian bytes. It is keyed on the requested path rather than the route's prefix, so equal-cost advertisers of one prefix share its paths instead of the first one taking them all, while one path resolves to the same advertiser on every relay that holds the same routes. When choosing which route to advertise for a prefix, the requested path is the prefix itself. -Every route covering a path serves the same Broadcast, so a relay MAY move a live subscription between them, continuing from the first frame the subscriber lacks, and a route change (a reconnect, a cheaper path, a draining session) is invisible to the subscriber. +Routes with the same Epoch serve the same Broadcast, so a relay MAY move a live subscription between them, continuing from the first frame the subscriber lacks, and a route change (a reconnect, a cheaper path, a draining session) is invisible to the subscriber. +A relay MUST NOT move a subscription to a route with a different Epoch, or between routes without one: it stays on its route and ends with it. +When a newer Epoch wins the path, the relay SHOULD end subscriptions to the older one, even ones in flight, with UNROUTABLE, so subscribers request the new Broadcast rather than stall on one that was replaced. +Epochs compare as their 16 bytes, which for a UUIDv7 orders them by creation time. +Ordering Epochs minted on different hosts therefore trusts their clocks: a restart on a host whose clock runs behind loses to the older instance until that one is retracted. #### Resolution {#resolution} A SUBSCRIBE, FETCH, or TRACK request names a path, and the receiver resolves it against the routes covering that path, after the per-subscriber exclusion above. @@ -884,6 +891,7 @@ ANNOUNCE_START Message { Path Base (i), Path Keep (i), Route Prefix Suffix (s), + Epoch (b), Hops (..), Warm Route Cost (i), Cold Route Cost (i), @@ -907,6 +915,12 @@ The suffix begins with the first `Path Keep` segments of the suffix advertised b The remaining segments of the suffix, which is combined with the requested prefix to form the route's full prefix. An empty suffix advertises the requested prefix itself, which is how a route covering more than the request presents (see [Announce](#announce)). +**Epoch**: +The publisher instance the route serves: 16 bytes holding a UUIDv7 [RFC9562], or empty when unknown. +Any other length, or a UUID of another version or variant, is a PROTOCOL_VIOLATION. +A relay forwards the Epoch unchanged and MUST NOT invent one for a route that arrived without one. +The Epoch is fixed for the advertisement's lifetime: a publisher replacing it sends ANNOUNCE_END and a new ANNOUNCE_START, and ANNOUNCE_UPDATE never changes it. + **Hop Base** and **Hop Keep**: The Hop ID list ends with the last `Hop Keep` entries of the list advertised by the base that `Hop Base` names (see [Compression](#announce-compression)), following the literal `Hop ID` entries. The two bases are independent. @@ -1000,6 +1014,7 @@ SUBSCRIBE Message { Message Length (i) Subscribe ID (i) Broadcast Path (s) + Epoch (b) Track Name (s) Subscriber Priority (8) Subscriber Max Age (i) @@ -1085,6 +1100,7 @@ It is the first message on a Track Stream (0x6). TRACK Message { Message Length (i) Broadcast Path (s) + Epoch (b) Track Name (s) } ~~~ @@ -1092,6 +1108,11 @@ TRACK Message { **Broadcast Path**: The broadcast path of the track. +**Epoch**: +The publisher instance the subscriber expects, encoded as in [ANNOUNCE_START](#announce-start), or empty for whichever wins the path. +A publisher MUST refuse a request with UNROUTABLE when the route it would serve from has another Epoch, so a request is never answered by a different instance than the one it named. +A relay fills in the Epoch of the route it resolved when forwarding a request upstream; SUBSCRIBE and FETCH carry it the same way. + **Track Name**: The name of the track. @@ -1211,6 +1232,7 @@ FETCH is sent by a subscriber to request a single group from a track. FETCH Message { Message Length (i) Broadcast Path (s) + Epoch (b) Track Name (s) Subscriber Priority (8) Group Sequence (i) @@ -1362,7 +1384,7 @@ The `Message Length` describes the payload size on the wire. - A refusal is not retried at another route of the same prefix either. - Made TRACK_INFO Publisher Max Age optional, encoded as milliseconds plus one with zero meaning no limit. - Added `Largest Group` and `Largest Frame` to SUBSCRIBE_OK: the publisher's largest position when it answers, which a subscriber takes as where the live feed is. A publisher MUST answer at once when the requested start is past it. Earlier versions carry no such position, so a subscriber takes the first frame instead. -- A path names one Broadcast whichever publisher serves it, and a publisher MUST NOT reuse a path for different content. A relay MAY move a subscription between any routes covering the path, continuing from the first frame the subscriber lacks instead of at a group boundary. Replaces the first-hop identity. +- Added `Epoch` to ANNOUNCE_START, SUBSCRIBE, TRACK, and FETCH: a UUIDv7 naming the publisher instance, or empty. A path and an Epoch name one Broadcast. A relay MAY move a subscription between routes with the same Epoch, continuing from the first frame the subscriber lacks instead of at a group boundary, and never between routes with different Epochs or none. The newest Epoch wins a path and ends subscriptions to the older one. Replaces the first-hop identity. - Assigned `moq-lite-07-wip` as this draft's protocol identifier until it is finalized as `moq-lite-07`. - Switched every variable-length integer, including SETUP parameter values, from QUIC's two-bit length prefix to moq-transport's leading-ones encoding, widening the range to 64 bits. diff --git a/js/clock/src/main.ts b/js/clock/src/main.ts index 73d0cbc0c6..6d32da417a 100755 --- a/js/clock/src/main.ts +++ b/js/clock/src/main.ts @@ -79,7 +79,8 @@ async function publish(config: Config) { // Create a new "broadcast", which is a collection of tracks. const broadcast = origin.createBroadcast(Moq.Path.from(config.broadcast)); - broadcast.announce(); + // A fresh epoch per run, so a restarted clock replaces the old broadcast. + broadcast.announce({ epoch: Moq.Epoch.mint() }); console.log("✅ Published broadcast:", config.broadcast); diff --git a/js/moq-boy/src/game.ts b/js/moq-boy/src/game.ts index 05caf1071a..660026da76 100644 --- a/js/moq-boy/src/game.ts +++ b/js/moq-boy/src/game.ts @@ -302,7 +302,7 @@ export class Game { const viewerBroadcast = origin.createBroadcast( Moq.Path.from(`${this.#viewerPrefix}/${this.sessionId}/${viewerId}`), ); - viewerBroadcast.announce(); + viewerBroadcast.announce({ epoch: Moq.Epoch.mint() }); effect.cleanup(() => { viewerBroadcast.close(); this.viewerId.set(undefined); diff --git a/js/net/src/broadcast.ts b/js/net/src/broadcast.ts index 99efcd09c3..d752df4ee5 100644 --- a/js/net/src/broadcast.ts +++ b/js/net/src/broadcast.ts @@ -19,6 +19,8 @@ export interface Announcer { announce(route: Route): void; /** Retract the advertisement from local consumers and peers alike. */ unannounce(): void; + /** The route this broadcast's path is advertised with, if it is. */ + route(): Route | undefined; } let attachAnnouncer: (producer: Producer, announcer: Announcer) => void; @@ -324,15 +326,26 @@ export class Producer { }; } + /** The route this broadcast is announced with, or undefined while it is not announced. */ + get route(): Route | undefined { + return this.#announcer?.route(); + } + /** - * Advertise this broadcast's exact path, or re-price a standing advertisement in place. + * Advertise this broadcast's exact path, or replace the standing advertisement's route. + * + * The route is taken as given, epoch included: a route with another epoch (or none) + * announces a new broadcast, so re-price from the current one, + * `announce({ ...broadcast.route, cost })`, to keep the instance. * * Call it once the tracks a subscriber needs first (a catalog) exist. Until then the * broadcast exists for nobody, on its own origin or at a peer. Retracts on * {@link unannounce} or {@link close}. Throws if this producer was not created through an * origin, or if the broadcast is already closed. */ - announce(route: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default): void { + announce( + route: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default, + ): void { if (this.#state.closed.peek() !== undefined) { throw new Error("broadcast is closed"); } diff --git a/js/net/src/epoch.test.ts b/js/net/src/epoch.test.ts index e4d4cf27c6..c6b002642a 100644 --- a/js/net/src/epoch.test.ts +++ b/js/net/src/epoch.test.ts @@ -1,14 +1,38 @@ -import { expect, spyOn, test } from "bun:test"; +import { expect, test } from "bun:test"; import * as Epoch from "./epoch.ts"; -test("mint uses UUIDv7 and orders successive identities with a fixed wall clock", () => { - const now = spyOn(Date, "now").mockReturnValue(1_700_000_000_000); - try { - const epoch = Epoch.mint(); - expect(Epoch.parse(epoch)).toBe(epoch); - expect(Epoch.time(epoch).getTime()).toBe(1_700_000_000_000); - expect(Epoch.mint() > epoch).toBe(true); - } finally { - now.mockRestore(); +test("mint uses UUIDv7 from the wall clock and orders successive identities", () => { + // No mocked clock: UUIDv7 never moves backwards, so any epoch minted earlier in the + // process would pin a mocked past time to the newest one seen. + const before = Date.now(); + const epoch = Epoch.mint(); + expect(Epoch.parse(epoch)).toBe(epoch); + expect(Epoch.time(epoch).getTime()).toBeGreaterThanOrEqual(before); + expect(Epoch.mint() > epoch).toBe(true); +}); + +// Shared with rs/moq-net's `epoch_vectors`, so both parse and order alike. +const vectors = (await Bun.file(new URL("../../../rs/moq-net/src/epoch.json", import.meta.url)).json()) as { + valid: Array<{ text: string; unix_ms: number }>; + invalid: string[]; + ordered: string[]; +}; + +test("shared epoch parse, reject, order and time vectors", () => { + for (const row of vectors.valid) { + const epoch = Epoch.parse(row.text); + expect(epoch).toBe(row.text as Epoch.Valid); + expect(Epoch.time(epoch).getTime()).toBe(row.unix_ms); + expect(Epoch.fromBytes(Epoch.toBytes(epoch))).toBe(epoch); } + for (const text of vectors.invalid) expect(() => Epoch.parse(text)).toThrow(RangeError); + const ordered = vectors.ordered.map(Epoch.parse); + expect([...ordered].reverse().sort()).toEqual(ordered); +}); + +test("fromBytes refuses anything but 16 bytes of UUIDv7", () => { + expect(() => Epoch.fromBytes(new Uint8Array(15))).toThrow(RangeError); + const v4 = Epoch.toBytes(Epoch.mint()); + v4[6] = (v4[6] & 0x0f) | 0x40; + expect(() => Epoch.fromBytes(v4)).toThrow(RangeError); }); diff --git a/js/net/src/epoch.ts b/js/net/src/epoch.ts index f3782d9c99..f454f52979 100644 --- a/js/net/src/epoch.ts +++ b/js/net/src/epoch.ts @@ -7,7 +7,7 @@ export type Valid = string & { readonly _brand: "epoch" }; const canonical = /^[0-9a-f]{8}-[0-9a-f]{4}-7[0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/; -/** Parse UUID text without the path's `@` marker, refusing noncanonical forms. */ +/** Parse UUID text, refusing noncanonical forms. */ export function parse(text: string): Valid { if (text.length !== 36 || !canonical.test(text)) throw new RangeError("invalid epoch: expected a lowercase hyphenated UUIDv7"); @@ -23,3 +23,18 @@ export function mint(): Valid { export function time(epoch: Valid): Date { return new Date(Number.parseInt(epoch.slice(0, 8) + epoch.slice(9, 13), 16)); } + +/** The UUID's 16 bytes, as the wire carries them. */ +export function toBytes(epoch: Valid): Uint8Array { + const hex = epoch.replaceAll("-", ""); + const bytes = new Uint8Array(16); + for (let i = 0; i < 16; i++) bytes[i] = Number.parseInt(hex.slice(i * 2, i * 2 + 2), 16); + return bytes; +} + +/** Parse the wire's 16 bytes, refusing anything but a UUIDv7 with the RFC variant. */ +export function fromBytes(bytes: Uint8Array): Valid { + if (bytes.length !== 16) throw new RangeError("invalid epoch: expected 16 bytes"); + const hex = Array.from(bytes, (byte) => byte.toString(16).padStart(2, "0")).join(""); + return parse(`${hex.slice(0, 8)}-${hex.slice(8, 12)}-${hex.slice(12, 16)}-${hex.slice(16, 20)}-${hex.slice(20)}`); +} diff --git a/js/net/src/hop.ts b/js/net/src/hop.ts index b5e69ebc76..fe38b8de6c 100644 --- a/js/net/src/hop.ts +++ b/js/net/src/hop.ts @@ -8,6 +8,7 @@ * @module */ import * as z from "zod/mini"; +import type * as Epoch from "./epoch.ts"; /** * One relay's identity in a broadcast's hop chain, encoded as a 62-bit varint on the wire. @@ -112,6 +113,13 @@ export const Cost = { * an announce event carries it so consumers can read it back. */ export interface Route { + /** + * The publisher instance the route serves, if known: routes with the same epoch serve + * the same bytes. Among routes at one prefix the newest epoch wins, and a route without + * one ranks last. Fixed for the advertisement's lifetime: changing it retracts and + * announces afresh. Mirrors `Route::epoch` in rs/moq-net. + */ + epoch?: Epoch.Valid; /** The chain of hops the route has traversed, oldest first. */ hops: Hop[]; /** What pulling content via this route costs; lower wins. */ @@ -124,12 +132,13 @@ export const Route = { default: { hops: [], cost: Cost.zero } as Route, /** Normalize a partial route, treating a bare bigint cost as both magnitudes alike. */ - normalize(route: Route | { hops?: readonly Hop[]; cost?: Cost | bigint } = {}): Route { + normalize(route: Route | { epoch?: Epoch.Valid; hops?: readonly Hop[]; cost?: Cost | bigint } = {}): Route { const hops = route.hops ? [...route.hops] : []; + const epoch = route.epoch; const cost = route.cost; - if (cost === undefined) return { hops, cost: Cost.zero }; - if (typeof cost === "bigint") return { hops, cost: { warm: cost, cold: cost } }; - return { hops, cost: { warm: cost.warm, cold: cost.cold } }; + if (cost === undefined) return { epoch, hops, cost: Cost.zero }; + if (typeof cost === "bigint") return { epoch, hops, cost: { warm: cost, cold: cost } }; + return { epoch, hops, cost: { warm: cost.warm, cold: cost.cold } }; }, }; @@ -143,11 +152,12 @@ export function isAnonymous(route: Route): boolean { return route.hops.includes(UNKNOWN_HOP); } -/** Whether two routes name the same hop chain and cost. */ +/** Whether two routes name the same epoch, hop chain, and cost. */ export function routesEqual(a: Route | undefined, b: Route | undefined): boolean { if (a === b) return true; if (!a || !b) return false; return ( + a.epoch === b.epoch && a.cost.warm === b.cost.warm && a.cost.cold === b.cost.cold && a.hops.length === b.hops.length && diff --git a/js/net/src/ietf/publisher.test.ts b/js/net/src/ietf/publisher.test.ts index 82590e9d9c..0acea570e7 100644 --- a/js/net/src/ietf/publisher.test.ts +++ b/js/net/src/ietf/publisher.test.ts @@ -1,6 +1,7 @@ import { expect, jest, mock, spyOn, test } from "bun:test"; import { Signal } from "@moq/signals"; import type { Producer as BroadcastProducer } from "../broadcast.ts"; +import * as Epoch from "../epoch.ts"; import { error } from "../error.ts"; import { Producer as GroupProducer, MAX_GROUP_FRAMES } from "../group.ts"; import { type Hop, HopSchema } from "../hop.ts"; @@ -884,6 +885,36 @@ test("republishing a path clears a refusal without unannouncing first", async () origin.close(); }); +/** + * A new epoch on the same broadcast handle is another publisher instance, so a refusal of + * the old one does not strand it. Rust gets this from the END and START an epoch change + * delivers. + */ +test("a new epoch at the same path clears a refusal", async () => { + const pair = createMockTransportPair(ALPN.DRAFT_19); + const { pub, origin } = publisher(pair.server); + + const broadcast = origin.createBroadcast(Path.from("restarted")); + broadcast.announce({ epoch: Epoch.mint() }); + + void pub.runPublishNamespaces(); + + const declined = await nextStream(pair.client); + if (!declined) throw new Error("the namespace was never advertised"); + expect(await readPublishNamespace(declined)).toBe(Path.from("restarted")); + await declinePublishNamespace(declined, 0n); + await new Promise((resolve) => setTimeout(resolve, SETTLE)); + + broadcast.announce({ ...broadcast.route, epoch: Epoch.mint() }); + + const retried = await nextStream(pair.client); + if (!retried) throw new Error("the new instance was never offered"); + expect(await readPublishNamespace(retried)).toBe(Path.from("restarted")); + await acceptPublishNamespace(retried); + + origin.close(); +}); + /** * A refusal belongs to the namespace, not the path forever. Unannouncing takes it with * it, so a fresh broadcast at the same path is offered again; keeping it would strand diff --git a/js/net/src/ietf/publisher.ts b/js/net/src/ietf/publisher.ts index a6a5cccd4a..061b976fe6 100644 --- a/js/net/src/ietf/publisher.ts +++ b/js/net/src/ietf/publisher.ts @@ -73,6 +73,8 @@ function change( next: Advertised | undefined, ): "same" | "restart" | Reprice { if (held === undefined || next === undefined || held.identity !== next.identity) return "restart"; + // A new epoch is another broadcast, even from the same entry. + if (held.route.epoch !== next.route.epoch) return "restart"; const from = clusterFor(base, held.route); const to = clusterFor(base, next.route); if (from === undefined || to === undefined) return from === to ? "same" : "restart"; @@ -130,8 +132,8 @@ interface Namespaces { /** What the peer refused, and whether coming back is worth anything. */ refused: Map; - /** The identity each refusal was about, so a republish at the same path clears it. */ - offered: Map; + /** The advertisement each refusal was about, so a republish or new epoch at the same path clears it. */ + offered: Map; } /** Where one announce loop sends its advertisements. */ @@ -1093,7 +1095,9 @@ export class Publisher { // by rebuilding the watched entry. for (const key of [...ns.refused.keys()]) { const snap = updated.get(key); - if (snap === undefined || ns.offered.get(key) !== snap.identity) { + const offered = ns.offered.get(key); + // A new epoch is another broadcast too, even from the same front. + if (snap === undefined || offered?.identity !== snap.identity || offered.route.epoch !== snap.route.epoch) { ns.refused.delete(key); ns.offered.delete(key); } @@ -1128,7 +1132,7 @@ export class Publisher { held.delete(key); if (answer !== "dropped") { ns.refused.set(key, answer); - ns.offered.set(key, snap.identity); + ns.offered.set(key, snap); } } } diff --git a/js/net/src/integration.test.ts b/js/net/src/integration.test.ts index 722bcac4d3..d2807d97eb 100644 --- a/js/net/src/integration.test.ts +++ b/js/net/src/integration.test.ts @@ -10,6 +10,7 @@ import { connect as connectSession, type Established, } from "./connection/index.ts"; +import * as Epoch from "./epoch.ts"; import { SessionCode, SessionError, StreamCode, StreamError, TooFarBehind } from "./error.ts"; import { Producer as GroupProducer } from "./group.ts"; import * as Ietf from "./ietf/index.ts"; @@ -816,9 +817,11 @@ test("integration: a peer is served the cheaper route it was offered, not the lo const pair = createMockTransportPair(Lite.ALPN_06); const origin = new OriginProducer(); const path = Path.from("test"); + // One publisher instance, so the two compete on cost alone. + const epoch = Epoch.mint(); const local = origin.createBroadcast(path); - local.announce({ cost: 5n }); - const dynamic = origin.dynamic(path, { cost: 1n }); + local.announce({ epoch, cost: 5n }); + const dynamic = origin.dynamic(path, { epoch, cost: 1n }); const [client, server] = await Promise.all([ connect(url, { transport: pair.client }), diff --git a/js/net/src/internal.ts b/js/net/src/internal.ts index e809fe0e98..153fe38a99 100644 --- a/js/net/src/internal.ts +++ b/js/net/src/internal.ts @@ -177,7 +177,7 @@ export const hooks: { /** Attach the origin advertisement of a created broadcast. */ attachAnnouncer: ( producer: BroadcastProducer, - announcer: { announce(route: Route): void; unannounce(): void }, + announcer: { announce(route: Route): void; unannounce(): void; route(): Route | undefined }, ) => void; /** Name a broadcast handle by the path an origin created or resolved it at. */ stampPath: (target: BroadcastProducer | BroadcastConsumer, path: Path.Valid) => void; diff --git a/js/net/src/lite/announce.test.ts b/js/net/src/lite/announce.test.ts index 86b212d308..12a0044ff2 100644 --- a/js/net/src/lite/announce.test.ts +++ b/js/net/src/lite/announce.test.ts @@ -287,7 +287,7 @@ const GOLDEN_RESOLVED = [ // Pinned from the Rust encoder (`lite::compress::tests::golden_stream_is_pinned`), so the // JS decoder is checked against real compressed output. const GOLDEN = - "001600000a726f6f6d2f612f63616d00029111a222000000000d0102036d69630101b3330100000209000101c04444010000010101000c020101620201c05555010000"; + "001700000a726f6f6d2f612f63616d0000029111a222000000000e0102036d6963000101b3330100000209000101c04444010000010101000d02010162000201c05555010000"; test("AnnounceHistory resolves the Rust encoder's compressed stream", async () => { expect(await resolveStream(unhex(GOLDEN))).toEqual(GOLDEN_RESOLVED); @@ -295,7 +295,7 @@ test("AnnounceHistory resolves the Rust encoder's compressed stream", async () = // JS always encodes literally; Rust decodes these bytes too (`js_literal_stream_decodes`). const JS_LITERAL = - "001600000a726f6f6d2f612f63616d00029111a222000000001600000a726f6f6d2f612f6d69630002b333a222000000020b000002c04444a2220000000101010013000006726f6f6d2f620002c05555a222000000"; + "001700000a726f6f6d2f612f63616d0000029111a222000000001700000a726f6f6d2f612f6d6963000002b333a222000000020b000002c04444a2220000000101010014000006726f6f6d2f62000002c05555a222000000"; test("the literal draft-07 stream matches what Rust decodes", async () => { const wire = await bytes(async (w) => { @@ -391,12 +391,12 @@ test("route costs saturate at 2^62-1 on every version", async () => { expect(got).toMatchObject({ cost: { warm: ceiling, cold: ceiling } }); } - // ANNOUNCE_START: path base, path keep, empty suffix, hop base, no hops, hop keep, then - // warm and cold at 2^64-1, which only lite-07's varints can carry. + // ANNOUNCE_START: path base, path keep, empty suffix, no epoch, hop base, no hops, hop + // keep, then warm and cold at 2^64-1, which only lite-07's varints can carry. const wire = await bytes(async (w) => { await w.u53(0); - await w.u53(24); - for (const b of [0, 0, 0, 0, 0, 0]) await w.u8(b); + await w.u53(25); + for (const b of [0, 0, 0, 0, 0, 0, 0]) await w.u8(b); await w.u62(huge); await w.u62(huge); }, Version.DRAFT_07); diff --git a/js/net/src/lite/announce.ts b/js/net/src/lite/announce.ts index 69ee0c0944..c77b4b3ea0 100644 --- a/js/net/src/lite/announce.ts +++ b/js/net/src/lite/announce.ts @@ -1,7 +1,9 @@ +import type * as Epoch from "../epoch.ts"; import { ProtocolViolation } from "../error.ts"; import { type Cost, type Hop, HopSchema, MAX_HOPS, UNKNOWN_HOP } from "../hop.ts"; import * as Path from "../path.ts"; import type { Reader, Writer } from "../stream.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; import { hasAnnounceCompression, @@ -48,8 +50,17 @@ export type AnnounceBroadcast = /** A broadcast is now available, carrying the path suffix, the hop chain, and * (lite-06+) the route cost. An absent cost encodes as zero; it decodes as * `undefined` on a wire with no room for one. On lite-07, `suffix` follows the - * segments `pathBase` copies and `hops` precede the ones `hopBase` copies. */ - | { status: "active"; suffix: Path.Valid; hops: Hop[]; cost?: Cost; pathBase?: Base; hopBase?: Base } + * segments `pathBase` copies and `hops` precede the ones `hopBase` copies. The epoch + * (lite-07+) is fixed for the announcement's lifetime: a new one ends it and starts afresh. */ + | { + status: "active"; + suffix: Path.Valid; + epoch?: Epoch.Valid; + hops: Hop[]; + cost?: Cost; + pathBase?: Base; + hopBase?: Base; + } /** Pre-lite-06: a broadcast is no longer available, retracted by path. */ | { status: "ended"; suffix: Path.Valid } /** Lite06+: a broadcast is no longer available, retracted by announce id. @@ -197,6 +208,7 @@ async function encodeAnnounce06Body(w: Writer, msg: AnnounceBroadcast, version: switch (msg.status) { case "active": await encodePath(w, version, msg.suffix, msg.pathBase); + await encodeEpoch(w, version, msg.epoch); await encodeHopsBlock(w, version, msg.hops, msg.hopBase); await encodeRouteCost(w, version, msg.cost); break; @@ -236,8 +248,9 @@ async function decodeAnnounce06Body(r: Reader, typ: number, version: Version): P switch (typ) { case ANNOUNCE_START: { const path = await decodePath(r, version); + const epoch = await decodeEpoch(r, version); const hops = await decodeHopsBlock(r, version); - return { status: "active", ...path, ...hops, cost: await decodeRouteCost(r, version) }; + return { status: "active", ...path, epoch, ...hops, cost: await decodeRouteCost(r, version) }; } case ANNOUNCE_END: return { status: "endedId", id: await r.u62() }; diff --git a/js/net/src/lite/epoch.test.ts b/js/net/src/lite/epoch.test.ts new file mode 100644 index 0000000000..57fb4227c9 --- /dev/null +++ b/js/net/src/lite/epoch.test.ts @@ -0,0 +1,70 @@ +import { expect, test } from "bun:test"; +import * as Epoch from "../epoch.ts"; +import * as Path from "../path.ts"; +import { Reader, Writer } from "../stream.ts"; +import { decodeAnnounceBroadcast, encodeAnnounceBroadcast } from "./announce.ts"; +import { Fetch } from "./fetch.ts"; +import { Subscribe } from "./subscribe.ts"; +import { Track } from "./track.ts"; +import { Version } from "./version.ts"; + +async function wire(f: (w: Writer) => Promise, version: Version): Promise { + const chunks: Uint8Array[] = []; + const writer = new Writer( + new WritableStream({ write: (chunk) => void chunks.push(new Uint8Array(chunk)) }), + version, + ); + await f(writer); + writer.close(); + await writer.closed; + const out = new Uint8Array(chunks.reduce((sum, c) => sum + c.byteLength, 0)); + let offset = 0; + for (const c of chunks) { + out.set(c, offset); + offset += c.byteLength; + } + return new Reader(undefined, out, version); +} + +const room = Path.from("room"); + +// Each message carries the epoch on draft-07 and drops it on draft-06, which has no room. +for (const [version, expected] of [ + [Version.DRAFT_07, true], + [Version.DRAFT_06, false], +] as const) { + test(`ANNOUNCE_START, TRACK, SUBSCRIBE and FETCH ${expected ? "carry" : "drop"} the epoch on ${version.toString(16)}`, async () => { + const epoch = Epoch.mint(); + const want = expected ? epoch : undefined; + + const announce = await decodeAnnounceBroadcast( + await wire( + (w) => encodeAnnounceBroadcast(w, { status: "active", suffix: room, epoch, hops: [] }, version), + version, + ), + version, + ); + expect(announce).toMatchObject({ status: "active", epoch: want }); + + const track = await Track.decode( + await wire((w) => new Track(room, "video", epoch).encode(w, version), version), + version, + ); + expect(track.epoch).toBe(want); + + const subscribe = new Subscribe({ id: 0n, broadcast: room, epoch, track: "video", priority: 0 }); + const decoded = await Subscribe.decode(await wire((w) => subscribe.encode(w, version), version), version); + expect(decoded.epoch).toBe(want); + + const fetch = new Fetch({ broadcast: room, epoch, track: "video", priority: 0, group: 0 }); + expect((await Fetch.decode(await wire((w) => fetch.encode(w, version), version), version)).epoch).toBe(want); + }); +} + +test("an absent epoch round-trips as none on draft-07", async () => { + const decoded = await Track.decode( + await wire((w) => new Track(room, "video").encode(w, Version.DRAFT_07), Version.DRAFT_07), + Version.DRAFT_07, + ); + expect(decoded.epoch).toBeUndefined(); +}); diff --git a/js/net/src/lite/epoch.ts b/js/net/src/lite/epoch.ts new file mode 100644 index 0000000000..a5c506402a --- /dev/null +++ b/js/net/src/lite/epoch.ts @@ -0,0 +1,20 @@ +import * as Epoch from "../epoch.ts"; +import type { Reader, Writer } from "../stream.ts"; +import { hasEpoch, type Version } from "./version.ts"; + +/** Encode an optional epoch as its 16 bytes, or empty for none. Older versions carry + * nothing, so the peer sees a route or request of unknown identity. Mirrors rs/moq-net. */ +export async function encodeEpoch(w: Writer, version: Version, epoch: Epoch.Valid | undefined): Promise { + if (!hasEpoch(version)) return; + const bytes = epoch ? Epoch.toBytes(epoch) : new Uint8Array(); + await w.u53(bytes.byteLength); + await w.write(bytes); +} + +/** Decode an optional epoch: empty is none, anything but a UUIDv7 is refused. */ +export async function decodeEpoch(r: Reader, version: Version): Promise { + if (!hasEpoch(version)) return undefined; + const size = await r.u53(); + if (size === 0) return undefined; + return Epoch.fromBytes(await r.read(size)); +} diff --git a/js/net/src/lite/fetch.ts b/js/net/src/lite/fetch.ts index 35fe2e17de..f182bad674 100644 --- a/js/net/src/lite/fetch.ts +++ b/js/net/src/lite/fetch.ts @@ -1,5 +1,7 @@ +import type * as Epoch from "../epoch.ts"; import * as Path from "../path.ts"; import type { Reader, Writer } from "../stream.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; import { hasFrameBounds, Version } from "./version.ts"; @@ -15,6 +17,8 @@ function guardFetch(version: Version) { export class Fetch { broadcast: Path.Valid; + /** The publisher instance the subscriber expects. Lite-07+. */ + epoch?: Epoch.Valid; track: string; priority: number; group: number; @@ -30,6 +34,7 @@ export class Fetch { constructor({ broadcast, + epoch, track, priority, group, @@ -37,6 +42,7 @@ export class Fetch { endFrame, }: { broadcast: Path.Valid; + epoch?: Epoch.Valid; track: string; priority: number; group: number; @@ -44,6 +50,7 @@ export class Fetch { endFrame?: number; }) { this.broadcast = broadcast; + this.epoch = epoch; this.track = track; this.priority = priority; this.group = group; @@ -53,6 +60,7 @@ export class Fetch { async #encode(w: Writer, version: Version) { await w.string(Path.encode(this.broadcast)); + await encodeEpoch(w, version, this.epoch); await w.string(this.track); await w.u8(this.priority); await w.u53(this.group); @@ -68,12 +76,13 @@ export class Fetch { static async #decode(r: Reader, version: Version): Promise { const broadcast = Path.decode(await r.string()); + const epoch = await decodeEpoch(r, version); const track = await r.string(); const priority = await r.u8(); const group = await r.u53(); if (!hasFrameBounds(version)) { - return new Fetch({ broadcast, track, priority, group }); + return new Fetch({ broadcast, epoch, track, priority, group }); } const startFrame = await r.u53(); @@ -84,6 +93,7 @@ export class Fetch { return new Fetch({ broadcast, + epoch, track, priority, group, diff --git a/js/net/src/lite/publisher.ts b/js/net/src/lite/publisher.ts index bde4aa1381..09a67a90a4 100644 --- a/js/net/src/lite/publisher.ts +++ b/js/net/src/lite/publisher.ts @@ -434,7 +434,7 @@ export class Publisher { if (hasAnnounceId(this.version)) announceIds.set(suffix, nextAnnounceId++); await encodeAnnounceBroadcast( stream.writer, - { status: "active", suffix, hops: wireHops(route), cost: route.cost }, + { status: "active", suffix, epoch: route.epoch, hops: wireHops(route), cost: route.cost }, this.version, ); }; @@ -541,11 +541,13 @@ export class Publisher { for (const [suffix, snap] of active) { const cur = updated.get(suffix); - if (!cur || cur.identity !== snap.identity) await retract(suffix); + // A new epoch is another broadcast, even from the same entry. + if (!cur || cur.identity !== snap.identity || cur.route.epoch !== snap.route.epoch) + await retract(suffix); } for (const [suffix, snap] of updated) { const prev = active.get(suffix); - if (!prev || prev.identity !== snap.identity) { + if (!prev || prev.identity !== snap.identity || prev.route.epoch !== snap.route.epoch) { await announce(suffix, snap.route); } else if (!routesEqual(onWire(prev.route), onWire(snap.route))) { await restart(suffix, snap.route); @@ -576,7 +578,8 @@ export class Publisher { try { front = this.#publish && - (wireOf(this.#publish).local(msg.broadcast) ?? (await wireOf(this.#publish).demand(msg.broadcast))); + (wireOf(this.#publish).local(msg.broadcast, msg.epoch) ?? + (await wireOf(this.#publish).demand(msg.broadcast, msg.epoch))); } catch (err: unknown) { stream.writer.reset(error(err)); return; @@ -700,7 +703,8 @@ export class Publisher { try { front = this.#publish && - (wireOf(this.#publish).local(msg.broadcast) ?? (await wireOf(this.#publish).demand(msg.broadcast))); + (wireOf(this.#publish).local(msg.broadcast, msg.epoch) ?? + (await wireOf(this.#publish).demand(msg.broadcast, msg.epoch))); } catch (err: unknown) { stream.writer.reset(error(err)); return; @@ -970,7 +974,8 @@ export class Publisher { try { const front = this.#publish && - (wireOf(this.#publish).local(msg.broadcast) ?? (await wireOf(this.#publish).demand(msg.broadcast))); + (wireOf(this.#publish).local(msg.broadcast, msg.epoch) ?? + (await wireOf(this.#publish).demand(msg.broadcast, msg.epoch))); if (!front) throw new NotFound(`broadcast ${msg.broadcast}`); const info = await this.#resolveTrackInfo(front, msg.track); diff --git a/js/net/src/lite/subscribe.ts b/js/net/src/lite/subscribe.ts index fecb237f40..084f68f40e 100644 --- a/js/net/src/lite/subscribe.ts +++ b/js/net/src/lite/subscribe.ts @@ -1,6 +1,8 @@ +import type * as Epoch from "../epoch.ts"; import * as Path from "../path.ts"; import type { Reader, Writer } from "../stream.ts"; import type { Location } from "../track.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; import { hasFrameBounds, hasGroupOrder, hasLargest, hasStreamCount, resolvesStart, Version } from "./version.ts"; @@ -230,6 +232,8 @@ export class SubscribeUpdate { export class Subscribe { id: bigint; broadcast: Path.Valid; + /** The publisher instance the subscriber expects. Lite-07+. */ + epoch?: Epoch.Valid; track: string; priority: number; /** Subscriber max latency in milliseconds; zero skips once a newer group is available. */ @@ -254,6 +258,7 @@ export class Subscribe { constructor(props: { id: bigint; broadcast: Path.Valid; + epoch?: Epoch.Valid; track: string; priority: number; maxAge?: number; @@ -264,6 +269,7 @@ export class Subscribe { }) { this.id = props.id; this.broadcast = props.broadcast; + this.epoch = props.epoch; this.track = props.track; this.priority = props.priority; this.maxAge = props.maxAge ?? 0; @@ -276,6 +282,7 @@ export class Subscribe { async #encode(w: Writer, version: Version) { await w.u62(this.id); await w.string(Path.encode(this.broadcast)); + await encodeEpoch(w, version, this.epoch); await w.string(this.track); await w.u8(this.priority); @@ -296,6 +303,7 @@ export class Subscribe { static async #decode(r: Reader, version: Version): Promise { const id = await r.u62(); const broadcast = Path.decode(await r.string()); + const epoch = await decodeEpoch(r, version); const track = await r.string(); const priority = await r.u8(); @@ -313,6 +321,7 @@ export class Subscribe { return new Subscribe({ id, broadcast, + epoch, track, priority, maxAge, diff --git a/js/net/src/lite/subscriber.test.ts b/js/net/src/lite/subscriber.test.ts index 7b668639bb..274621bde6 100644 --- a/js/net/src/lite/subscriber.test.ts +++ b/js/net/src/lite/subscriber.test.ts @@ -1,6 +1,7 @@ import { expect, spyOn, test } from "bun:test"; import { Signal } from "@moq/signals"; import type { Probe as ProbeStats } from "../connection/stats.ts"; +import * as Epoch from "../epoch.ts"; import { error, fromTransport, reason, StreamCode, StreamError } from "../error.ts"; import { HopSchema, isAnonymous, MAX_HOPS, Route, UNKNOWN_HOP } from "../hop.ts"; import * as Path from "../path.ts"; @@ -560,6 +561,35 @@ test("a restart replaces an announce that was skipped as a reflected loop", asyn subscriber.close(); }); +test("a restart keeps the epoch of an announce skipped as a reflected loop", async () => { + const SELF = 1n; + const { subscriber, send, settle } = announceHarness(Version.DRAFT_07, SELF); + const announced = subscriber.announced(); + await settle(); + + await send((w) => new AnnounceOk(PEER, 0).encode(w, Version.DRAFT_07)); + + // Skipped: the chain reflects back through us. The epoch still names the instance. + const epoch = Epoch.mint(); + await send((w) => + encodeAnnounceBroadcast( + w, + { status: "active", suffix: Path.from("room"), epoch, hops: [HopSchema.parse(SELF)] }, + Version.DRAFT_07, + ), + ); + await settle(); + + // A restart never carries the epoch, so the attached route takes the one the start named. + await send((w) => encodeAnnounceBroadcast(w, { status: "restart", id: 0n, hops: [PUBLISHER_A] }, Version.DRAFT_07)); + const started = await nextRoute(announced); + expect(started).toMatchObject({ prefix: Path.from("room"), kind: "start" }); + expect(started?.route.epoch).toBe(epoch); + + announced.close(); + subscriber.close(); +}); + test("retiring an id whose announce was skipped ends nothing", async () => { const SELF = 1n; const { subscriber, send, settle } = announceHarness(Version.DRAFT_06, SELF); diff --git a/js/net/src/lite/subscriber.ts b/js/net/src/lite/subscriber.ts index 6f59818192..1ae539fc6d 100644 --- a/js/net/src/lite/subscriber.ts +++ b/js/net/src/lite/subscriber.ts @@ -3,6 +3,7 @@ import * as announce from "../announced.ts"; import * as broadcast from "../broadcast.ts"; import type { Probe as ProbeStats } from "../connection/stats.ts"; import { BroadcastCache } from "../consume.ts"; +import type * as Epoch from "../epoch.ts"; import { controlTimeout, error, ProtocolViolation, reason, StreamCode, StreamError, sessionCause } from "../error.ts"; import * as netGroup from "../group.ts"; import { Cost, type Hop, MAX_HOPS, type Route, randomHop, routesEqual, stampHops, UNKNOWN_HOP } from "../hop.ts"; @@ -138,6 +139,10 @@ export class Subscriber { // Dedup consumed broadcasts per path: repeat consume() calls share one subscription. #consumes = new BroadcastCache(); + // The epoch each live advertisement named, by path. A consumed broadcast captures it + // once and asks for it on every request, so a later epoch never feeds an older handle. + #epochs = new Map(); + // A random Hop ID of this connection's own, written as the first hop of any chain that // names no publisher, so a publisher that reconnects reads as a new one. #stamp: Hop; @@ -331,6 +336,8 @@ export class Subscriber { // Present on active/restart; ended messages never carry hops worth checking. let hops: Hop[] | undefined; let cost: Cost | undefined; + // Present on active; a restart never changes it. + let epoch: Epoch.Valid | undefined; switch (announce.status) { case "active": { @@ -341,6 +348,7 @@ export class Subscriber { active = true; hops = resolved.hops; cost = announce.cost; + epoch = announce.epoch; break; } case "ended": @@ -388,6 +396,7 @@ export class Subscriber { const retract = () => { const previous = advertised.get(path); advertised.delete(path); + this.#epochs.delete(path); if (!previous?.live) return; this.#consumes.evict(path); console.debug(`announced: broadcast=${path} active=false`); @@ -399,6 +408,10 @@ export class Subscriber { }); }; + // A restart keeps the epoch its announcement named, even through a placeholder + // below: a later restart that is not reflected still names that instance. + epoch ??= advertised.get(path)?.route.epoch; + // In Lite05+ the sender's origin arrives via AnnounceOk, not in each hop // list, so fold it back in before checking. if (hops !== undefined) { @@ -411,7 +424,7 @@ export class Subscriber { retract(); advertised.set(path, { live: false, - route: { hops: full, cost: Cost.zero }, + route: { epoch, hops: full, cost: Cost.zero }, captures: undefined, }); continue; @@ -438,12 +451,12 @@ export class Subscriber { console.debug(`announced: broadcast=${path} dropped (hop chain at MAX_HOPS)`); advertised.set(path, { live: false, - route: { hops: [], cost: Cost.zero }, + route: { epoch, hops: [], cost: Cost.zero }, captures: undefined, }); continue; } - const route: Route = { hops: fullHops, cost: cost ?? Cost.zero }; + const route: Route = { epoch, hops: fullHops, cost: cost ?? Cost.zero }; const captures = scopeCaptures(scope, path); if (!visible(path)) { advertised.set(path, { live: false, route, captures }); @@ -466,6 +479,7 @@ export class Subscriber { } advertised.set(path, { live: true, route, captures }); + if (epoch) this.#epochs.set(path, epoch); console.debug(`announced: broadcast=${path} active=true`); announced.append({ prefix: path, captures, kind: "start", route }); @@ -509,20 +523,21 @@ export class Subscriber { // A consumed broadcast resolves info() and fetchGroup() over the wire by reaching // back into this Subscriber (see ConsumeBroadcast below), rather than the wire // installing callbacks on the broadcast. - const consumer = new ConsumeBroadcast(this, path); + const epoch = this.#epochs.get(path); + const consumer = new ConsumeBroadcast(this, path, epoch); void (async () => { for (;;) { const request = await wireOf(consumer).requested(); if (!request) break; - void this.#runSubscribe(path, request); + void this.#runSubscribe(path, epoch, request); } })(); return consumer; } - async #runSubscribe(broadcast: Path.Valid, request: track.Request) { + async #runSubscribe(broadcast: Path.Valid, epoch: Epoch.Valid | undefined, request: track.Request) { const id = this.#subscribeNext++; const subscription = request.subscription; const initialBounds = groupBounds(subscription.groups); @@ -541,6 +556,7 @@ export class Subscriber { const msg = new Subscribe({ id, broadcast, + epoch, track: request.name, priority: subscription.priority ?? 0, maxAge: subscription.maxAge, @@ -660,7 +676,7 @@ export class Subscriber { if (supportsTrackStream(this.version)) { // Fetch the immutable properties once via the TRACK stream. - const info = await this.#trackInfo(msg.broadcast, msg.track); + const info = await this.#trackInfo(msg.broadcast, msg.epoch, msg.track); producer = request.accept(this.#toModelInfo(info)); timescale.set(info.timescale); } else { @@ -703,10 +719,10 @@ export class Subscriber { } // Opens a TRACK stream, reads the single TRACK_INFO, and FINs. Lite-05+ only. - async #trackInfo(broadcast: Path.Valid, track: string): Promise { + async #trackInfo(broadcast: Path.Valid, epoch: Epoch.Valid | undefined, track: string): Promise { return this.#exchange({ version: this.version }, async (stream) => { await stream.writer.u53(StreamId.Track); - await new TrackMessage(broadcast, track).encode(stream.writer, this.version); + await new TrackMessage(broadcast, track, epoch).encode(stream.writer, this.version); const info = await TrackInfo.decode(stream.reader, this.version); // The publisher FINs after TRACK_INFO; FIN our side too. stream.close(); @@ -748,11 +764,11 @@ export class Subscriber { // Resolve a track's immutable model info via a TRACK stream (lite-05+), for the // ConsumeBroadcast backing track.Consumer.query(). On older drafts there's no TRACK // stream, so this rejects rather than fabricating defaults. - async resolveTrackInfo(broadcast: Path.Valid, track: string): Promise { + async resolveTrackInfo(broadcast: Path.Valid, track: string, epoch?: Epoch.Valid): Promise { if (!supportsTrackStream(this.version)) { throw new Error("track info requires moq-lite-05 or newer"); } - return this.#toModelInfo(await this.#trackInfo(broadcast, track)); + return this.#toModelInfo(await this.#trackInfo(broadcast, epoch, track)); } // Open a FETCH stream for one group and stream its bare frames into a group, for the @@ -762,6 +778,7 @@ export class Subscriber { track: string, sequence: number, options: track.FetchGroupOptions = {}, + epoch?: Epoch.Valid, ): Promise { options.signal?.throwIfAborted(); @@ -772,7 +789,7 @@ export class Subscriber { // demand from the start, and a fast FIN cannot discard frames before these callers // receive their handles. An abort closes only this caller's mirror, so the stream is // cancelled once the last one leaves. - const key = JSON.stringify([broadcast, track, sequence]); + const key = JSON.stringify([broadcast, epoch, track, sequence]); let entry = this.#fetches.get(key); let consumer: netGroup.Consumer; if (entry && !entry.group.isClosed) { @@ -780,7 +797,10 @@ export class Subscriber { } else { const group = new netGroup.Producer(sequence); consumer = group.mirror(); - entry = { group, accepted: this.#runFetch(broadcast, track, sequence, options.priority ?? 0, group) }; + entry = { + group, + accepted: this.#runFetch(broadcast, epoch, track, sequence, options.priority ?? 0, group), + }; this.#fetches.set(key, entry); void group.closed.then(() => { if (this.#fetches.get(key)?.group === group) this.#fetches.delete(key); @@ -801,6 +821,7 @@ export class Subscriber { // has abandoned is cancelled the same way. async #runFetch( broadcast: Path.Valid, + epoch: Epoch.Valid | undefined, track: string, sequence: number, priority: number, @@ -814,7 +835,7 @@ export class Subscriber { // Lite has no FETCH_OK, so a publisher that never answers would hold the setup forever. // Subscriber.close() closing the group releases every caller at any stage, and resets // the streams the setup opened. - const setup = this.#fetchSetup(broadcast, track, sequence, priority, group); + const setup = this.#fetchSetup(broadcast, epoch, track, sequence, priority, group); let accepted: { stream: Stream; info: TrackInfo }; try { accepted = await untilAbandoned(group, setup); @@ -838,15 +859,19 @@ export class Subscriber { // Closing the group during that wait resets the stream. async #fetchSetup( broadcast: Path.Valid, + epoch: Epoch.Valid | undefined, track: string, sequence: number, priority: number, group: netGroup.Producer, ): Promise<{ stream: Stream; info: TrackInfo }> { - const info = await untilClosed(group, this.#trackInfo(broadcast, track)); + const info = await untilClosed(group, this.#trackInfo(broadcast, epoch, track)); return this.#exchange({ sendOrder: sendOrder({ priority }), version: this.version }, async (stream) => { await stream.writer.u53(StreamId.Fetch); - await new FetchMessage({ broadcast, track, priority, group: sequence }).encode(stream.writer, this.version); + await new FetchMessage({ broadcast, epoch, track, priority, group: sequence }).encode( + stream.writer, + this.version, + ); // A byte or an empty-group FIN accepts the fetch; a reset rejects it. // done() buffers that byte so the response pump can decode it normally. await untilClosed(group, stream.reader.done()); @@ -1307,20 +1332,22 @@ async function untilAbandoned(group: netGroup.Producer, step: Promise): Pr class ConsumeBroadcast extends broadcast.Consumer { #subscriber: Subscriber; #path: Path.Valid; + #epoch?: Epoch.Valid; - constructor(subscriber: Subscriber, path: Path.Valid, state?: never) { + constructor(subscriber: Subscriber, path: Path.Valid, epoch: Epoch.Valid | undefined, state?: never) { super(state); overrideBroadcastWire(this, { - resolveTrackInfo: (name) => subscriber.resolveTrackInfo(path, name), - fetchGroup: (name, sequence, options) => subscriber.fetchGroup(path, name, sequence, options), + resolveTrackInfo: (name) => subscriber.resolveTrackInfo(path, name, epoch), + fetchGroup: (name, sequence, options) => subscriber.fetchGroup(path, name, sequence, options, epoch), }); this.#subscriber = subscriber; this.#path = path; + this.#epoch = epoch; } // Preserve the subclass (and its wire-backed info/fetchGroup) when the consume cache shares // this broadcast across callers. override clone(): ConsumeBroadcast { - return new ConsumeBroadcast(this.#subscriber, this.#path, this.shareState()); + return new ConsumeBroadcast(this.#subscriber, this.#path, this.#epoch, this.shareState()); } } diff --git a/js/net/src/lite/track.ts b/js/net/src/lite/track.ts index d07db186b2..146c8cc3a6 100644 --- a/js/net/src/lite/track.ts +++ b/js/net/src/lite/track.ts @@ -1,6 +1,8 @@ +import type * as Epoch from "../epoch.ts"; import * as Path from "../path.ts"; import type { Reader, Writer } from "../stream.ts"; import { Timescale } from "../time.ts"; +import { decodeEpoch, encodeEpoch } from "./epoch.ts"; import * as Message from "./message.ts"; import { hasGroupOrder, Version } from "./version.ts"; @@ -24,31 +26,36 @@ function guardTrack(version: Version) { export class Track { broadcast: Path.Valid; track: string; + /** The publisher instance the subscriber expects. Lite-07+. */ + epoch?: Epoch.Valid; - constructor(broadcast: Path.Valid, track: string) { + constructor(broadcast: Path.Valid, track: string, epoch?: Epoch.Valid) { this.broadcast = broadcast; this.track = track; + this.epoch = epoch; } - async #encode(w: Writer) { + async #encode(w: Writer, version: Version) { await w.string(Path.encode(this.broadcast)); + await encodeEpoch(w, version, this.epoch); await w.string(this.track); } - static async #decode(r: Reader): Promise { + static async #decode(r: Reader, version: Version): Promise { const broadcast = Path.decode(await r.string()); + const epoch = await decodeEpoch(r, version); const track = await r.string(); - return new Track(broadcast, track); + return new Track(broadcast, track, epoch); } async encode(w: Writer, version: Version): Promise { guardTrack(version); - return Message.encode(w, (w) => this.#encode(w)); + return Message.encode(w, (w) => this.#encode(w, version)); } static async decode(r: Reader, version: Version): Promise { guardTrack(version); - return Message.decode(r, (r) => Track.#decode(r)); + return Message.decode(r, (r) => Track.#decode(r, version)); } } diff --git a/js/net/src/lite/version.ts b/js/net/src/lite/version.ts index a69e8fea0d..286853bfc3 100644 --- a/js/net/src/lite/version.ts +++ b/js/net/src/lite/version.ts @@ -10,7 +10,7 @@ export const Version = { /// Also adds frame-precise subscribe/fetch bounds and a GROUP frame offset. DRAFT_06: 0xff0dad06, /// Work-in-progress lite-07, only negotiated when explicitly offered. - /// Adds the ANNOUNCE_REQUEST hidden opt-in. + /// Adds the ANNOUNCE_REQUEST hidden opt-in and the publisher epoch. DRAFT_07: 0xff0dad07, } as const; @@ -269,6 +269,23 @@ export function waitsForSubscriberFin(version: Version): boolean { } } +/** Whether ANNOUNCE_START, TRACK, SUBSCRIBE, and FETCH carry the publisher epoch. Added in lite-07. + * Older versions carry nothing, so a received route has no epoch and is never resumed elsewhere. */ +export function hasEpoch(version: Version): boolean { + // Explicitly list older versions so future versions keep the lite-07+ behavior. + switch (version) { + case Version.DRAFT_01: + case Version.DRAFT_02: + case Version.DRAFT_03: + case Version.DRAFT_04: + case Version.DRAFT_05: + case Version.DRAFT_06: + return false; + default: + return true; + } +} + /** Whether ANNOUNCE_START and ANNOUNCE_UPDATE may copy a path head or hop-chain tail from a live announcement. Added in lite-07. */ export function hasAnnounceCompression(version: Version): boolean { // Explicitly list older versions so future versions keep the lite-07+ behavior. diff --git a/js/net/src/origin.test.ts b/js/net/src/origin.test.ts index f42839d6c2..7dc871fc6b 100644 --- a/js/net/src/origin.test.ts +++ b/js/net/src/origin.test.ts @@ -1,6 +1,7 @@ import { expect, spyOn, test } from "bun:test"; import { getter, Once, race, Signal } from "@moq/signals"; import { type Consumer as BroadcastConsumer, Producer as BroadcastProducer } from "./broadcast.ts"; +import * as Epoch from "./epoch.ts"; import { StreamCode, StreamError } from "./error.ts"; import { HopSchema, Route, stampHops } from "./hop.ts"; import { spreadHash } from "./internal.ts"; @@ -27,6 +28,8 @@ function publish(origin: Producer, path: Path.Valid) { /** A peer's hop, for a route that is not local. */ const PEER = HopSchema.parse(10n); +// One publisher instance, so a local broadcast and a remote route compete on cost alone. +const EPOCH = Epoch.parse("01900000-0000-7000-8000-000000000001"); /** Land a received prefix, served from `consume`, the way a session does. */ function serve(origin: Producer, prefix: Path.Valid, consume: () => BroadcastConsumer, route: Route = Route.default) { @@ -83,9 +86,9 @@ test("a borrowed table exposes dynamic serving and a live scoped broadcast map", expect(live.peek().has(localPath)).toBe(false); expect(live.peek().has(Path.from("other"))).toBe(false); - local.announce({ cost: 4n }); + local.announce({ epoch: EPOCH, cost: 4n }); await settle(); - expect(live.peek().get(localPath)).toEqual(Route.normalize({ cost: 4n })); + expect(live.peek().get(localPath)).toEqual(Route.normalize({ epoch: EPOCH, cost: 4n })); expect(changes.some((value) => value.get(localPath)?.cost.warm === 4n)).toBe(true); const prefix = Path.from("room"); @@ -149,23 +152,23 @@ test("an announced local path competes with a received route on cost", async () const origin = new Producer(); const consumer = origin.consume(); const path = Path.from("room/alice"); - const remote = wireOf(origin).receive(path, { hops: [PEER], cost: 5n }); + const remote = wireOf(origin).receive(path, { epoch: EPOCH, hops: [PEER], cost: 5n }); const live = consumer.broadcasts(); - expect(live.peek().get(path)).toEqual(Route.normalize({ hops: [PEER], cost: 5n })); + expect(live.peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 5n })); // Unannounced, it competes for nothing. const local = origin.createBroadcast(path); - expect(live.peek().get(path)).toEqual(Route.normalize({ hops: [PEER], cost: 5n })); + expect(live.peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 5n })); // Cheaper wins; a tie falls to the local broadcast. - local.announce({ cost: 9n }); - expect(live.peek().get(path)).toEqual(Route.normalize({ hops: [PEER], cost: 5n })); - local.announce({ cost: 5n }); - expect(live.peek().get(path)).toEqual(Route.normalize({ cost: 5n })); + local.announce({ epoch: EPOCH, cost: 9n }); + expect(live.peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 5n })); + local.announce({ epoch: EPOCH, cost: 5n }); + expect(live.peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, cost: 5n })); local.close(); await settle(); - expect(live.peek().get(path)).toEqual(Route.normalize({ hops: [PEER], cost: 5n })); + expect(live.peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 5n })); remote.close(); await settle(); @@ -182,7 +185,7 @@ test("a scoped route remains visible when an exact local path is outside the sco const scope = Path.Pattern.parse("room/*"); const live = origin.broadcasts(scope); - expect(origin.broadcasts().peek().get(path)).toEqual(Route.default); + expect(origin.broadcasts().peek().get(path)).toMatchObject(Route.default); expect(live.peek().get(path)).toEqual(Route.normalize({ cost: 5n })); local.close(); @@ -1195,9 +1198,9 @@ test("a request prefers a cheaper received route over an announced local broadca const consumer = origin.consume(); const path = Path.from("room"); const upstream = new BroadcastProducer(); - const dispose = serve(origin, path, provider(upstream), Route.normalize({ hops: [PEER], cost: 1n })); + const dispose = serve(origin, path, provider(upstream), Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 1n })); const local = origin.createBroadcast(path); - local.announce({ cost: 3n }); + local.announce({ epoch: EPOCH, cost: 3n }); // The cheaper route is served on demand, so it resolves only once its handler answers. const request = consumer.request(path); @@ -1209,7 +1212,7 @@ test("a request prefers a cheaper received route over an announced local broadca expect(await nextRoute(announced)).toMatchObject({ prefix: path, route: { hops: [PEER] } }); // Re-priced below it, the local broadcast wins at once, and the remote front it replaced closes. - local.announce({ cost: 0n }); + local.announce({ epoch: EPOCH, cost: 0n }); expect(await nextRoute(announced)).toMatchObject({ prefix: path, kind: "end" }); expect(await nextRoute(announced)).toMatchObject({ prefix: path, kind: "start", route: Route.default }); expect(request.active.peek()).not.toBe(remote); @@ -1233,12 +1236,12 @@ test("the cheapest received route competes with the local broadcast, not the new served.push(name); return upstream.consume(); }; - const cheap = serve(origin, path, via("cheap"), Route.normalize({ hops: [PEER], cost: 1n })); - const pricey = serve(origin, path, via("pricey"), Route.normalize({ hops: [PEER], cost: 10n })); + const cheap = serve(origin, path, via("cheap"), Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 1n })); + const pricey = serve(origin, path, via("pricey"), Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 10n })); const local = origin.createBroadcast(path); - local.announce({ cost: 5n }); + local.announce({ epoch: EPOCH, cost: 5n }); - expect(consumer.broadcasts().peek().get(path)).toEqual(Route.normalize({ hops: [PEER], cost: 1n })); + expect(consumer.broadcasts().peek().get(path)).toEqual(Route.normalize({ epoch: EPOCH, hops: [PEER], cost: 1n })); const request = consumer.request(path); await settle(); expect(request.active.peek()).toBeDefined(); @@ -1344,10 +1347,12 @@ test("a peer is offered and served the cheapest originated route", async () => { const origin = new Producer(); const consumer = origin.consume(); const prefix = Path.from("live"); - const cheap = origin.dynamic(prefix, { cost: 1n }); - const pricey = origin.dynamic(prefix, { cost: 10n }); + const cheap = origin.dynamic(prefix, { epoch: EPOCH, cost: 1n }); + const pricey = origin.dynamic(prefix, { epoch: EPOCH, cost: 10n }); - expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual(Route.normalize({ cost: 1n })); + expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual( + Route.normalize({ epoch: EPOCH, cost: 1n }), + ); const pending = wireOf(consumer).demand(Path.from("live/cam")); const { value: req } = await cheap.requested().next(); const upstream = new BroadcastProducer(); @@ -1356,10 +1361,14 @@ test("a peer is offered and served the cheapest originated route", async () => { // An exact-path local broadcast competes with them on cost too. const local = origin.createBroadcast(prefix); - local.announce({ cost: 5n }); - expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual(Route.normalize({ cost: 1n })); - local.announce({ cost: 0n }); - expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual(Route.normalize({ cost: 0n })); + local.announce({ epoch: EPOCH, cost: 5n }); + expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual( + Route.normalize({ epoch: EPOCH, cost: 1n }), + ); + local.announce({ epoch: EPOCH, cost: 0n }); + expect(wireOf(consumer).advertised.peek()?.get(prefix)?.[0]?.route).toEqual( + Route.normalize({ epoch: EPOCH, cost: 0n }), + ); local.close(); upstream.close(); @@ -1879,9 +1888,9 @@ test("a scoped reader sees a local broadcast that only loses to a route outside const origin = new Producer(); const chat = origin .scope(Path.empty(), new Path.Patterns([Path.Pattern.parse("room/*/chat")])) - .dynamic(Path.from("room/alice"), { cost: 0n }); + .dynamic(Path.from("room/alice"), { epoch: EPOCH, cost: 0n }); const local = origin.createBroadcast(Path.from("room/alice")); - local.announce({ cost: 5n }); + local.announce({ epoch: EPOCH, cost: 5n }); const reader = origin.scope(Path.empty(), new Path.Patterns([Path.Pattern.parse("room/*")])); expect(reader.broadcasts().peek().get(Path.from("room/alice"))?.cost.warm).toBe(5n); @@ -1926,3 +1935,76 @@ test("broadcast handles carry the path they were created or requested at", async broadcast.close(); origin.close(); }); + +test("the newest epoch wins the path over a cheaper one and arrives as a new broadcast", async () => { + const origin = new Producer(); + const consumer = origin.consume(); + const path = Path.from("room/alice"); + const announced = consumer.announced(); + + const old = origin.createBroadcast(path); + old.announce({ epoch: Epoch.mint(), cost: 1n }); + expect(await nextRoute(announced)).toMatchObject({ prefix: path, kind: "start" }); + + // Minted after the local broadcast's, so it is the newer publisher despite the cost. + const newer = wireOf(origin).receive(path, { epoch: Epoch.mint(), hops: [PEER], cost: 9n }); + expect(await nextRoute(announced)).toMatchObject({ prefix: path, kind: "end" }); + expect(await nextRoute(announced)).toMatchObject({ prefix: path, kind: "start", route: { hops: [PEER] } }); + + newer.close(); + old.close(); + announced.close(); + origin.close(); +}); + +test("demand naming an epoch is refused when the route changes instance while it waits", async () => { + const origin = new Producer(); + const consumer = origin.consume(); + const handle = origin.dynamic(Path.from("live"), { epoch: EPOCH }); + const it = handle.requested(); + const pending = wireOf(consumer).demand(Path.from("live/cam"), EPOCH); + const { value: req } = await it.next(); + + // Another instance now serves the route; its answer is not the one the peer named. + handle.update({ ...handle.route, epoch: Epoch.mint() }); + req?.accept(new BroadcastProducer().consume()); + await expect(pending).rejects.toThrow("unroutable"); + + handle.close(); + origin.close(); +}); + +test("re-pricing from the current route keeps the epoch", () => { + const origin = new Producer(); + const handle = origin.dynamic(Path.from("live"), { epoch: EPOCH, cost: 5n }); + handle.update({ ...handle.route, cost: 1n }); + expect(handle.route).toEqual(Route.normalize({ epoch: EPOCH, cost: 1n })); + + const broadcast = origin.createBroadcast(Path.from("room")); + expect(broadcast.route).toBeUndefined(); + broadcast.announce({ epoch: EPOCH, cost: 5n }); + broadcast.announce({ ...broadcast.route, cost: 1n }); + expect(broadcast.route).toEqual(Route.normalize({ epoch: EPOCH, cost: 1n })); + broadcast.unannounce(); + expect(broadcast.route).toBeUndefined(); + + broadcast.close(); + handle.close(); + origin.close(); +}); + +test("a peer naming another epoch is refused rather than served a different instance", async () => { + const origin = new Producer(); + const consumer = origin.consume(); + const path = Path.from("room"); + const local = origin.createBroadcast(path); + local.announce({ epoch: EPOCH }); + + expect(wireOf(consumer).local(path, EPOCH)).toBeDefined(); + const other = Epoch.parse("01900000-0000-7000-8000-000000000002"); + expect(wireOf(consumer).local(path, other)).toBeUndefined(); + await expect(wireOf(consumer).demand(path, other)).rejects.toThrow("unroutable"); + + local.close(); + origin.close(); +}); diff --git a/js/net/src/origin.ts b/js/net/src/origin.ts index c23847301a..16e4b0972c 100644 --- a/js/net/src/origin.ts +++ b/js/net/src/origin.ts @@ -13,6 +13,7 @@ import { Derived, type Dispose, type GetPromise, type Getter, getter, Once, Signal } from "@moq/signals"; import * as announce from "./announced.ts"; import * as broadcast from "./broadcast.ts"; +import type * as Epoch from "./epoch.ts"; import { StreamCode, StreamError } from "./error.ts"; import { isAnonymous, Route, routesEqual } from "./hop.ts"; import { hiddenBelow, hooks, scopeCaptures, scopeHead, scopeOverlaps, spreadHash } from "./internal.ts"; @@ -221,8 +222,15 @@ function compareCandidates(prefix: Path.Valid, a: Candidate, b: Candidate): numb return ha < hb ? -1 : ha > hb ? 1 : 0; } -/** Orders two routes by preference: identified before anonymous, then lower warm cost, then lower cold cost. */ +/** Orders two routes by preference: the newest epoch with none last, then identified before + * anonymous, then lower warm cost, then lower cold cost. */ function compareRoutes(a: Route, b: Route): number { + if (a.epoch !== b.epoch) { + // An older epoch is a publisher that was replaced. + if (a.epoch === undefined) return 1; + if (b.epoch === undefined) return -1; + return a.epoch > b.epoch ? -1 : 1; + } const anonymous = Number(isAnonymous(a)) - Number(isAnonymous(b)); if (anonymous !== 0) return anonymous; if (a.cost.warm !== b.cost.warm) return a.cost.warm < b.cost.warm ? -1 : 1; @@ -670,7 +678,10 @@ export interface Table { announced(scope?: Path.Pattern, options?: announce.Options): announce.Consumer; /** Advertise a prefix and serve requests under it; see {@link Producer.dynamic}. */ - dynamic(prefix: Path.Valid, route?: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint }): Dynamic; + dynamic( + prefix: Path.Valid, + route?: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint }, + ): Dynamic; } /** Options for resolving a broadcast path. */ @@ -757,8 +768,9 @@ export class Producer implements Table { * * Close the producer to drop it. Creating a path again supersedes the previous * broadcast: the origin drops its handle on the old one, which closes it unless the - * application still holds a consumer clone. An announced local broadcast competes with - * a remote route at the same path on cost, winning ties. + * application still holds a consumer clone. Announce with a {@link Route.epoch} + * (`Epoch.mint()` per run) so a restart replaces the old broadcast rather than resuming + * into it; at the same epoch, a local broadcast competes on cost and wins ties. */ createBroadcast(path: Path.Valid): broadcast.Producer { path = this.#scope.path(path); @@ -772,6 +784,10 @@ export class Producer implements Table { hooks.attachAnnouncer(producer, { announce: (route) => this.#advertiseExact(path, front, route), unannounce: () => this.#retractExact(path, front), + route: () => + this.#state.local.peek()?.get(path) === front + ? this.#state.advertisedLocal.peek()?.get(path) + : undefined, }); const previous = created.get(path); @@ -830,7 +846,7 @@ export class Producer implements Table { */ dynamic( prefix: Path.Valid, - route: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default, + route: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default, ): Dynamic { return this.#insertRoute(prefix, Route.normalize(route), true); } @@ -843,7 +859,7 @@ export class Producer implements Table { */ #receive( prefix: Path.Valid, - route: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default, + route: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint } = Route.default, ): Dynamic { return this.#insertRoute(prefix, Route.normalize(route), false); } @@ -1204,8 +1220,8 @@ export class Consumer { scope === Scope.all ? state.originated : new Derived([state.originated], (routes) => scope.projectRoutes(routes)), - local: (path) => this.#local(scope.path(path)), - demand: (path) => this.#demand(scope.path(path)), + local: (path, epoch) => this.#local(scope.path(path), epoch), + demand: (path, epoch) => this.#demand(scope.path(path), epoch), }); } @@ -1479,7 +1495,8 @@ export class Consumer { for (const [path, snap] of active) { const cur = next.get(path); - if (!cur || cur.identity !== snap.identity) + // A new epoch is another broadcast, even from the same entry. + if (!cur || cur.identity !== snap.identity || cur.route.epoch !== snap.route.epoch) producer.append({ prefix: path, captures: snap.captures, @@ -1489,7 +1506,7 @@ export class Consumer { } for (const [path, snap] of next) { const prev = active.get(path); - if (!prev || prev.identity !== snap.identity) { + if (!prev || prev.identity !== snap.identity || prev.route.epoch !== snap.route.epoch) { producer.append({ prefix: path, captures: snap.captures, @@ -1544,10 +1561,11 @@ export class Consumer { * The announced local broadcast at `path`, when it beats the originated routes there. * Resolves through what rebuildOriginated advertised: a peer never sees received routes. */ - #local(path: Path.Valid): broadcast.Consumer | undefined { + #local(path: Path.Valid, epoch?: Epoch.Valid): broadcast.Consumer | undefined { const local = this.#state.local.peek()?.get(path); - if (local && this.#state.localWins(path, this.#state.bestEntry(path, received))) return local; - return undefined; + if (!local || !this.#state.localWins(path, this.#state.bestEntry(path, received))) return undefined; + if (epoch !== undefined && this.#state.advertisedLocal.peek()?.get(path)?.epoch !== epoch) return undefined; + return local; } /** @@ -1556,11 +1574,17 @@ export class Consumer { * * @internal */ - async #demand(path: Path.Valid): Promise { + async #demand(path: Path.Valid, epoch?: Epoch.Valid): Promise { const local = this.#local(path); - if (local) return local; + if (local) { + // A peer naming one publisher instance is never handed another. + if (epoch !== undefined && this.#state.advertisedLocal.peek()?.get(path)?.epoch !== epoch) + throw unroutable(); + return local; + } const entry = this.#state.bestEntry(path, received); if (!entry?.server) return undefined; + if (epoch !== undefined && entry.route.peek().epoch !== epoch) throw unroutable(); const server = entry.server; const live = server.served.get(path); @@ -1570,6 +1594,9 @@ export class Consumer { server.demanding.set(path, (server.demanding.get(path) ?? 0) + 1); try { for (;;) { + // The route may name another instance by the time its handler answers. + if (epoch !== undefined && this.#state.bestEntry(path, received)?.route.peek().epoch !== epoch) + throw unroutable(); const served = server.served.get(path); if (served && served.closed.peek() === undefined) return served; const rejected = server.rejected.get(path); @@ -1622,8 +1649,18 @@ export class Dynamic { makeDynamic = (prefix, entry, state, retract) => new Dynamic(prefix, entry, state, retract); } - /** Re-price the route in place. The prefix is fixed at announce time. */ - update(route: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint }): void { + /** The route this handle advertises. */ + get route(): Route { + return this.#entry.route.peek(); + } + + /** + * Replace the route in place. The prefix is fixed at announce time. + * + * The route is taken as given, epoch included: another epoch (or none) names another + * publisher instance, so re-price from the current one, `update({ ...dynamic.route, cost })`. + */ + update(route: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint }): void { if (this.#closed) throw new Error("dynamic is closed"); this.#entry.route.set(Route.normalize(route)); this.#state.rebuildOriginated(); diff --git a/js/net/src/path.test.ts b/js/net/src/path.test.ts index 51d3e8a000..96ef51fd72 100644 --- a/js/net/src/path.test.ts +++ b/js/net/src/path.test.ts @@ -1,5 +1,4 @@ import { expect, test } from "bun:test"; -import * as Epoch from "./epoch.ts"; import * as Path from "./path.ts"; /** Brand a literal as a relative reference; the tests feed raw strings on purpose. */ @@ -327,41 +326,3 @@ test("encode enforces the max part count", () => { expect(Path.encode(atLimit)).toBe(atLimit); expect(() => Path.encode(`${atLimit}/extra` as Path.Valid)).toThrow(); }); - -// The Rust and JS path suites consume one fixture so parsing and ordering agree. -const epochs = (await Bun.file(new URL("../../../rs/moq-net/src/path/epoch.json", import.meta.url)).json()) as { - valid: Array<{ text: string; unix_ms: number }>; - invalid: string[]; - ordered: string[]; - paths: Array<{ path: string; name: string; epoch: string | null }>; -}; - -test("shared epoch parse, reject, order and time vectors", () => { - for (const row of epochs.valid) { - const epoch = Epoch.parse(row.text); - expect(epoch).toBe(row.text as Epoch.Valid); - expect(Epoch.time(epoch).getTime()).toBe(row.unix_ms); - } - for (const text of epochs.invalid) expect(() => Epoch.parse(text)).toThrow(RangeError); - const ordered = epochs.ordered.map(Epoch.parse); - expect([...ordered].reverse().sort()).toEqual(ordered); -}); - -test("shared epoch path vectors", () => { - for (const row of epochs.paths) { - const path = Path.from(row.path); - const { name, epoch } = Path.splitEpoch(path); - expect(name).toBe(row.name as Path.Valid); - expect(epoch ?? null).toBe(row.epoch as Epoch.Valid | null); - expect(Path.joinEpoch(name, epoch)).toBe(path); - } -}); - -test("epoch segments are literal pattern components", () => { - const epoch = Epoch.parse(epochs.valid[1].text); - const path = Path.joinEpoch(Path.from("demo/video"), epoch); - expect(Path.Pattern.parse("demo/**").matches(path)).toBe(true); - expect(Path.Pattern.parse("demo/video/@*").matches(path)).toBe(true); - expect(Path.Pattern.parse("demo/video").matches(path)).toBe(false); - expect(Path.Pattern.parse(path).matches(path)).toBe(true); -}); diff --git a/js/net/src/path.ts b/js/net/src/path.ts index 6c92478b76..0ea78131d6 100644 --- a/js/net/src/path.ts +++ b/js/net/src/path.ts @@ -35,7 +35,6 @@ * ``` * @module */ -import * as Epoch from "./epoch.ts"; /** A normalized broadcast path. */ export type Valid = string & { __brand: "Name" }; @@ -340,23 +339,3 @@ export { type Segment, type Specificity, } from "@moq/pattern"; - -/** Split off a final `@` segment, leaving all other paths unchanged. */ -export function splitEpoch(path: Valid): { name: Valid; epoch?: Epoch.Valid } { - const slash = path.lastIndexOf("/"); - const segment = path.slice(slash + 1); - if (segment.startsWith("@")) { - try { - const epoch = Epoch.parse(segment.slice(1)); - return { name: path.slice(0, Math.max(0, slash)) as Valid, epoch }; - } catch { - // `@alice` and other application segments remain valid literal paths. - } - } - return { name: path }; -} - -/** Append an optional epoch as a final `@` segment. */ -export function joinEpoch(name: Valid, epoch?: Epoch.Valid): Valid { - return epoch === undefined ? name : join(name, from(`@${epoch}`)); -} diff --git a/js/net/src/wire.ts b/js/net/src/wire.ts index a71c9ab1e4..e2cad504c5 100644 --- a/js/net/src/wire.ts +++ b/js/net/src/wire.ts @@ -10,6 +10,7 @@ import type { Dispose, GetPromise, Getter } from "@moq/signals"; import type * as broadcast from "./broadcast.ts"; import type { Drain } from "./connection/goaway.ts"; +import type * as Epoch from "./epoch.ts"; import type { Consumer as GroupConsumer } from "./group.ts"; import type { Route } from "./hop.ts"; import type * as origin from "./origin.ts"; @@ -32,7 +33,7 @@ export interface OriginProducer { accepts(prefix: Path.Valid): boolean; receive( prefix: Path.Valid, - route?: Route | { hops?: Route["hops"]; cost?: Route["cost"] | bigint }, + route?: Route | { epoch?: Route["epoch"]; hops?: Route["hops"]; cost?: Route["cost"] | bigint }, ): origin.Dynamic; attach(discovery: boolean): Dispose; expect(): Dispose; @@ -54,9 +55,11 @@ export interface OriginConsumer { routes(path: Path.Valid): boolean; readonly broadcasts: Getter | undefined>; readonly advertised: Getter; - /** The announced local broadcast at `path`, when it is the route peers are offered there. */ - local(path: Path.Valid): broadcast.Consumer | undefined; - demand(path: Path.Valid): Promise; + /** The announced local broadcast at `path`, when it is the route peers are offered there. + * A request naming an `epoch` only matches a route serving it. */ + local(path: Path.Valid, epoch?: Epoch.Valid): broadcast.Consumer | undefined; + /** Resolve `path` for serving; throws `Unroutable` when the route serves another `epoch`. */ + demand(path: Path.Valid, epoch?: Epoch.Valid): Promise; } /** One originated advertisement exposed to the publishing wire. */ diff --git a/js/publish/src/broadcast.ts b/js/publish/src/broadcast.ts index 807229943b..58aecf739b 100644 --- a/js/publish/src/broadcast.ts +++ b/js/publish/src/broadcast.ts @@ -225,6 +225,9 @@ export class Broadcast { // broadcast and new subscriptions land on the same producer. const broadcast = origin.createBroadcast(name); effect.cleanup(() => broadcast.close()); + // One publisher instance per broadcast, kept across unannounce and announce, so a + // restart is a new broadcast rather than a resume into the old one. + const epoch = Moq.Epoch.mint(); // Expose it before serving so an application reacting to `net` can insert its own tracks. this.net.set(broadcast); @@ -256,7 +259,7 @@ export class Broadcast { this.catalog.serve(track, inner, { compression }); } - broadcast.announce(); + broadcast.announce({ epoch }); }); // Static tracks fan out to every subscriber. Keep the encoder-facing handle demand-gated diff --git a/quest/m0/README.md b/quest/m0/README.md index d4139613da..2183b3d335 100644 --- a/quest/m0/README.md +++ b/quest/m0/README.md @@ -2,13 +2,11 @@ ## Goal -The work in flight now, in four independent tracks. Relay hardening: legal +The work in flight now, in three independent tracks. Relay hardening: legal moq-transport input never fails a session ahead of Seattle interop on 2026-10-12, every resource a peer can make the relay hold is bounded by what -it sent or by a budget, and no peer input panics the process. Routing: a -service claims the prefix it could serve instead of enumerating broadcasts. -Identity: nothing treats who published a route as what it carries; a path, -with its `@epoch`, is the only content identity, and every first-party +it sent or by a budget, and no peer input panics the process. Identity: nothing treats who published a route as what it carries; a path and +the epoch on its route are the only content identity, and every first-party publisher that can restart mints a fresh epoch, so #4741 stalls nobody. Audio playout: the target is a measured estimate of arrival timing in both languages, and a browser regression fails a nightly run. @@ -23,13 +21,13 @@ The Pronto GPU integration lives in moq.pro. The DoS hardening from an external review on 2026-09-29, verified against `main`, stays in m0 as security work. Its quests describe fixes, not exploits. -Routing: the wildcard line is prefix-only on the wire; its resolve and demand -work is done on the line branch and waits to land. Serving the relay's -ingested-only view (`origin::Consumer::local()`) to localhost workers belongs -to moq.pro's edge, which embeds moq-relay; it moved there on 2026-09-28. +Routing: Wildcard landed in #4403, so a service claims the prefix it could +serve instead of enumerating broadcasts. Serving the relay's ingested-only +view (`origin::Consumer::local()`) to localhost workers belongs to moq.pro's +edge, which embeds moq-relay; it moved there on 2026-09-28. Identity: the [broadcast epoch](/quest/m0/broadcast-epoch/README.md) line -ranks right after Wildcard and gates the next release (decided 2026-10-03: +gates the next release (decided 2026-10-03: #4741 resumes an un-epoched republish into the old broadcast and stalls its viewers). #4741 can merge to main, but no release ships until first-party publishers mint epochs. @@ -47,8 +45,7 @@ a published `@moq/watch` break. - [Request caps](/quest/m0/request-caps.md) - lite message sizes, IETF request IDs, and per-session announces and subscriptions are bounded - [Prefix route fronts](/quest/m0/prefix-route-fronts.md) - a prefix route cannot be made to mint one front per requested path - [qmux credit](/quest/m0/qmux-credit.md) - qmux returns connection credit for dropped and stopped streams and delivers its close frame, on both lines -- [Wildcard](/quest/m0/wildcard/README.md) - a relay resolves subscriptions against advertised prefixes, a service claims the prefix it could serve and refuses the rest instead of enumerating broadcasts, and the browser player treats a covering claim as availability -- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - every first-party publisher that can restart mints a fresh `@` epoch, viewers follow the newest live one, and bare names still resolve on every version +- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - every first-party publisher that can restart mints a fresh route epoch, the newest wins a path, and only routes with the same epoch resume a subscription - [Audio jitter target](/quest/m0/audio-jitter-target/README.md) - the audio playout target is a measured estimate of arrival timing in both languages, not a round-trip guess ## Related diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index b7ccfa5f5c..baa2a44c72 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -2,86 +2,76 @@ ## Goal -A path, with its `@epoch`, is the only content identity, and no first-party -publisher reuses one for different content. With #4741, any route covering a -path resumes its subscriptions from the first frame the subscriber lacks, -whoever serves it. So epochs are a correctness requirement: a publisher that -restarts its group numbering at 0 under an un-epoched name, while its old -route lingers, is resumed into the old broadcast and stalls viewers until its -sequence catches up. +A path and the epoch on its route are the only content identity, and no +first-party publisher reuses a pair for different content. Only routes with +the same epoch resume a subscription from the first frame it lacks; a route +without one keeps its subscriptions until it goes. So epochs are what make +failover seamless, and a restart is a new epoch at the same path: the newest +epoch wins and ends subscriptions to the old one, so viewers re-request +rather than stall on a replaced broadcast. -By default, each publish of `demo/BBB.hang` goes out as -`demo/BBB.hang/@`, so a restart is a new broadcast. A viewer of the -bare name follows the newest live epoch as soon as it is announced. Moving to -a new epoch is a clean boundary (a fresh catalog and tracks, never resumed -across), and each run stays addressable by its full path. +The epoch rides moq-lite 07 announcements and requests as metadata, so the +path never changes and every older version and moq-transport keeps working: +their routes carry no epoch, stay on one route, and see a restart as an end +and start at the same path. -The epoch rides in the path, so it survives any moq-transport relay, and no -wire message changes. At an epoch-aware relay, a request for a bare name -resolves to its newest live epoch on every protocol version. - -Non-goals: pooling, which needs nothing here, since every publisher of one -path is already one source; a redundant pair shares an explicit epoch through -[`--hop` removal](/quest/m0/broadcast-epoch/hop-removal.md). Also out of -scope: trusting the publisher's clock (a far-future epoch wins until its -route goes away). +Non-goals: pooling, which needs nothing here; a redundant pair shares an +explicit epoch through [`--hop` removal](/quest/m0/broadcast-epoch/hop-removal.md). +Also out of scope: trusting the publisher's clock (a far-future epoch wins +until its route goes away). ## Plan Decided: -- This line gates the next release (decided 2026-10-03: #4741 can merge to - main, but without epochs every restarting first-party publisher stalls its - viewers). -- The marker is a child segment `@`, parsed into - `Option` by [the shared primitive](/doc/concept/moq-lite.md#publisher-epochs). Not a lite-07 flag: - the path is the only carrier. -- The broadcast-publish path mints an epoch unless the path already carries - one. A caller who passes an explicit epoch, such as a redundant publisher, - keeps it. The prefix-route `announce(prefix, route)` stays raw, and that is - the opt-out. So prefix vs broadcast is an API distinction, not a wire flag. -- Viewers follow the greatest epoch with a live route. When it goes away and an - older one is still live, they fall back to it. -- A bare request with no route of its own resolves to that same epoch on every - version, so lite-06 and IETF clients keep working through an epoch-aware - relay. When a newer epoch appears, the bare subscription ends with a typed - reset, and the client's normal resubscribe lands on the new one. -- An unmodified third-party relay routes `foo/@` but never resolves a - bare `foo`, since a route covers its descendants, not its parent. A - bare-name viewer behind one needs a publisher that opts out with the raw - prefix route. Document this rather than promise it works. -- Publishers on the default publish path, such as moq-boy and moq-room, - inherit the epoch from Origin. moq-stats mints its own through - [Stats epochs](/quest/m0/broadcast-epoch/stats-epoch.md), which also gates the release - (decided 2026-10-04): a restarted stats node under a reused name stalls its - viewers the same way. -- Decided in the 2026-10-05 audit: the m1 quests gating this line (stats - epochs, the bounded stats aggregate it requires, and retracted demand - release) moved under it, and the OBS half of GStreamer and OBS moved to m1 - as [OBS publishes under epochs](/quest/m1/obs-epoch.md), so the release - gate no longer waits on m1 work. -- Derived output mirrors the epoch it came from - (`.pro/transcode//foo.hang/@e`, per the - [wildcard](/quest/m0/wildcard/README.md) line's derived-output layout), so - the service's prefix claim still covers it. +- This line gates the next release (decided 2026-10-03: without epochs every + restarting first-party publisher stalls its viewers). +- The epoch is route metadata, not a path segment (decided 2026-10-06, + replacing the `@` segment: a path suffix changes the name old + clients subscribe to). It is a lite-07 field on ANNOUNCE_START, TRACK, + SUBSCRIBE, and FETCH, with no negotiation and nothing on published versions. +- The route's epoch is taken as given: nothing mints one by default (decided + 2026-10-06). Each first-party publisher mints one per run and announces it; + a replica announces a shared one. A route without one, such as a + transcoder's prefix claim, stays on the worker that first served a + subscription and is never stitched to another worker's output. +- The newest epoch wins a prefix ahead of cost (decided 2026-10-06), and + replaces the old one with a hard switch: subscriptions in flight end with + `Unroutable`. When it goes and an older one is still live, the older one + wins again as a new broadcast. +- A catalog `broadcast` reference by name follows the newest epoch, since a + path cannot name one. +- Every first-party publisher that can restart mints its own: the apps, + moq-boy, and moq-room through [Apps](/quest/m0/broadcast-epoch/apps.md), the + ingest gateways, moqsink, and the bindings below. moq-stats mints one per group announcement + through [Stats epochs](/quest/m0/broadcast-epoch/stats-epoch.md), which + also gates the release (decided 2026-10-04). +- The m1 quests gating this line moved under it in the 2026-10-05 audit, and + the OBS half moved to m1 as [OBS publishes under epochs](/quest/m1/obs-epoch.md). + +- Until lite-07 is offered by default, default sessions (lite-06) carry no + epoch, so a cluster GOAWAY redial or a standby takeover ends subscriptions + instead of resuming them (accepted 2026-10-06 over negotiating the field on + lite-06). A release that needs seamless failover promotes lite-07 first. + +Open: -This README owns: +- Older versions and moq-transport lose cross-route resume, since their routes + have no epoch. The IETF joining-FETCH resume and the lite-05/06 resume + points are unreachable for them now; delete them or keep them for a future + moq-transport epoch extension. -- An end-to-end relay test: republish a name while the old publisher's session - stays open. A new-API viewer and a lite-06 or IETF bare-path viewer both - reach the new epoch within one RTT-scale bound rather than the idle timeout. - Killing the newest epoch falls back to a still-live older one. -- A `doc/concept` page on broadcast naming: what an epoch is, publish and - consume behavior, takeover and fallback, bare-path resolution, and the - prefix-route opt-out. +This README owns an end-to-end relay test: republish a name while the old +publisher's session stays open. A lite-07 viewer and a lite-06 or IETF viewer +both reach the new epoch within one RTT-scale bound rather than the idle +timeout, and killing the newest epoch falls back to a still-live older one. ## Required -- [Origin](/quest/m0/broadcast-epoch/origin.md) - moq-net publish mints an epoch, consumers follow the newest live one, and bare requests resolve to it on every version -- [Apps](/quest/m0/broadcast-epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web publish under epochs and play bare names +- [Apps](/quest/m0/broadcast-epoch/apps.md) - moq-cli, the browser publish and watch components, and demo/web restart into a new epoch and reset on the switch - [Gateways](/quest/m0/broadcast-epoch/gateways.md) - RTMP, SRT, and WHIP ingest mint an epoch per incoming connection, so an encoder reconnect is a clean takeover - [TS restart](/quest/m0/broadcast-epoch/ts-restart.md) - a signalled backward TS discontinuity finishes the broadcast and continues the same input under a fresh epoch -- [Bindings](/quest/m0/broadcast-epoch/bindings.md) - moq-ffi and every wrapper expose the epoch and inherit the default +- [Bindings](/quest/m0/broadcast-epoch/bindings.md) - moq-ffi and every wrapper expose the epoch and let a publisher announce one - [GStreamer](/quest/m0/broadcast-epoch/gst.md) - moqsink publishes each run under a fresh epoch - [Remove `--hop`](/quest/m0/broadcast-epoch/hop-removal.md) - `moq` takes an optional `--epoch` instead of `--hop`, a plain publisher declares a random Hop ID, and the per-session hop stamp is gone - [Stats epochs](/quest/m0/broadcast-epoch/stats-epoch.md) - moq-stats publishes each group announcement under its own epoch, so neither a restarted node nor a returning idle group stalls its viewers diff --git a/quest/m0/broadcast-epoch/apps.md b/quest/m0/broadcast-epoch/apps.md index cfbbdf28c2..3c758d3b03 100644 --- a/quest/m0/broadcast-epoch/apps.md +++ b/quest/m0/broadcast-epoch/apps.md @@ -1,23 +1,21 @@ -# [M] Apps publish under epochs and play bare names +# [M] Apps restart into a new epoch and reset on the switch ## Goal -`moq-cli` publish and play, `@moq/publish`, `@moq/watch`, and `demo/web` use -the origin default. Each publish run is a new epoch, so a restart while the +`moq-cli` publish and play, `@moq/publish`, `@moq/watch`, `demo/web`, +moq-boy, and moq-room announce a freshly minted epoch per run. Each publish +run is a new epoch, so a restart while the old route lingers is a new broadcast rather than a resume into the old one, -which stalls viewers until the new run's group sequence catches up. Watching -a bare name switches to a republish within an RTT. The UI and logs show the -full epoch path, and a watch link can pin one. +which stalls viewers until the new run's group sequence catches up. A viewer +switches to a republish within an RTT. Logs show the epoch. ## Plan -- Publish sides need little beyond passing bare names. Check that nothing - caches the announced path across a restart. +- Publish sides mint per run as of #4942: moq-cli publish, HLS import, + archive replay, WHEP import, and transcode output; moq-boy; `@moq/publish`, + the clock, and moq-boy's viewer feedback. What remains is below. - Watch sides handle "the broadcast changed" as a fresh catalog and decoder reset. Test a republish mid-playback in the browser and native players. - Update `doc/bin/cli.md` and every example invocation that shows a published path. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the publish default and follow logic diff --git a/quest/m0/broadcast-epoch/bindings.md b/quest/m0/broadcast-epoch/bindings.md index 4ee0c1b2a7..d1c883b753 100644 --- a/quest/m0/broadcast-epoch/bindings.md +++ b/quest/m0/broadcast-epoch/bindings.md @@ -2,10 +2,9 @@ ## Goal -`moq-ffi` and the py, swift, kt, go, and dart wrappers publish under the -default epoch and follow bare names like Rust. The generated C and C++ -bindings pick it up from moq-ffi; libmoq gets no new API. The epoch of a published or -consumed broadcast is readable, and a caller can pass an explicit one. The +`moq-ffi` and the py, swift, kt, go, and dart wrappers can announce an epoch like Rust, with a minting helper. The generated C and C++ +bindings pick it up from moq-ffi; libmoq gets no new API. The epoch on an announced route is +readable, and a publisher can pass an explicit one. The reconnect counter `session.epoch()` is renamed so "epoch" has one meaning. ## Plan @@ -24,6 +23,3 @@ Decided in the 2026-10-05 audit: this lands before the same wrappers and keeps `epoch()` today. It rebases onto this quest and adopts the epoch surface and the rename, so the wrappers break once each. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the behavior the bindings surface diff --git a/quest/m0/broadcast-epoch/gateways.md b/quest/m0/broadcast-epoch/gateways.md index 2124764d1f..fef1bca1a7 100644 --- a/quest/m0/broadcast-epoch/gateways.md +++ b/quest/m0/broadcast-epoch/gateways.md @@ -10,14 +10,12 @@ up. ## Plan -Use the origin default where the gateways create broadcasts (`moq-rtmp`, -`moq-srt`, `moq-rtc`, and the relay wiring). Test a reconnect under the same -key with the stale connection still open. Update `doc/bin/relay/` where it +Mint an epoch per incoming connection and announce it on the broadcast's +route (`moq-rtmp`, `moq-srt`, `moq-rtc`, and the relay wiring); nothing mints +by default. Test a reconnect under the +same key with the stale connection still open. Update `doc/bin/relay/` where it describes ingest paths. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the publish default ## Related diff --git a/quest/m0/broadcast-epoch/gst.md b/quest/m0/broadcast-epoch/gst.md index f4ec65fd19..a647f7f0f6 100644 --- a/quest/m0/broadcast-epoch/gst.md +++ b/quest/m0/broadcast-epoch/gst.md @@ -7,7 +7,7 @@ a clean takeover for viewers. ## Plan -`moqsink` takes the origin default per session. When +`moqsink` mints an epoch per session and announces it. When [#3115](/quest/m2/3115-moqsink-the-publication-has-no-generation-so-a-flush.md) lands, each of its publication generations is a new epoch. Update `doc/bin/gstreamer.md` if it shows paths. @@ -15,12 +15,9 @@ lands, each of its publication generations is a new epoch. Update Decided in the 2026-10-05 audit: the OBS half moved to [OBS publishes under epochs](/quest/m1/obs-epoch.md), so the release gate no longer waits on the C++ line. `moqsink` is Rust on moq-net and needs nothing -from it. moq-c announces through `broadcast::Producer::announce`, so it only -inherits an epoch if Origin mints one there; the OBS quest checks that. +from it. moq-c announces through `broadcast::Producer::announce`, which takes the +route's epoch as given, so the OBS quest has to pass one. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the publish default moqsink takes ## Related diff --git a/quest/m0/broadcast-epoch/hop-removal.md b/quest/m0/broadcast-epoch/hop-removal.md index e5bfad0bcb..56f5f310f9 100644 --- a/quest/m0/broadcast-epoch/hop-removal.md +++ b/quest/m0/broadcast-epoch/hop-removal.md @@ -2,10 +2,10 @@ ## Goal -Nothing names a publisher's identity outside its path. `moq` loses `--hop`, +Nothing names a publisher's identity outside its epoch. `moq` loses `--hop`, `MOQ_HOP`, and the hidden `--origin` alias; an optional `--epoch` takes their -place for publishing. Omitted, each run gets a fresh epoch (the origin -default); a redundant pair passes the same value. A plain publisher declares +place for publishing. Omitted, `moq` itself mints a fresh epoch per run, since +nothing mints one by default; a redundant pair passes the same value. A plain publisher declares a random Hop ID per process with no flag, as the bindings and js already do, so a multi-homed publisher still catches its own loops. `--cluster-id` names a node and no longer falls back to `--hop`. Relays stop stamping an unnamed @@ -26,11 +26,6 @@ publisher's hop chain. would let a shared front hairpin through a peer in a cluster. Viewers share a front anyway, because `Horizon::effective` ignores a hop no route chain names (#4922). -- NO_CAPACITY's removal (decided 2026-10-03: every refusal is terminal) - lands with the [wildcard](/quest/m0/wildcard/README.md) line, whose branch - already deletes it from js/net, moq-net, and both drafts (found in the - 2026-10-05 audit). Don't redo it here; rebase onto that line if it lands - first. - Drafts: lite-07 and cluster-02 are in progress, so edit their text and changelogs in place. In `drafts/draft-lcurley-moq-lite.md`: the stamping rule, "the identity it stamps", @@ -53,6 +48,3 @@ Public API: removes `--hop`, `MOQ_HOP`, and `--origin`, adds `--epoch`, and `--cluster-id` stops reading `--hop`. Wire: relays stop stamping. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the default mint `--epoch` falls back to diff --git a/quest/m0/broadcast-epoch/origin.md b/quest/m0/broadcast-epoch/origin.md deleted file mode 100644 index 18ef98bf23..0000000000 --- a/quest/m0/broadcast-epoch/origin.md +++ /dev/null @@ -1,57 +0,0 @@ -# [L] Origin: mint, follow, and resolve epochs - -## Goal - -In `moq-net` (Rust and JS), publishing a broadcast at a bare path announces it -under a fresh epoch. A consumer of a bare name follows the newest epoch with -a live route and falls back to an older live one. A bare request resolves the -same way on every protocol version, so the relay serves old clients without -any wire change. - -## Plan - -- Publish: the broadcast create-and-announce path appends `Epoch::mint()` - unless the path already has an epoch. `announce(prefix, route)` stays raw. - A bare path already at the 32-part limit has no room for the epoch segment: - refuse it at publish, pointing at the raw route, rather than fail at encode. -- Auth: a grant that admits bare `foo` admits `foo/@` for both publish - and subscribe, and a grant on one epoch never widens to the bare name or its - siblings. Check how exact grants and patterns match today, and test both - sides in moq-relay's auth tests. -- Consume: `routed` and `request_broadcast` on a bare name watch the routes - one `@` segment below it. They pick the greatest live epoch, and on a table - change, re-select: move up at once, or fall back when the current one is - retracted. A path that names an epoch pins it and never moves. Each move is - a new broadcast to the caller, never a splice. Within one epoch path, any - route resumes a subscription, whoever serves it. -- Bare resolution: a bare request with no route of its own resolves to that - epoch. A takeover ends the bare subscription with a typed reset, never a - silent switch. Choose the code so existing clients resubscribe rather than - give up. -- A nested epoch (`name/@e/derived/@f`) resolves per level. Split-horizon - exclusion and the per-path fronts from #3312 stay intact, keyed by effective - exclusion (`Horizon::effective`, #4922, decided 2026-10-06): a requester's - hop that no covering route chain names excludes nothing, so viewers share - one front per epoch path and only a peer in a chain gets a filtered one. - Following a bare name computes it over the epoch route it selects. -- Benchmark resolution swept over epochs per name and names per origin, so - following does not scan the table. -- Settle here, in this m0 quest, whether a catalog `broadcast` reference by - bare name pins the epoch its catalog came from or follows the newest - (decided in the 2026-10-05 audit: it is not handed to - [Catalog track alias](/quest/m1/catalog-track-alias.md), an m1 quest that - never mentions epochs). Record the answer in the line README. -- Update `doc/concept` and `drafts/draft-lcurley-moq-lite.md` wherever they - describe resolution or takeover. The rule is a relay behavior, so state it - in the draft even though no field changes. - -Public API: behavior change on publish (the announced path gains an epoch, -and a 32-part bare path is refused), on bare-name consume, and on what an -exact grant admits. -Wire: none. - -## Related - -- [Catalog track alias](/quest/m1/catalog-track-alias.md) - cross-broadcast catalog references, which must pick an epoch - -- [#2991](/quest/m1/2991-net-coalesce-dynamic-tracks-and-preserve-sequences-across.md) - a new epoch starts each track at sequence 0 diff --git a/quest/m0/broadcast-epoch/stats-epoch.md b/quest/m0/broadcast-epoch/stats-epoch.md index 85f21240f1..8fe0f13219 100644 --- a/quest/m0/broadcast-epoch/stats-epoch.md +++ b/quest/m0/broadcast-epoch/stats-epoch.md @@ -2,10 +2,10 @@ ## Goal -Each stats group broadcast publishes under its own epoch, -`[/]/node//@`, minted each time the group is -announced, so neither a restarted node nor a group returning from idle reuses -a broadcast name or the group numbers a relay cached under it. +Each stats group broadcast at `[/]/node/` announces under +its own epoch, minted each time the group is announced, so neither a restarted +node nor a group returning from idle reuses a broadcast identity or the group +numbers a relay cached under it. ## Plan @@ -16,17 +16,15 @@ a broadcast name or the group numbers a relay cached under it. linger and drops its totals; when it returns it announces under a new epoch counted from zero. Why: a producer never holds an idle group's state, so its memory is bounded by active and lingering groups, and readers see every - reset as a new path rather than detecting one. The cost: a reader that + reset as a new broadcast rather than detecting one. The cost: a reader that misses every frame across the linger loses that epoch's tail; billing under-bills by that tail, consistent with the 0-bill baseline. At depth 0 the single broadcast never unannounces, so its epoch still lasts the producer's life. - [#4739](https://github.com/moq-dev/moq/pull/4739)'s original commits are - prior art: path syntax, `parse_node_path`, the - aggregator's restarted-epoch handling, and the tests. The PR landed only the + prior art for the aggregator's restarted-epoch handling and the tests; its + path syntax is superseded, since the epoch is on the route. The PR landed only the producer-wide group allocator. -- Decide what an unset node becomes once the epoch follows it (#4739 used - `local`). - The aggregator treats a new epoch as a new node: its counters add to the merged total instead of regressing it, and the old epoch folds into the retired total once `aggregate::Config::grace` elapses. Test it. @@ -45,5 +43,5 @@ a broadcast name or the group numbers a relay cached under it. - demo/web stats keys include the epoch. Update `doc/concept/stats.md`, `doc/bin/relay/config.md`, and the relay stats config docs. -Public API: path shape change for every stats consumer. Wire: stats broadcast -names gain a trailing epoch segment. +Public API: stats consumers read the epoch from the route. Wire: none beyond +the route epoch. diff --git a/quest/m0/broadcast-epoch/stats-split.md b/quest/m0/broadcast-epoch/stats-split.md index 51d3addf00..222bc2c2d9 100644 --- a/quest/m0/broadcast-epoch/stats-split.md +++ b/quest/m0/broadcast-epoch/stats-split.md @@ -18,7 +18,7 @@ Decided 2026-10-05 (planned from moq-dev/moq.pro#2202): ends inside a group a reader missed shows in no later frame, so a lagging reader undercounts. The maps also grow with every live broadcast, and readers like billing want only per-project sums. -- **Totals.** Per group broadcast (`[/]/node//@`), +- **Totals.** Per group broadcast (`[/]/node/`), per tier and role, cumulative within the [stats epoch](/quest/m0/broadcast-epoch/stats-epoch.md). Every group announcement, and so every restart, starts a new epoch counted from zero; nothing is serialized to disk. `Registry`'s unpruned lifetime totals @@ -41,10 +41,10 @@ Decided 2026-10-05 (planned from moq-dev/moq.pro#2202): the linger (same epoch, totals continue), and one returning after it (new epoch from zero). - **Prefix tracks** (decided 2026-10-05, replacing per-broadcast tracks). - Routing is by prefix, and with epochs a broadcast is `/@`, so - a reader requests any prefix of a member path (so its depth is bounded by - `Path::MAX_PARTS` less the group's own segments) and gets one track: the prefix's rollup plus a one-level map of - each direct child's rollup (a broadcast's epochs, a channel's broadcasts). + Routing is by prefix, so a reader requests any prefix of a member path (so + its depth is bounded by `Path::MAX_PARTS` less the group's own segments) and + gets one track: the prefix's rollup plus a one-level map of each direct + child's rollup (a channel's broadcasts). Nothing is produced for a prefix no one holds. The relay's `--stats-depth` only places the group broadcasts; requests are not capped by it. - Flow counters (bytes, frames, groups, fetches) count from zero when the @@ -114,3 +114,4 @@ across nodes. - [Media stats](/quest/m1/stats/README.md) - publisher and viewer media stats stay hang tracks, separate from the relay's stats +- [QoS](/quest/m1/qos/README.md) - stats-split lands first, then #4133 rebases its lag and drift histograms onto the totals and prefix tracks diff --git a/quest/m0/broadcast-epoch/ts-restart.md b/quest/m0/broadcast-epoch/ts-restart.md index f302c717bf..75561176ec 100644 --- a/quest/m0/broadcast-epoch/ts-restart.md +++ b/quest/m0/broadcast-epoch/ts-restart.md @@ -19,12 +19,12 @@ and a flagged forward jump publishes break markers and carries on. Decided (maintainer, 2026-09-30): -- A rewind is new content, so it is always a new broadcast at a new epoch - path, never a continuation of the old name. Viewers of the bare name follow - it through [Origin](/quest/m0/broadcast-epoch/origin.md). +- A rewind is new content, so it is always a new broadcast at a new epoch, + never a continuation of the old one. Viewers follow it because the newest + epoch wins the path. - `decode` stops at the flagged rewind and reports it. The caller finishes the old broadcast (a clean end, not an abort, so its viewers read to its end), - publishes a new broadcast at a fresh epoch path, and calls + publishes a new broadcast at the same path, minting a fresh epoch, and calls `import.restart(broadcast)`. The importer carries over only the PAT/PMT layout and the bytes it has not consumed, so there is no wait for the next PSI repetition; tracks, groups, and timestamps start fresh. `ts::Programs` @@ -43,9 +43,6 @@ connection carries both epochs. Update `doc/bin/cli.md` and `doc/bin/srt.md`. Public API: breaking in moq-mux, `ts::Import::decode` reports a restart and `restart` is new. Wire: none. -## Required - -- [Origin](/quest/m0/broadcast-epoch/origin.md) - the fresh epoch path the rest of the feed publishes under ## Closes diff --git a/quest/m0/wildcard/README.md b/quest/m0/wildcard/README.md deleted file mode 100644 index a901c85577..0000000000 --- a/quest/m0/wildcard/README.md +++ /dev/null @@ -1,216 +0,0 @@ -# [S] Wildcard advertisements - -## Goal - -A service claims the prefix it could serve rather than enumerating every -broadcast under it; the client library filters that claim against the -pattern interest the caller asked for, so nothing on the wire spells a -wildcard. A claim is priced at what starting the work would cost. The longest -covering prefix wins first: a concrete announcement shadows a broader claim -regardless of cost, and prices compete only among claims of the same prefix. -A terminal concrete refusal does not fall through to a catch-all; its claim -must be withdrawn. Retracting a claim stops new work without shedding what is -already running. - -Three workloads need this. A transcode -worker today announces a standby derivative for every matching live broadcast, -so announcements scale as workers times broadcasts; claiming one service -prefix, it advertises once for the whole fleet. A chat backend -cannot enumerate at all: rooms exist independently of any broadcast, and the -subtree pattern `/chat/**` expresses them. An archive serving recordings -over FETCH wants to say "if nobody is publishing this live, I have it", which -is the catch-all `**`, a claim about every path at once. - -The cost of enumerating is real even though its last measurement is stale. -One announcement measured 8.8 KB per relay plus 4.3 KB per additional route -before prefix routes made a standby route a table entry. Whatever the current -number, every relay that hears an announcement pays it whether or not anything -there subscribes, so "workers times broadcasts" is that number multiplied -across the fleet in resident memory. - -## Plan - -Decided in [#3770](https://github.com/moq-dev/moq/pull/3770): publishing is -prefix-only on every wire and patterns never leave the token or the client library. -`dynamic(prefix, route)` -advertises a prefix; the catch-all claim is the root prefix, and the request -is the authority, so the advertise half of this questline is re-scoped to -prefix claims resolved against pattern interest. The three workloads above -still hold: the transcoder claims its service prefix -([Where derived output lives](#where-derived-output-lives)), and the archive -claims the root and refuses what it does not have. - -### What already exists, and what does not - -Route cost already names this case: "The original publisher seeds it with its -production cost: zero for a live publish, something large for a standby that -would have to start working (a cold transcoder)" -(`drafts/draft-lcurley-moq-lite.md`). `moq_auth::Claims.publish` and -`origin::Producer` gained versioned patterns on the -[Auth](/quest/m1/auth/README.md) line, so tokens and filters reuse the -same matcher; advertisements stay prefixes. `Cost { warm, cold }` -(`rs/moq-net/src/model/origin.rs:404`) is the route cost since -[#2925](https://github.com/moq-dev/moq/pull/2925). - -[moq#3225](https://github.com/moq-dev/moq/pull/3225) moved a long way toward -this, and #3770 settled the wire: an announcement carries a path prefix on -every protocol, and a consumer filters announced paths against its pattern -interest locally. - -Request resolution exists too. `Consumer::request_broadcast` mints a front per -path (`rs/moq-net/src/model/front.rs`) that selects through `best_route`: a -local broadcast first, then the longest covering prefix, filtered by the -requester's effective excluded hop (ignored unless a covering chain names it, -so viewers share a front) and ordered by `route_order`, whose hash is keyed on -the requested path so one prefix's pool shares its paths. A front follows the -best route and resumes through any covering one, and FETCH resolves the same -way. A standing refusal ends the front (#4875), so it never reaches a sibling -advertiser or a shorter prefix. The pattern matcher itself exists: -`moq_net::{Pattern, Patterns, Segment}` and `Path.Pattern` / -`Path.Patterns` in `js/net/src/path.ts` own the shared matching, containment, -specificity, and rebasing tokens and filters reuse. - -Content identity is the path: it names one broadcast whoever serves it, and -any covering route resumes a subscription from the first frame the subscriber -lacks (#4741). The `@` segment (#4706) is what makes a restart a new -path. So this questline settles collisions with interchangeable output at one -path, not with a route identity or a generation field. - -### Decisions - -- **One prefix on the wire, one pattern in the token and the filter.** An - advertisement is a path prefix; the pattern dialect from the - [Auth](/quest/m1/auth/README.md) line is what tokens and the consume-side filter use, matched by the - shared matcher, so nothing resembles a second grammar and nothing on the - wire spells a wildcard. -- **Longest prefix wins, and its refusal is final.** `best_route` filters to - the longest covering prefix before it compares cost, and the lite and - cluster drafts say the same. Advertisers of one prefix form one pool that - cost and the request hash order. A refusal from the winning tier IS the - answer and never falls through to a shorter prefix or to another - advertiser, so a transcoder refusing a path does not leak the request to - the archive's catch-all, and one unserved path costs one round trip. The - accepted consequence: an offline derivative is not reachable through the - catch-all while a longer prefix covers it. -- **A claim is a POOL, not a competitor.** Several advertisers of one - prefix is the normal state, not a hazard: every transcode worker claims the - same prefix and takes a share. What distributes them is a - deterministic hash of the REQUESTED path against each advertiser, so distinct - paths spread rather than one advertiser winning the whole prefix. - Distribution is the requirement, not any particular pair: a correct hash may - legitimately rank the same advertiser first for two given paths, so what must - hold is that a large path set spreads and that one path always resolves the - same way. Cost orders the pool first, which keeps work local and makes a - distant advertiser the overflow rather than an equal peer. -- **A claim is priced, not special-cased.** Within a tier, route selection - stays one comparison on one metric. Concrete-versus-claim is not decided - by price at all: a concrete announcement is the longest prefix, so "longest - prefix wins" above already shadows every claim behind it at any cost. The - accepted consequence follows from that rule's finality: a live session's - concrete claim shadows a healthy pool even when its service is - broken, its terminal refusal does not fall through, and the shadow lasts - exactly as long as the claiming session that carries it. - The seed still has a floor, because standby and running claims of the same - prefix do meet: a standby concrete claim (`with_cost(1000)` is the - existing per-broadcast convention) shares a tier with a running publisher's - concrete announcement and with warm-advertise's exact-path warm routes. The - floor MUST exceed the deployment's enforced maximum charged-link count - times its enforced maximum link cost (32 links at cost at most 5 gives - a bound of 160, with producing origins seeded at 0), or a nearby standby outranks a distant running copy and the - mesh starts a second encode of a stream it is already serving. That floor - replaces the ad-hoc standby bias the moq.pro (downstream) transcode worker - carries today. -- **A claim is a capability, not an inventory.** It advertises what the - sender could serve, never that a given path exists. Refusal is how a specific - path is denied. This is why an over-claiming advertisement is not a defect: - claiming the root is legal, and answering "not that one" is the mechanism. -- **Overlap with the publish scope is what authorization checks.** An - advertised prefix MUST overlap the sender's granted patterns or it is - refused. A prefix wider than the grant is accepted, but it only routes - requests for paths the grant covers. Fleet-wide services use the cluster - identity; a customer service serves only what its own grant contains. - The publish scope authorizes advertising; there is no separate advertise - scope (dropped with the processor questline in the 2026-10-06 audit). -- **Claims are visible to subscribers.** A claim is an ordinary prefix - announcement, so it tells a client it may subscribe beneath it, and its - withdrawal tells the client the capability is gone. The browser player's - gate (`js/watch`'s `#isPathAnnounced`) lists a catalog rendition under any - covering prefix, so a lazily-produced rendition is discoverable without an - announcement that only demand would produce. -- **Every refusal is a terminal typed stream reset, with no negative cache.** - An advertiser resets a subscribe it will not serve, and the reset carries - which KIND of refusal it is (`Error::to_code` already puts a typed code on - the wire). Every refusal propagates: a path no rule covers, an unauthorized - one, one that does not exist, or one the advertiser has no capacity for. - There is no re-resolution (decided 2026-10-03: NO_CAPACITY is removed, so - no refusal is retryable). An advertiser sheds load by withdrawing or - re-pricing its claim; a request that crosses the retraction is refused, and - the client's ordinary resubscribe resolves again. Scanning unserved paths - costs one round trip per path. No negative cache; rate limiting stays with - the advertiser and the per-project auth gate. -- **A double claim is settled by interchangeable output, not by route - identity or a lease.** Two relays can hash one path to different workers - before either concrete announcement propagates, and both land at the SAME - literal path. A path is one broadcast whoever serves it, so claim workers - mirror the source's epoch and group numbers, publish a deterministic - catalog, and start at group boundaries - ([Transcoders start at group boundaries](/quest/m0/wildcard/transcode-group-start.md)). - Two workers at one path are then one broadcast, and a relay moving between - them does so at a group boundary (decided 2026-10-03: #4741 drops - first-hop identity, so routing can no longer tell two workers apart). - Wildcard routing invents neither a lease nor a generation. That quest is - Required here (decided 2026-10-04), so this line cannot ship with workers - that can be spliced mid-group. -- **A serving front follows the best route.** A join or re-price re-selects - every front below the prefix, and a front moves when another route now - wins: cheaper, or the rendezvous winner among equal costs (decided - 2026-10-04, replacing Stay). This is safe because a path is one broadcast - and any covering route resumes it (#4741). -- **No reply Origin.** The lite-07 `Origin` field in SUBSCRIBE_OK and FETCH_OK, - and the rule that a relay MUST NOT splice across differing Origins, are - dropped (decided 2026-10-03: nothing needs them for correctness). -- **Patterns are independent of clustering.** The `moq-pattern` crate owns - the matching semantics tokens and filters share, with no draft of its own; - no announce message carries a pattern on either protocol (AUTH grants on - the wip lite version do, per the [Auth](/quest/m1/auth/README.md) line). - moq-cluster adds hop lists, costs, pool selection, and request resolution - to prefix advertisements. - -### Where derived output lives - -A prefix claim needs the variable part of a path trailing, so a fleet-wide -service claims its own prefix and mirrors the source path beneath it -(`.pro/transcode//foo.hang`, moq.pro's convention) rather than publishing -beneath the source. The source's catalog reaches the contribution through a -cross-broadcast reference. - -The leading `.` is deliberate. Existing customers on moq-lite-06 or older must -never see `.pro/` broadcasts, which could confuse their business logic. Those -versions cannot opt into hidden routes, so the relay never announces them -there. Hidden routes are a moq-lite-07 feature, so the player's covering check -opts into them and sees a claim only when lite-07 is negotiated; the token's -scope must still reach them. Finalizing lite-07 is a rollout condition, not a -blocker for this line (decided in the 2026-09-30 audit), since the check works -whenever lite-07 is negotiated. A customer who wants transcodes upgrades, or -subscribes to the explicit `.pro//...` path, which works on any -version. Grants and metering are the deployment's; moq.pro's are in its -[wildcard questline](https://github.com/moq-dev/moq.pro/blob/main/quest/m2/wildcard/README.md). - -The archive serves the source path itself: a recording IS the broadcast, -served from storage through the root claim, and a live publisher's concrete -announcement shadows it. A claim names no generation, so a client that must -distinguish recording generations reads the catalog's archive entry -([archive](/quest/m1/archive/README.md)) rather than announce state. - -## Required - -- [Transcoders start at group boundaries](/quest/m0/wildcard/transcode-group-start.md) - two claim workers at one path are one broadcast, so a relay never splices them mid-group - -## Related - -- [archive](/quest/m1/archive/README.md) - an archive claims the root, and its - catalog names the generations a claim cannot -- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - derived output - mirrors the source path, `@` segment included -- [Announcement shapes](/quest/m2/announce-shapes.md) - moq-lite-only exact, - suffix, and prefix+suffix claims that survive relay hops diff --git a/quest/m0/wildcard/transcode-group-start.md b/quest/m0/wildcard/transcode-group-start.md deleted file mode 100644 index 1496afdb67..0000000000 --- a/quest/m0/wildcard/transcode-group-start.md +++ /dev/null @@ -1,32 +0,0 @@ -# [S] Transcoders start at group boundaries - -## Goal - -Two moq-transcode instances serving one derived path are interchangeable at -group boundaries, so a relay moving a subscription between them never splices -two encoders' output mid-group. moq-transcode (`rs/moq-transcode`, and -`moq transcode` in moq-cli) resolves a subscription that starts at (N, M>0) -to (N+1, 0), and refuses a FETCH that starts mid-group. Output groups mirror -the source's sequence numbers, and the catalog derives only from the input and -config. - -## Plan - -- The lite draft already lets a publisher skip a group it cannot serve from - the requested frame and resolve to a later one, so no wire change. -- Output groups already mirror the source's sequences (`rung.rs`). Keep that, - and check that nothing per-process (start time, random ids, encoder - defaults) leaks into the catalog. -- Tests: a subscription and a FETCH that start mid-group, and two instances - fed the same source that publish the same catalog and group sequences. -- Done in [#4812](https://github.com/moq-dev/moq/pull/4812): the rung refuses a - mid-group FETCH, and two instances mirror the source's group sequences and - timestamps. The two-instance catalog check is dropped until - [#4817](https://github.com/moq-dev/moq/pull/4817) decides whether the catalog - must be deterministic; the rest waits on its bare-path rule. - -Public API: none. Wire: none. - -## Related - -- [Wildcard](/quest/m0/wildcard/README.md) - a double claim's two workers at one path are settled by interchangeable output diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index 039eeac55b..feb8ad71e6 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -169,6 +169,4 @@ owned by that prerequisite, not duplicated in archive storage. - [Bounded HLS playlists](/quest/m1/hls-bounded.md) - the live renderer rules a replayed archive inherits, planned outside this line - [Catalog track identity](/quest/m2/catalog-tracks.md) - explore immutable definitions or explicit version binding independently of archives - -- [wildcard](/quest/m0/wildcard/README.md) - catch-all routing exposes an archive at its stable replay path - [e2ee](/quest/m1/e2ee/README.md) - protected broadcasts are excluded initially diff --git a/quest/m1/archive/dvr.md b/quest/m1/archive/dvr.md index 3dcdfc42cb..b0885ab506 100644 --- a/quest/m1/archive/dvr.md +++ b/quest/m1/archive/dvr.md @@ -29,9 +29,8 @@ whether expiry during a seek needs a group eviction API in `moq-net`. `moq-hls` reads a timeline from the catalog's own broadcast, and an `archive.replay` path only marks it non-durable. Decided (09-29): the exporter does not follow `replay`, and viewers don't address a separate replay -broadcast. A recording is the broadcast. Under the -[wildcard](/quest/m0/wildcard/README.md) plan the archive serves the source -path through the root claim, and a live announcement shadows it. So when live +broadcast. A recording is the broadcast. Since Wildcard (#4403) the archive +serves the source path through the root claim, and a live announcement shadows it. So when live ends, `moq-hls` resolves the same name and falls through to the recording: playlists keep serving for rewind and for players finishing the last segments. The fall-through needs the recording to publish its catalog live, @@ -56,7 +55,3 @@ numbers when the name moves from the live publisher to the archive. ## Closes - [#2275](https://github.com/moq-dev/moq/issues/2275) - close this issue when the quest finishes - -## Related - -- [Wildcard](/quest/m0/wildcard/README.md) - landed (#4403) the root claim that serves the source path once the live announcement ends diff --git a/quest/m1/cluster-routing/README.md b/quest/m1/cluster-routing/README.md index 9809bcdaed..6c69ebf292 100644 --- a/quest/m1/cluster-routing/README.md +++ b/quest/m1/cluster-routing/README.md @@ -36,7 +36,7 @@ vector the 2026-09-30 cache-tiers audit kept: condition (RFC 8966) keyed by node id. Babel was set aside earlier because it is not loop-free for a prefix with several origins (RFC 8966 s2.7); routing to a node removes that case, and a redundant pair is two ANNOUNCEs - of one epoch path. + of one path under one epoch. - **Next-hop authority.** A node stores every neighbour's announces but treats as live only those from its current next hop toward the origin. Babel's next hops toward one origin form a tree, so an END is final and no cycle of @@ -139,9 +139,8 @@ Once every child has landed: ## Related -- [Wildcard](/quest/m0/wildcard/README.md) - the longest-prefix rule and pool spread Selection builds on - [P2P](/quest/m3/p2p/README.md) - browser and native peers that become routing nodes over this layer -- [Remove `--hop`](/quest/m0/broadcast-epoch/hop-removal.md) - redundant publishers share an explicit `@` +- [Remove `--hop`](/quest/m0/broadcast-epoch/hop-removal.md) - redundant publishers share an explicit epoch - [One route cost](/quest/m1/route-cost.md) - Warm and Cold collapse to one static cost - [Same-epoch importers](/quest/m1/hop-aligned-import.md) - the importer half of a redundant pair under one explicit epoch - [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - a redundant pair shares one epoch diff --git a/quest/m1/cluster-routing/selection.md b/quest/m1/cluster-routing/selection.md index 71100d7e83..9636711a4a 100644 --- a/quest/m1/cluster-routing/selection.md +++ b/quest/m1/cluster-routing/selection.md @@ -6,31 +6,33 @@ A broadcast under overlapping announcements routes to one origin chosen deterministically, and a relay spreads paths over equally ranked next hops (an edge over its region's cores) so every relay sends a given path the same way. -Origins that announce the same concrete path are one source, so a subscriber -resumes on another when the incumbent ends or becomes unreachable. +Origins that announce the same concrete path under the same epoch are one +source, so a subscriber resumes on another when the incumbent ends or becomes +unreachable. ## Plan -Most of this landed with [Wildcard](/quest/m0/wildcard/README.md) (#4403) and -#4741: `route_order` ranks the longest covering prefix first, keys its hash on -the requested path so equal-cost advertisers of one prefix split paths between -them, and a concrete path is one source with mid-group resume. Terminal -refusal landed too. What remains (cut 2026-10-06, after #4403 merged): the HRW -split test, the same-path failover test, and spreading paths over upstream -links. +Most of this landed with Wildcard (#4403), #4741, and publisher epochs +(#4942): `route_order` ranks the longest covering prefix first, then the +newest epoch, keys its hash on the requested path so equal-cost advertisers of +one prefix split paths between them, and a path plus epoch is one source with +mid-group resume. Terminal refusal landed too. What remains (cut 2026-10-06, +after #4403 merged): the HRW split test, the same-path failover test, and +spreading paths over upstream links. Decided: - Origin choice is per broadcast, not per announced prefix. -- A concrete path, with or without an epoch, is a source's identity: every - origin announcing it is interchangeable, which is how a redundant pair - works under one explicit epoch. There is no per-origin identity, for claim - output or for paths without an epoch, and the reply names no origin - (decided 2026-10-03: #4741 makes a path one broadcast whoever serves it). - Failover resumes from the first frame the subscriber lacks, mid-group, - except where a publisher starts at a group boundary, as claim workers do - per [Wildcard](/quest/m0/wildcard/README.md)'s double-claim rule. So a - relay never needs to tell claim output from a redundant pair. +- A path plus its [publisher epoch](/doc/concept/moq-lite.md#publisher-epochs) is + a source's identity (decided 2026-10-06, replacing the 2026-10-03 + path-only rule): origins announcing the same path under the same epoch + are interchangeable, which is how a redundant pair works under one + explicit epoch, and the reply names no origin. Each publisher instance, + claim workers included, mints its own epoch, so instances are never + spliced: the newest epoch wins and the older one's subscriptions end. A + route without an epoch names no instance and is never resumed elsewhere. + Failover within an epoch resumes from the first frame the subscriber + lacks, mid-group. - Every hop re-selects (decided 2026-10-01): SUBSCRIBE names no origin, since a pin breaks subscription aggregation. - A refusal is terminal, and an origin sheds load by withdrawing or @@ -56,4 +58,4 @@ with no timestamp rewind. ## Related -- [Wildcard](/quest/m0/wildcard/README.md) - landed the longest-prefix rule and pool spread this builds on +- [Broadcast epochs](/quest/m0/broadcast-epoch/README.md) - the per-instance identity failover resumes within diff --git a/quest/m1/e2ee/README.md b/quest/m1/e2ee/README.md index 69c11db08a..a67dcf4889 100644 --- a/quest/m1/e2ee/README.md +++ b/quest/m1/e2ee/README.md @@ -4,7 +4,7 @@ TypeScript and Rust publishers and subscribers interoperate over encrypted MoQ broadcasts while every relay, cache, and control plane remains unable to recover content. -Every application payload and semantic track name is confidential and authenticated. MoQ still exposes the outer broadcast path (an opaque name and the epoch), opaque physical track names, group and frame structure, timestamps, sizes, and traffic patterns; padding and metadata-flow confidentiality are out of scope. +Every application payload and semantic track name is confidential and authenticated. MoQ still exposes the outer broadcast path (an opaque name), the route's epoch, opaque physical track names, group and frame structure, timestamps, sizes, and traffic patterns; padding and metadata-flow confidentiality are out of scope. ## Plan @@ -19,7 +19,7 @@ The contract is [draft-lcurley-moq-e2ee](/drafts/draft-lcurley-moq-e2ee.md), pro ### Epoch and identity -- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `/@`, where `` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. The epoch is the shared [`Epoch`](/doc/concept/moq-lite.md#publisher-epochs) primitive. Subscribers discover instances under the `/` prefix and take the greatest epoch, which sorts newest; a known full path carries its epoch in the last segment. +- Every publisher instance mints an epoch, a UUIDv7 in lowercase text, and publishes at `` with the epoch on its route, where `` derives from the credential and the semantic broadcast name without the epoch. The epoch is an input to every HKDF derivation, so a restart, takeover, or explicit group sequence cannot repeat a nonce under a key: nothing is persisted across instances and no generation counter is redistributed. The epoch is the shared [`Epoch`](/doc/concept/moq-lite.md#publisher-epochs) primitive. Subscribers resolve `` and take the epoch from its route, where the newest wins; on a wire without route epochs the application supplies it. - The epoch is untrusted and unauthenticated. A wrong epoch fails authentication and a withheld one denies service; neither can make a nonce repeat, because only the publisher instance chooses what it encrypts under. This is the same trust a cache needs to serve the right instance. Plaintext broadcasts adopt the same epoch by default through [broadcast epochs](/quest/m0/broadcast-epoch/README.md). - Nothing in the path says the bytes are encrypted. The format after decryption (`meeting.hang`) is inside the opaque name; a plaintext player, exporter, or matcher that opens a protected broadcast finds no catalog it can read and fails with its usual typed refusal, the same as for any format it does not support. - One 32-byte secret authorizes the whole broadcast; HKDF-SHA-256 derives separate AES-128-GCM keys for each physical track and for grouped-frame versus datagram domains. A grouped frame uses the 96-bit nonce `uint64_be(group) || uint32_be(frame)`; a datagram uses its sequence with frame zero under the datagram domain. Empty AAD: every immutable end-to-end field is in the HKDF info or the nonce. diff --git a/quest/m1/obs-epoch.md b/quest/m1/obs-epoch.md index 0866b9fac0..33a69befc0 100644 --- a/quest/m1/obs-epoch.md +++ b/quest/m1/obs-epoch.md @@ -15,12 +15,9 @@ release gate waits only on `moqsink`. - OBS gets the epoch through the generated C++ bindings from moq-ffi, which the C++ line moves the plugin onto (decided in the 2026-09-30 audit: libmoq, now moq-c, gets no new API). -- First check whether the plugin already inherits it: today it announces - through moq-c's `moq_publish_announce`, which calls - `broadcast::Producer::announce`. If Origin mints the epoch on that path, - the plugin needs no code, only a test. -- Show the full epoch path in the dock, and update `doc/bin/obs.md` if it - shows paths. +- Nothing mints by default: the plugin mints one per Start Streaming and + announces it through the route. +- Show the epoch in the dock, and update `doc/bin/obs.md` if it shows it. ## Required diff --git a/quest/m1/route-cost.md b/quest/m1/route-cost.md index 19476c1110..7395a02a82 100644 --- a/quest/m1/route-cost.md +++ b/quest/m1/route-cost.md @@ -19,8 +19,8 @@ state, so Warm has no job. ignore Cold, and write the one cost as Warm with Cold at the saturation ceiling. - The single cost still prices a standby claim above a live one, per the - wildcard line's standby floor. Update that text in - `quest/m0/wildcard/README.md` and anything else naming `Cost { warm, cold }`. + standby floor Wildcard landed in #4403. Update anything naming + `Cost { warm, cold }`. - Replace `Cost { warm, cold }` in `rs/moq-net/src/model/origin.rs` and its JS mirror, and delete what computes Warm from cache state. - The moq-transport cluster extension already carries one cost. diff --git a/quest/m1/route-wakes.md b/quest/m1/route-wakes.md index d73d2c2576..b573b29c11 100644 --- a/quest/m1/route-wakes.md +++ b/quest/m1/route-wakes.md @@ -69,7 +69,5 @@ Public API: none. Wire: none. ## Related -- [Wildcard](/quest/m0/wildcard/README.md) - landed (#4403) the `sync_route`, `poke_below`, and `origin/pool_churn` bench this reworks - - [Front deadlines](/quest/m1/front-deadline-index.md) - each spurious wake also pays that per-track poll and deadline scan - [Front parking](/quest/m1/origin-front-parks.md) - replaces the `routed_broadcast` watch loop this wakes diff --git a/quest/m1/transcode-fetch-demand.md b/quest/m1/transcode-fetch-demand.md index 1d79610c83..5334380fed 100644 --- a/quest/m1/transcode-fetch-demand.md +++ b/quest/m1/transcode-fetch-demand.md @@ -19,7 +19,3 @@ Build on #4812, which landed first and made the same `fetch()` refuse a mid-group start before it fetches the source. Public API: none. Wire: none. - -## Related - -- [Transcoders start at group boundaries](/quest/m0/wildcard/transcode-group-start.md) - #4812 changed the same `fetch()` diff --git a/quest/m2/announce-shapes.md b/quest/m2/announce-shapes.md index 4e1953d52d..a21ec92ad9 100644 --- a/quest/m2/announce-shapes.md +++ b/quest/m2/announce-shapes.md @@ -100,11 +100,10 @@ routes independently of Rust. The slopes decide between a reversed-segment index and dropping suffix shapes from the quest. A shaped advertisement is authorized against the sender's publish claim, as -[Wildcard](/quest/m0/wildcard/README.md) does for prefix advertisements today +Wildcard (#4403) does for prefix advertisements today (decided in the 2026-10-06 audit). Token patterns (`moq-pattern`, `moq_auth::Claims`) already match suffixes, so no auth change is needed. ## Related -- [Wildcard](/quest/m0/wildcard/README.md) - prefix-only advertisements and the service-prefix layout this extends - [Cluster routing](/quest/m1/cluster-routing/README.md) - forwards announcements between relays, which must keep their shape diff --git a/rs/moq-boy/src/main.rs b/rs/moq-boy/src/main.rs index df54bf2baa..94e2bf3aa4 100644 --- a/rs/moq-boy/src/main.rs +++ b/rs/moq-boy/src/main.rs @@ -237,7 +237,7 @@ async fn run(config: &Config) -> Result<()> { .create_broadcast(&broadcast_path) .context("failed to create broadcast")?; broadcast - .announce(Default::default()) + .announce(moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint())) .context("failed to announce broadcast")?; // Consume origin: viewer broadcasts under the viewer prefix. diff --git a/rs/moq-cli/src/archive.rs b/rs/moq-cli/src/archive.rs index 2863a7f403..6aa0653993 100644 --- a/rs/moq-cli/src/archive.rs +++ b/rs/moq-cli/src/archive.rs @@ -119,7 +119,7 @@ pub async fn import(origin: moq_net::origin::Producer, name: String, args: Impor .with_context(|| format!("no readable recording at {}", args.store))?; let serve = reader.serve(); broadcast - .announce(Default::default()) + .announce(moq_tokio::moq_net::origin::Route::default().with_epoch(moq_tokio::moq_net::Epoch::mint())) .context("failed to announce broadcast")?; tracing::info!(%name, store = %args.store, "replaying"); diff --git a/rs/moq-cli/src/hls.rs b/rs/moq-cli/src/hls.rs index 6a50170c84..9d754ef295 100644 --- a/rs/moq-cli/src/hls.rs +++ b/rs/moq-cli/src/hls.rs @@ -60,7 +60,7 @@ pub async fn import(target: ImportTarget, playlist: String) -> anyhow::Result<() .with_bandwidth(bandwidth); let catalog = moq_mux::catalog::Producer::new(&mut producer, config)?; producer - .announce(Default::default()) + .announce(moq_tokio::moq_net::origin::Route::default().with_epoch(moq_tokio::moq_net::Epoch::mint())) .context("failed to announce broadcast")?; let playlist = playlist_url(&playlist)?; diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index c6192604a9..0a851a3830 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -400,7 +400,7 @@ impl Publish { return Ok(()); }; broadcast - .announce(Default::default()) + .announce(moq_tokio::moq_net::origin::Route::default().with_epoch(moq_tokio::moq_net::Epoch::mint())) .context("failed to announce broadcast") } diff --git a/rs/moq-cli/src/rtc.rs b/rs/moq-cli/src/rtc.rs index 8449a01890..d9e55666e0 100644 --- a/rs/moq-cli/src/rtc.rs +++ b/rs/moq-cli/src/rtc.rs @@ -97,7 +97,7 @@ pub async fn connect_import(target: ImportTarget, url: Url) -> anyhow::Result<() // The WHEP pull fills the tracks as they arrive; announce up front so viewers // can discover the broadcast while it connects. producer - .announce(Default::default()) + .announce(moq_tokio::moq_net::origin::Route::default().with_epoch(moq_tokio::moq_net::Epoch::mint())) .context("failed to announce broadcast")?; tracing::info!(url = %RedactedUrl::new(&url), %name, "WHEP client pulling"); diff --git a/rs/moq-cli/src/transcode.rs b/rs/moq-cli/src/transcode.rs index d13f36e3e3..dc25e4cabe 100644 --- a/rs/moq-cli/src/transcode.rs +++ b/rs/moq-cli/src/transcode.rs @@ -166,7 +166,7 @@ pub async fn run(moq: MoqSide, args: Args, net: Net) -> anyhow::Result<()> { .create_broadcast(&output_path) .context("failed to create the derivative broadcast")?; output - .announce(Default::default()) + .announce(moq_tokio::moq_net::origin::Route::default().with_epoch(moq_tokio::moq_net::Epoch::mint())) .context("failed to announce the derivative broadcast")?; tracing::info!(source = %source_path, output = %output_path, "transcoding"); diff --git a/rs/moq-e2ee/README.md b/rs/moq-e2ee/README.md index c2cf2fa5b8..c6c85d68b2 100644 --- a/rs/moq-e2ee/README.md +++ b/rs/moq-e2ee/README.md @@ -23,13 +23,14 @@ authenticated channel. Every publisher instance mints a shared `moq_net::Epoch` ```rust let credential = Credential::new(credential::Config { context, kid, secret })?; let generation = credential.generation(moq_net::Epoch::mint()); -let path = credential.path("meeting.hang")?.join_epoch(Some(generation.epoch())); +let path = credential.path("meeting.hang")?; let name = generation.name("video")?; let producer = generation.produce(broadcast.create_track(name.as_str(), None)?)?; +broadcast.announce(Route::default().with_epoch(generation.epoch().clone()))?; ``` -A subscriber discovers instances under `credential.path(semantic)`, takes the greatest -epoch from the last `@` path segment using `Path::split_epoch`, binds the same generation, and calls +A subscriber discovers the broadcast at `credential.path(semantic)`, takes the epoch from the +announced `Route::epoch` (or from the application's channel on a wire older than lite-07), binds the same generation, and calls `generation.consume(subscriber)` on a track whose name it derived or read from the decrypted catalog. Nothing survives a publisher instance: a restart mints a new epoch. diff --git a/rs/moq-e2ee/src/credential.rs b/rs/moq-e2ee/src/credential.rs index 581f9ddbd9..e424b1f90f 100644 --- a/rs/moq-e2ee/src/credential.rs +++ b/rs/moq-e2ee/src/credential.rs @@ -77,7 +77,7 @@ impl Credential { self.0.kid } - /// The opaque broadcast path for a semantic broadcast name; instances publish at `/`. + /// The opaque broadcast path for a semantic broadcast name; each instance announces its epoch on the route. /// /// # Errors /// diff --git a/rs/moq-e2ee/src/tests.rs b/rs/moq-e2ee/src/tests.rs index 072c6f573a..5b54a14413 100644 --- a/rs/moq-e2ee/src/tests.rs +++ b/rs/moq-e2ee/src/tests.rs @@ -339,15 +339,10 @@ fn datagram_ciphertext(generation: &Generation, name: &Name, sequence: u64, plai } #[test] -fn path_joins_epoch() { - let cred = test_credential(); - let epoch = Epoch::mint(); - let path = cred.path("meeting.hang").unwrap(); +fn path_is_opaque() { + let path = test_credential().path("meeting.hang").unwrap(); assert_eq!(path.as_str().len(), 22); - let full = path.join_epoch(Some(&epoch)); - let (opaque, parsed) = full.split_epoch(); - assert_eq!(opaque, path); - assert_eq!(parsed.unwrap(), epoch); + assert!(!path.as_str().contains("meeting")); } #[test] diff --git a/rs/moq-e2ee/tests/transport.rs b/rs/moq-e2ee/tests/transport.rs index 42fb1f0886..f2a6b58fb9 100644 --- a/rs/moq-e2ee/tests/transport.rs +++ b/rs/moq-e2ee/tests/transport.rs @@ -33,7 +33,7 @@ async fn connect_protected(version: Version, track: &str) -> Fixture { }) .unwrap(); let generation = cred.generation(Epoch::mint()); - let path = cred.path("meeting.hang").unwrap().join_epoch(Some(generation.epoch())); + let path = cred.path("meeting.hang").unwrap(); let name = generation.name(track).unwrap(); let publisher = produce_origin(1); @@ -41,18 +41,27 @@ async fn connect_protected(version: Version, track: &str) -> Fixture { let broadcast = publisher.create_broadcast(&path).unwrap(); let net = broadcast.create_track(name.as_str(), None).unwrap(); - broadcast.announce(Default::default()).unwrap(); + broadcast + .announce(moq_net::origin::Route::default().with_epoch(generation.epoch().clone())) + .unwrap(); let mut options = MockConnectOptions::new(version); options.server_publish = Some(publisher); options.client_subscribe = Some(consumer_origin.clone()); let pair = connect_mock(options).await; - // The subscriber knows the opaque prefix and takes the epoch from the discovered path. + // The subscriber knows the opaque path and takes the epoch from the announced route. + // A wire without epochs leaves it to the application's own channel. let consumer = consumer_origin.consume(); - consumer.routed(&path).await.unwrap(); - let (_, epoch) = path.split_epoch(); - let epoch = epoch.unwrap(); + let route = consumer.routed(&path).await.unwrap(); + if version.to_string().starts_with("moq-lite-07") { + assert_eq!( + route.epoch.as_ref(), + Some(generation.epoch()), + "lite-07 announces the epoch" + ); + } + let epoch = route.epoch.unwrap_or_else(|| generation.epoch().clone()); let generation = cred.generation(epoch); let remote = consumer.request_broadcast(&path).await.unwrap(); let subscriber = remote @@ -75,7 +84,7 @@ async fn connect_protected(version: Version, track: &str) -> Fixture { #[tokio::test] async fn grouped_over_lite() { tokio::time::timeout(TEST_TIMEOUT, async { - let mut fixture = connect_protected("moq-lite-05".parse().unwrap(), "video").await; + let mut fixture = connect_protected("moq-lite-07-wip".parse().unwrap(), "video").await; let mut group = fixture.producer.append_group().unwrap(); group .write_frame(Timestamp::from_millis(1).unwrap(), b"lite-frame") diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index 9d99686ce9..580a1446c2 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -1645,12 +1645,17 @@ mod tests { #[tokio::test(start_paused = true)] async fn truncated_resumed_group_skips_to_the_next_clean_group() { + // One publisher instance behind every route, so each may resume the others. + let epoch = moq_net::Epoch::mint(); let origin = crate::source::produce_origin(); let hops = moq_net::Hops::try_from(vec![moq_net::Hop::new(10).unwrap()]).unwrap(); let first_route = origin .dynamic( "live", - moq_net::origin::Route::default().with_hops(hops.clone()).with_cost(5), + moq_net::origin::Route::default() + .with_epoch(epoch.clone()) + .with_hops(hops.clone()) + .with_cost(5), ) .unwrap(); let pending = origin.consume().request_broadcast("live"); @@ -1692,6 +1697,7 @@ mod tests { .dynamic( "live", moq_net::origin::Route::default() + .with_epoch(epoch.clone()) .with_hops(hops.clone()) .with_cost(5 - sequence), ) diff --git a/rs/moq-mux/src/container/ts/programs.rs b/rs/moq-mux/src/container/ts/programs.rs index 7489c4b308..c05915bce5 100644 --- a/rs/moq-mux/src/container/ts/programs.rs +++ b/rs/moq-mux/src/container/ts/programs.rs @@ -75,8 +75,10 @@ impl Programs { // Seeded with the scanner's PAT rather than replaying its packets, so every section // reaches the importer and nothing the scanner counted is counted again. import.handle_pat(&pat)?; + // Each program is its own publisher instance for this run, so a restart + // replaces it rather than resuming into it. broadcast - .announce(Default::default()) + .announce(moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint())) .with_context(|| format!("failed to announce broadcast {name}"))?; self.programs.push(Program { _broadcast: broadcast, @@ -290,6 +292,24 @@ mod test { assert_eq!(renditions(&origin, "event/2.hang").await, 1); } + /// Each program announces as its own publisher instance for the run, so a restart + /// replaces it rather than resuming into it. + #[test] + fn every_program_mints_its_own_epoch() { + let origin = crate::source::produce_origin(); + let mut programs = Programs::new(origin, "event.hang", catalog::Config::default()); + programs.decode(&two_section_pat()).unwrap(); + programs.finish().unwrap(); + let epochs: Vec<_> = programs + .programs + .iter() + .map(|program| program._broadcast.route().and_then(|route| route.epoch)) + .collect(); + assert_eq!(epochs.len(), 2); + assert!(epochs.iter().all(Option::is_some), "every program names its instance"); + assert_ne!(epochs[0], epochs[1]); + } + /// A partial PAT holds only itself, not the multiplex that follows it. #[test] fn a_partial_pat_does_not_hold_the_multiplex() { diff --git a/rs/moq-net/src/epoch.json b/rs/moq-net/src/epoch.json new file mode 100644 index 0000000000..baa8065bf8 --- /dev/null +++ b/rs/moq-net/src/epoch.json @@ -0,0 +1,48 @@ +{ + "valid": [ + { + "text": "00000000-0000-7000-8000-000000000000", + "unix_ms": 0 + }, + { + "text": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", + "unix_ms": 1759727860778 + }, + { + "text": "0199b7f4-3c2a-7d1e-af0b-2b6c1a9d8e7f", + "unix_ms": 1759727860778 + }, + { + "text": "0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a", + "unix_ms": 1759727885840 + }, + { + "text": "ffffffff-ffff-7fff-bfff-ffffffffffff", + "unix_ms": 281474976710655 + } + ], + "invalid": [ + "", + "alice", + "2026/09", + "0199B7F4-3C2A-7D1E-9F0B-2B6C1A9D8E7F", + "0199b7f43c2a7d1e9f0b2b6c1a9d8e7f", + "@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", + "{0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f}", + "urn:uuid:0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-4d1e-9f0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-7d1e-cf0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-7d1e-7f0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f\n", + " 0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f ", + "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7g" + ], + "ordered": [ + "00000000-0000-7000-8000-000000000000", + "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", + "0199b7f4-3c2a-7d1e-af0b-2b6c1a9d8e7f", + "0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a", + "ffffffff-ffff-7fff-bfff-ffffffffffff" + ] +} diff --git a/rs/moq-net/src/epoch.rs b/rs/moq-net/src/epoch.rs index 39b64ea638..f47b2d1117 100644 --- a/rs/moq-net/src/epoch.rs +++ b/rs/moq-net/src/epoch.rs @@ -19,7 +19,7 @@ impl Epoch { Self(uuid::Uuid::now_v7().hyphenated().to_string().into()) } - /// The UUID text, without the path segment's `@` marker. + /// The UUID text. pub fn as_str(&self) -> &str { &self.0 } @@ -33,6 +33,20 @@ impl Epoch { } } +impl Epoch { + /// The UUID's 16 bytes, as the wire carries them. + pub(crate) fn to_bytes(&self) -> [u8; 16] { + // Parsing enforces the canonical form, so this cannot fail. + uuid::Uuid::parse_str(&self.0).unwrap().into_bytes() + } + + /// Parse the wire's 16 bytes, refusing anything but a UUIDv7 with the RFC variant. + pub(crate) fn from_bytes(bytes: &[u8]) -> Result { + let uuid = uuid::Uuid::from_slice(bytes).map_err(|_| InvalidEpoch)?; + uuid.hyphenated().to_string().parse() + } +} + impl FromStr for Epoch { type Err = InvalidEpoch; @@ -76,4 +90,37 @@ mod tests { assert_eq!(epoch.as_str().parse::().unwrap(), epoch); assert!(Epoch::mint() > epoch); } + + /// Shared with `js/net/src/epoch.test.ts`, so both parse and order alike. + #[test] + fn epoch_vectors() { + let vectors: serde_json::Value = serde_json::from_str(include_str!("epoch.json")).unwrap(); + for row in vectors["valid"].as_array().unwrap() { + let text = row["text"].as_str().unwrap(); + let epoch: Epoch = text.parse().unwrap(); + assert_eq!(epoch.as_str(), text); + assert_eq!( + epoch.time().duration_since(UNIX_EPOCH).unwrap().as_millis(), + row["unix_ms"].as_u64().unwrap() as u128 + ); + } + for row in vectors["invalid"].as_array().unwrap() { + assert!(row.as_str().unwrap().parse::().is_err(), "{row}"); + } + let ordered: Vec = vectors["ordered"] + .as_array() + .unwrap() + .iter() + .map(|row| row.as_str().unwrap().parse().unwrap()) + .collect(); + assert!(ordered.windows(2).all(|pair| pair[0] < pair[1])); + } + + #[test] + fn bytes_roundtrip_and_refuse_other_versions() { + let epoch = Epoch::mint(); + assert_eq!(Epoch::from_bytes(&epoch.to_bytes()).unwrap(), epoch); + assert!(Epoch::from_bytes(uuid::Builder::from_random_bytes([7; 16]).into_uuid().as_bytes()).is_err()); + assert!(Epoch::from_bytes(&epoch.to_bytes()[..15]).is_err()); + } } diff --git a/rs/moq-net/src/fuzz.rs b/rs/moq-net/src/fuzz.rs index cecdfd5b1d..e2a75b59dd 100644 --- a/rs/moq-net/src/fuzz.rs +++ b/rs/moq-net/src/fuzz.rs @@ -198,6 +198,7 @@ impl AnnounceWriter { Announced::Start(suffix, hops) => { let (_, suffix, hops) = self.encoder.start(suffix.clone(), hops.clone()); lite::AnnounceBroadcast::Active { + epoch: None, suffix, hops, cost: Default::default(), @@ -325,6 +326,7 @@ impl LiteSample { .encode(w, version) .unwrap(), Self::Subscribe => lite::Subscribe { + epoch: None, id: 3, broadcast: Path::new("room/alice"), track: "video".into(), @@ -508,6 +510,7 @@ impl Default for Messages { Self { lite_subscribe: lite::Subscribe { + epoch: None, id: 7, broadcast: Path::new("room/alice"), track: "video".into(), diff --git a/rs/moq-net/src/lite/announce.rs b/rs/moq-net/src/lite/announce.rs index 4f7177ed6f..7bfbf9578a 100644 --- a/rs/moq-net/src/lite/announce.rs +++ b/rs/moq-net/src/lite/announce.rs @@ -1,6 +1,6 @@ use num_enum::{IntoPrimitive, TryFromPrimitive}; -use crate::{Hop, Hops, Path, coding::*, origin::Cost}; +use crate::{Epoch, Hop, Hops, Path, coding::*, origin::Cost}; use super::{Message, Version, message::decode_size}; @@ -41,9 +41,11 @@ pub fn restart_supported(version: Version) -> bool { pub enum AnnounceBroadcast<'a> { /// ANNOUNCE_START (lite-06) / active (older): a broadcast is now available. /// Carries the path suffix, the hop chain, and (lite-06+) the warm and cold - /// route costs, and assigns the next announce id. + /// route costs, and assigns the next announce id. The epoch (lite-07+) is fixed + /// for the announcement's lifetime: a new one is an end and a fresh start. Active { suffix: PathRef<'a>, + epoch: Option, hops: HopsRef, cost: Cost, }, @@ -165,12 +167,18 @@ impl AnnounceBroadcast<'_> { #[cfg(test)] pub fn into_owned(self) -> AnnounceBroadcast<'static> { match self { - Self::Active { suffix, hops, cost } => AnnounceBroadcast::Active { + Self::Active { + suffix, + epoch, + hops, + cost, + } => AnnounceBroadcast::Active { suffix: PathRef { base: suffix.base, keep: suffix.keep, rest: suffix.rest.into_owned(), }, + epoch, hops, cost, }, @@ -228,8 +236,14 @@ impl Encode for AnnounceBroadcast<'_> { let prefix = w.prefix_varint(); match self { - Self::Active { suffix, hops, cost } => { + Self::Active { + suffix, + epoch, + hops, + cost, + } => { suffix.encode(w, version)?; + super::epoch::encode_epoch(w, version, epoch.as_ref())?; hops.encode(w, version)?; cost.encode(w, version)?; } @@ -282,6 +296,7 @@ impl Decode for AnnounceBroadcast<'_> { let msg = match typ { ANNOUNCE_START => Self::Active { suffix: PathRef::decode(&mut body, version)?, + epoch: super::epoch::decode_epoch(&mut body, version)?, hops: HopsRef::decode(&mut body, version)?, cost: Cost::decode(&mut body, version)?, }, @@ -339,6 +354,7 @@ impl AnnounceBroadcast<'_> { Ok(match status { AnnounceStatus::Active => Self::Active { suffix: PathRef::literal(suffix), + epoch: None, hops: HopsRef::literal(hops), cost: Cost::UNKNOWN, }, @@ -350,6 +366,7 @@ impl AnnounceBroadcast<'_> { // invalid value there. AnnounceStatus::Restart if restart_supported(version) => Self::Active { suffix: PathRef::literal(suffix), + epoch: None, hops: HopsRef::literal(hops), cost: Cost::UNKNOWN, }, @@ -534,6 +551,7 @@ mod tests { // Encode a normal Active, then flip its status byte (1 -> 2). let mut buf = Vec::new(); AnnounceBroadcast::Active { + epoch: None, suffix: PathRef::literal(Path::new("foo/bar")), hops: HopsRef::default(), cost: Cost::default(), @@ -624,6 +642,7 @@ mod tests { let mut hops = Hops::new(); hops.push(Hop::new(7).unwrap()).unwrap(); let msg = AnnounceBroadcast::Active { + epoch: None, suffix: PathRef::literal(Path::new("room/cam")), hops: HopsRef::literal(hops.clone()), cost: Cost::UNKNOWN, @@ -647,6 +666,7 @@ mod tests { let cost = Cost { warm: 12, cold: 30 }; let active = AnnounceBroadcast::Active { + epoch: None, suffix: PathRef::literal(Path::new("room/cam")), hops: HopsRef::literal(hops.clone()), cost, @@ -672,6 +692,7 @@ mod tests { let cost = Cost { warm: 12, cold: 30 }; let active = AnnounceBroadcast::Active { + epoch: None, suffix: PathRef { base: 2, keep: 3, @@ -702,8 +723,8 @@ mod tests { #[test] fn a_keep_without_a_base_is_rejected() { for (path_keep, hop_keep) in [(1u8, 0u8), (0, 1)] { - // Path base, path keep, empty rest, hop base, no hops, hop keep, cost. - let body = [0, path_keep, 0, 0, 0, hop_keep, 0, 0]; + // Path base, path keep, empty rest, no epoch, hop base, no hops, hop keep, cost. + let body = [0, path_keep, 0, 0, 0, 0, hop_keep, 0, 0]; let mut buf = vec![ANNOUNCE_START as u8, body.len() as u8]; buf.extend_from_slice(&body); assert!(matches!( @@ -717,6 +738,7 @@ mod tests { #[test] fn a_base_needs_lite07() { let msg = AnnounceBroadcast::Active { + epoch: None, suffix: PathRef { base: 1, keep: 1, @@ -769,6 +791,7 @@ mod tests { #[test] fn route_cost_is_dropped_before_lite06() { let msg = AnnounceBroadcast::Active { + epoch: None, suffix: PathRef::literal(Path::new("room/cam")), hops: HopsRef::default(), cost: Cost { warm: 9, cold: 9 }, @@ -777,6 +800,7 @@ mod tests { assert_eq!( got, AnnounceBroadcast::Active { + epoch: None, suffix: PathRef::literal(Path::new("room/cam")), hops: HopsRef::default(), cost: Cost::UNKNOWN, diff --git a/rs/moq-net/src/lite/compress.rs b/rs/moq-net/src/lite/compress.rs index 1a5c9add0e..257323bd52 100644 --- a/rs/moq-net/src/lite/compress.rs +++ b/rs/moq-net/src/lite/compress.rs @@ -591,7 +591,7 @@ mod tests { /// A compressed stream pinned byte for byte: `js/net/src/lite/announce.test.ts` /// decodes the same hex, so a codec change on either side breaks both. - const GOLDEN: &str = "001600000a726f6f6d2f612f63616d00029111a222000000000d0102036d69630101b3330100000209000101c04444010000010101000c020101620201c05555010000"; + const GOLDEN: &str = "001700000a726f6f6d2f612f63616d0000029111a222000000000e0102036d6963000101b3330100000209000101c04444010000010101000d02010162000201c05555010000"; fn golden() -> Vec { use crate::fuzz::Announced; @@ -614,7 +614,7 @@ mod tests { /// The literal lite-07 stream `js/net` writes for the same announcements, which /// never picks a base. - const JS_LITERAL: &str = "001600000a726f6f6d2f612f63616d00029111a222000000001600000a726f6f6d2f612f6d69630002b333a222000000020b000002c04444a2220000000101010013000006726f6f6d2f620002c05555a222000000"; + const JS_LITERAL: &str = "001700000a726f6f6d2f612f63616d0000029111a222000000001700000a726f6f6d2f612f6d6963000002b333a222000000020b000002c04444a2220000000101010014000006726f6f6d2f62000002c05555a222000000"; #[test] fn js_literal_stream_decodes() { diff --git a/rs/moq-net/src/lite/epoch.rs b/rs/moq-net/src/lite/epoch.rs new file mode 100644 index 0000000000..3bbd083800 --- /dev/null +++ b/rs/moq-net/src/lite/epoch.rs @@ -0,0 +1,68 @@ +use crate::{ + Epoch, + coding::{DecodeError, Decoder, EncodeError, Encoder}, +}; + +use super::Version; + +/// Encode an optional epoch as its 16 bytes, or empty for none. Older versions +/// carry nothing, so the peer sees a route or request of unknown identity. +pub(super) fn encode_epoch(w: &mut Encoder<'_>, version: Version, epoch: Option<&Epoch>) -> Result<(), EncodeError> { + if !version.has_epoch() { + return Ok(()); + } + match epoch { + Some(epoch) => w.bytes(&epoch.to_bytes()), + None => w.bytes(&[]), + } +} + +/// Decode an optional epoch: empty is none, anything but a UUIDv7 is refused. +pub(super) fn decode_epoch(r: &mut Decoder<'_>, version: Version) -> Result, DecodeError> { + if !version.has_epoch() { + return Ok(None); + } + match r.bytes()? { + [] => Ok(None), + bytes => Epoch::from_bytes(bytes) + .map(Some) + .map_err(|_| DecodeError::InvalidValue), + } +} + +#[cfg(test)] +mod test { + use super::*; + + fn roundtrip(version: Version, epoch: Option<&Epoch>) -> Option { + let mut buf = Vec::new(); + encode_epoch(&mut Encoder::new(&mut buf, version.into()), version, epoch).unwrap(); + let mut decoder = Decoder::new(&buf, version.into()); + let decoded = decode_epoch(&mut decoder, version).unwrap(); + assert!(decoder.is_empty()); + decoded + } + + #[test] + fn lite07_carries_the_epoch() { + let epoch = Epoch::mint(); + assert_eq!(roundtrip(Version::Lite07, Some(&epoch)), Some(epoch)); + assert_eq!(roundtrip(Version::Lite07, None), None); + } + + #[test] + fn older_versions_drop_it() { + let epoch = Epoch::mint(); + assert_eq!(roundtrip(Version::Lite06, Some(&epoch)), None); + } + + #[test] + fn refuses_a_non_v7_uuid() { + let mut buf = Vec::new(); + Encoder::new(&mut buf, Version::Lite07.into()) + .bytes(uuid::Builder::from_random_bytes([7; 16]).into_uuid().as_bytes()) + .unwrap(); + let mut decoder = Decoder::new(&buf, Version::Lite07.into()); + assert!(decode_epoch(&mut decoder, Version::Lite07).is_err()); + } +} diff --git a/rs/moq-net/src/lite/fetch.rs b/rs/moq-net/src/lite/fetch.rs index 7bb16636c6..48d7bd0ab5 100644 --- a/rs/moq-net/src/lite/fetch.rs +++ b/rs/moq-net/src/lite/fetch.rs @@ -13,6 +13,9 @@ use super::{Message, Version}; #[derive(Clone, Debug)] pub struct Fetch<'a> { pub broadcast: Path<'a>, + /// The publisher instance the subscriber expects; see [`crate::origin::Route::epoch`]. + /// Lite07+ only. + pub epoch: Option, pub track: Cow<'a, str>, pub priority: u8, pub group: u64, @@ -33,6 +36,7 @@ impl Message for Fetch<'_> { } let broadcast = Path::decode(r, version)?; + let epoch = super::epoch::decode_epoch(r, version)?; let track = Cow::Owned(r.string()?); let priority = r.u8()?; let group = r.varint()?; @@ -48,6 +52,7 @@ impl Message for Fetch<'_> { Ok(Self { broadcast, + epoch, track, priority, group, @@ -65,6 +70,7 @@ impl Message for Fetch<'_> { } self.broadcast.encode(w, version)?; + super::epoch::encode_epoch(w, version, self.epoch.as_ref())?; w.string(&self.track)?; w.u8(self.priority); w.varint(self.group)?; @@ -87,6 +93,7 @@ mod test { fn fetch_sample() -> Fetch<'static> { Fetch { + epoch: None, broadcast: Path::new("room").to_owned(), track: Cow::Borrowed("video"), priority: 3, diff --git a/rs/moq-net/src/lite/mod.rs b/rs/moq-net/src/lite/mod.rs index 566f2c5b8a..cff9345d99 100644 --- a/rs/moq-net/src/lite/mod.rs +++ b/rs/moq-net/src/lite/mod.rs @@ -7,6 +7,7 @@ mod announce; mod compress; mod datagram; +mod epoch; mod fetch; mod goaway; mod group; diff --git a/rs/moq-net/src/lite/publisher.rs b/rs/moq-net/src/lite/publisher.rs index 5d6e30b840..8e05c0efd7 100644 --- a/rs/moq-net/src/lite/publisher.rs +++ b/rs/moq-net/src/lite/publisher.rs @@ -704,10 +704,14 @@ impl AnnounceRun { suffix: crate::PathOwned, hops: Hops, cost: crate::origin::Cost, + // Fixed while the peer holds the advertisement: the origin delivers a new + // epoch as a retraction and a fresh start. + epoch: Option, ) -> Result<(), Error> { let (id, wire, chain) = self.encoder.start(suffix.clone(), hops.clone()); self.live.insert(suffix, Advertised { id, hops, cost }); stream.writer.buffer(&lite::AnnounceBroadcast::Active { + epoch, suffix: wire, hops: chain, cost, @@ -792,7 +796,7 @@ impl AnnounceRun { // stashing suffix+hops so we can both COUNT them for AnnounceOk and re-send // them afterward. The receiver stamps our origin onto each hop chain, so we // forward the stored chain as-is (no self push here). - let mut initial: Vec<(crate::PathOwned, Hops, crate::origin::Cost)> = Vec::new(); + let mut initial: Vec<(crate::PathOwned, Hops, crate::origin::Cost, Option)> = Vec::new(); while let Some(event) = announced.try_next() { let (update, active) = match event { announce::Event::Start(update) | announce::Event::Update(update) => (update, true), @@ -810,7 +814,7 @@ impl AnnounceRun { }; tracing::debug!(route = %absolute, "announce"); initial.retain(|(s, ..)| s != &suffix); - initial.push((suffix, hops, cost)); + initial.push((suffix, hops, cost, update.route.epoch)); } else { // A potential race: a just-announced route already retracted. tracing::debug!(route = %absolute, "unannounce"); @@ -825,8 +829,8 @@ impl AnnounceRun { active: initial.len() as u64, }; stream.writer.buffer(&ok)?; - for (suffix, hops, cost) in initial { - self.start(stream, suffix, hops, cost)?; + for (suffix, hops, cost, epoch) in initial { + self.start(stream, suffix, hops, cost, epoch)?; } } _ => { @@ -933,6 +937,7 @@ impl AnnounceRun { } // lite-05: a duplicate ANNOUNCE, which assigns no id. None => stream.writer.buffer(&lite::AnnounceBroadcast::Active { + epoch: None, suffix: lite::PathRef::literal(suffix), hops: lite::HopsRef::literal(hops), cost, @@ -944,7 +949,7 @@ impl AnnounceRun { Some(_) => {} None => { tracing::debug!(route = %absolute, "announce"); - self.start(stream, suffix, hops, cost)?; + self.start(stream, suffix, hops, cost, update.route.epoch.clone())?; } }, // The chain must not be forwarded (reflected, or full): retract @@ -982,6 +987,9 @@ trait Request: Sized { /// The broadcast and track the request names. fn target(msg: &Self::Message) -> (&crate::Path<'static>, &str); + /// The publisher instance the request names, if any. + fn epoch(msg: &Self::Message) -> Option<&crate::Epoch>; + /// Start answering, once the broadcast resolves. fn start(shared: &Shared, msg: Self::Message, broadcast: crate::broadcast::Consumer) -> Result; @@ -1109,7 +1117,8 @@ impl> RequestServe { // served on demand by the route covering it (an `origin::Dynamic`), or // errors when there is none. let origin = ready!(self.shared.poll_serving_origin(waiter)); - *requesting = Some(origin.request_broadcast(R::target(msg).0).into_inner()); + let path = R::target(msg).0.clone(); + *requesting = Some(origin.request(path, R::epoch(msg)).into_inner()); } let broadcast = ready!(requesting.as_ref().expect("requesting").poll_ok(waiter))?; // Any placeholder but `Decode`, so a failed start is answered on the stream. @@ -1178,6 +1187,10 @@ impl Request for TrackInfoServe { (&msg.broadcast, &msg.track) } + fn epoch(msg: &Self::Message) -> Option<&crate::Epoch> { + msg.epoch.as_ref() + } + fn start(_: &Shared, msg: Self::Message, broadcast: crate::broadcast::Consumer) -> Result { let querying = broadcast.track(&msg.track)?.query().into_inner(); Ok(Self { querying }) @@ -1212,7 +1225,8 @@ impl Request for TrackInfoServe { enum SubscribeServe { /// Waiting for the model subscription to be confirmed. Confirm { - msg: lite::Subscribe<'static>, + /// Boxed: the request is far larger than the other states' handles. + msg: Box>, subscribing: track::Subscribing, /// The newest group when the SUBSCRIBE arrived; see [`position_cursor`]. latest: Option, @@ -1236,6 +1250,10 @@ impl Request for SubscribeServe { (&msg.broadcast, &msg.track) } + fn epoch(msg: &Self::Message) -> Option<&crate::Epoch> { + msg.epoch.as_ref() + } + fn start(shared: &Shared, msg: Self::Message, broadcast: crate::broadcast::Consumer) -> Result { let subscription = crate::track::Subscription { priority: msg.priority, @@ -1250,7 +1268,7 @@ impl Request for SubscribeServe { let latest = track.latest(); let subscribing = track.subscribe(subscription).into_inner(); Ok(Self::Confirm { - msg, + msg: Box::new(msg), subscribing, latest, update: None, @@ -1331,7 +1349,7 @@ impl Request for SubscribeServe { opens: Default::default(), }; - let bounds = Bounds::from(&msg); + let bounds = Bounds::from(msg.as_ref()); position_cursor(&mut track, shared.version, bounds.start_group, latest); let mut run = TrackRun::new(sub, track, bounds, track_priority_tx); if let Some(update) = update { @@ -1393,6 +1411,10 @@ impl Request for FetchServe { (&msg.broadcast, &msg.track) } + fn epoch(msg: &Self::Message) -> Option<&crate::Epoch> { + msg.epoch.as_ref() + } + fn start(_: &Shared, msg: Self::Message, broadcast: crate::broadcast::Consumer) -> Result { let fetching = broadcast .track(&msg.track)? @@ -1823,7 +1845,7 @@ mod announce_test { let mut wire = Wire { writes, cursor: 0 }; assert_eq!(wire.take_ok().active, 1, "expected one initial announce"); match wire.take_announces().as_slice() { - [lite::AnnounceBroadcast::Active { suffix, hops, cost }] => { + [lite::AnnounceBroadcast::Active { suffix, hops, cost, .. }] => { assert_eq!(suffix.rest.as_str(), "cam"); assert_eq!(hops, &lite::HopsRef::literal(pub_hops())); assert_eq!(*cost, crate::origin::Cost::new(7)); @@ -4020,12 +4042,14 @@ mod tests { for wait in [Wait::Broadcast, Wait::Track] { for close in [Close::Fin, Close::Reset] { let track = lite::Track { + epoch: None, broadcast: broadcast.clone(), track: "video".into(), }; leave_request::(version, &track, wait, close).await; let subscribe = lite::Subscribe { + epoch: None, id: 0, broadcast: broadcast.clone(), track: "video".into(), @@ -4039,6 +4063,7 @@ mod tests { leave_request::>(version, &subscribe, wait, close).await; let fetch = lite::Fetch { + epoch: None, broadcast: broadcast.clone(), track: "video".into(), priority: 0, @@ -4077,6 +4102,7 @@ mod tests { let mut script = Vec::new(); lite::Subscribe { + epoch: None, id: 0, broadcast: crate::Path::new("room"), track: "video".into(), diff --git a/rs/moq-net/src/lite/subscribe.rs b/rs/moq-net/src/lite/subscribe.rs index 84ce0f3eee..b81cc2c4ea 100644 --- a/rs/moq-net/src/lite/subscribe.rs +++ b/rs/moq-net/src/lite/subscribe.rs @@ -14,6 +14,9 @@ use super::{Message, Version}; pub struct Subscribe<'a> { pub id: u64, pub broadcast: Path<'a>, + /// The publisher instance the subscriber expects; see [`crate::origin::Route::epoch`]. + /// Lite07+ only. + pub epoch: Option, pub track: Cow<'a, str>, pub priority: u8, pub max_age: std::time::Duration, @@ -47,6 +50,7 @@ impl Message for Subscribe<'_> { fn decode_msg(r: &mut Decoder<'_>, version: Version) -> Result { let id = r.varint()?; let broadcast = Path::decode(r, version)?; + let epoch = super::epoch::decode_epoch(r, version)?; let track = Cow::Owned(r.string()?); let priority = r.u8()?; @@ -67,6 +71,7 @@ impl Message for Subscribe<'_> { Ok(Self { id, broadcast, + epoch, track, priority, max_age, @@ -80,6 +85,7 @@ impl Message for Subscribe<'_> { fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { w.varint(self.id)?; self.broadcast.encode(w, version)?; + super::epoch::encode_epoch(w, version, self.epoch.as_ref())?; w.string(&self.track)?; w.u8(self.priority); @@ -706,6 +712,7 @@ mod test { fn subscribe_sample() -> Subscribe<'static> { Subscribe { + epoch: None, id: 1, broadcast: Path::new("room").to_owned(), track: Cow::Borrowed("video"), diff --git a/rs/moq-net/src/lite/subscriber.rs b/rs/moq-net/src/lite/subscriber.rs index 497cbc660b..dd1ba87316 100644 --- a/rs/moq-net/src/lite/subscriber.rs +++ b/rs/moq-net/src/lite/subscriber.rs @@ -86,7 +86,7 @@ pub(super) struct Subscriber { cost: Option, /// Sources created by the announce half, drained by the driver into /// [`SourceServe`] machines. - sources: kio::Queue<(PathOwned, crate::broadcast::Dynamic)>, + sources: kio::Queue<(PathOwned, Option, crate::broadcast::Dynamic)>, going_away: crate::goaway::GoingAway, /// What this session may allocate up front for frames still arriving. frames: frame::Budget, @@ -189,7 +189,12 @@ impl Subscriber { run: &mut PrefixRun, ) -> Result<(), Error> { match announce { - lite::AnnounceBroadcast::Active { suffix, hops, cost } => { + lite::AnnounceBroadcast::Active { + suffix, + epoch, + hops, + cost, + } => { let (suffix, hops) = match self.version.has_announce_id() { // Every `active` assigns the next ordinal, even ones we drop locally. true => run.decoder.start(suffix, hops)?, @@ -216,6 +221,7 @@ impl Subscriber { } else { self.start_announce( path, + epoch, hops, cost, run.link_cost, @@ -257,9 +263,12 @@ impl Subscriber { /// Records the advertisement either way. Returns `Ok(true)` if it was accepted /// and attached a route, or `Ok(false)` if it was declined locally. + #[allow(clippy::too_many_arguments)] fn start_announce( &mut self, path: PathOwned, + // The publisher instance the peer announced, fixed until it retracts. + epoch: Option, mut hops: crate::Hops, // The route cost off the wire, i.e. as the peer advertised it. // [`Cost::UNKNOWN`] before lite-06, leaving the hop chain as the only @@ -282,7 +291,7 @@ impl Subscriber { // The peer holds this prefix now. Everything below either accepts the announcement, // replacing this, or declines it and leaves it exactly as reserved. - announced.reserve(path.clone()); + announced.reserve(path.clone(), epoch); if let Some(responder) = responder_origin { // A chain already naming the sender came back through it: a reflection, and @@ -328,7 +337,7 @@ impl Subscriber { // resolve through this session on demand. An error means the prefix is // outside our scope, so don't serve it. Reflections are already // filtered above. - let route = self.announced_route(hops, cost, link_cost, responder_origin); + let route = self.announced_route(&path, hops, cost, link_cost, responder_origin, announced); let Ok(dynamic) = self.origin.dynamic(&path, route.clone()) else { return Ok(false); }; @@ -344,17 +353,25 @@ impl Subscriber { /// Once the peer has sent a GOAWAY every route it announces starts out draining, /// including a restart of one already attached: a connection on its way out must /// not win selection, however good the path it advertises looks. + /// + /// The epoch is the one the peer announced at `path`: a restart carries none, + /// since it never changes the content. fn announced_route( &self, + path: &PathOwned, hops: crate::Hops, cost: crate::origin::Cost, link_cost: u64, responder: Option, + announced: &Announced, ) -> crate::origin::Route { - let mut route = crate::origin::Route::default() - .with_hops(hops) - .with_cost(cost.charged(link_cost)) - .with_via(self.via(responder)); + let mut route = crate::origin::Route { + epoch: announced.epoch(path), + ..Default::default() + } + .with_hops(hops) + .with_cost(cost.charged(link_cost)) + .with_via(self.via(responder)); if self.going_away.is_set() { route.cost = crate::origin::Cost::DRAIN; @@ -420,7 +437,7 @@ impl Subscriber { } tracing::debug!(route = %self.log_path(&path), hops = hops.len(), "restart"); - let metadata = self.announced_route(hops, cost, link_cost, responder_origin); + let metadata = self.announced_route(&path, hops, cost, link_cost, responder_origin, announced); // A restart is a metadata update: the route keeps its prefix (and its // served paths) and re-prices in place. In-flight tracks keep flowing. @@ -590,9 +607,9 @@ impl SubscriberDriver { // Sources created by the announce half; their completion never ends the // session (the origin delivers the unannounce itself). - while let Poll::Ready(Ok((path, dynamic))) = self.subscriber.sources.poll_pop(waiter) { + while let Poll::Ready(Ok((path, epoch, dynamic))) = self.subscriber.sources.poll_pop(waiter) { self.sources - .push(SourceServe::new(self.subscriber.clone(), path, dynamic)); + .push(SourceServe::new(self.subscriber.clone(), path, epoch, dynamic)); } let _ = self.sources.poll(waiter); @@ -1367,6 +1384,7 @@ impl AnnouncePrefix { // model when this enters the origin via `create_broadcast`. self.subscriber.start_announce( path, + None, crate::Hops::new(), crate::origin::Cost::UNKNOWN, 0, @@ -1436,6 +1454,8 @@ impl AnnouncePrefix { struct SourceServe { subscriber: Subscriber, path: PathOwned, + /// The publisher instance the route announced, asked for on every request. + epoch: Option, dynamic: crate::broadcast::Dynamic, // A dedicated close-watch handle, since each pending operation needs its own. closed: S, @@ -1445,11 +1465,17 @@ struct SourceServe { } impl SourceServe { - fn new(subscriber: Subscriber, path: PathOwned, dynamic: crate::broadcast::Dynamic) -> Self { + fn new( + subscriber: Subscriber, + path: PathOwned, + epoch: Option, + dynamic: crate::broadcast::Dynamic, + ) -> Self { let closed = subscriber.session.clone(); Self { subscriber, path, + epoch, dynamic, closed, tracks: kio::Tasks::new(), @@ -1479,6 +1505,7 @@ impl kio::Task for SourceServe { let serve = TrackServe { subscriber: self.subscriber.clone(), path: self.path.clone(), + epoch: self.epoch.clone(), name: request.name().to_string(), }; // One machine per track serves its lone subscription and any number @@ -1729,6 +1756,7 @@ mod tests { let serve = TrackServe { subscriber, path: Path::new("room").to_owned(), + epoch: None, name: "video".to_string(), }; let broadcast = crate::broadcast::Info::new().produce(); @@ -1781,6 +1809,7 @@ mod tests { let serve = TrackServe { subscriber, path: Path::new("room").to_owned(), + epoch: None, name: "video".to_string(), }; let broadcast = crate::broadcast::Info::new().produce(); @@ -1949,6 +1978,7 @@ mod tests { TrackServe { subscriber: subscriber.clone(), path: Path::new("room").to_owned(), + epoch: None, name: "video".to_string(), }, request, @@ -2005,6 +2035,7 @@ mod tests { TrackServe { subscriber, path: Path::new("room").to_owned(), + epoch: None, name: "video".to_string(), }, request, @@ -2053,6 +2084,7 @@ mod tests { let serve = TrackServe { subscriber, path: Path::new("room/host").to_owned(), + epoch: None, name: "catalog.json".to_string(), }; @@ -2129,6 +2161,7 @@ mod tests { serve: TrackServe { subscriber, path: Path::new("room/host").to_owned(), + epoch: None, name: "catalog.json".to_string(), }, session, @@ -2557,6 +2590,7 @@ mod tests { subscriber .start_announce( path.clone(), + None, crate::Hops::new(), crate::origin::Cost::default(), 0, @@ -2573,6 +2607,7 @@ mod tests { matches!( subscriber.start_announce( path.clone(), + None, reflected, crate::origin::Cost::default(), 0, @@ -2609,6 +2644,7 @@ mod tests { subscriber .start_announce( Path::new(&format!("room/{i}")).to_owned(), + None, crate::Hops::new(), crate::origin::Cost::default(), 0, @@ -2668,6 +2704,7 @@ mod tests { !subscriber .start_announce( path.clone(), + None, hops, crate::origin::Cost::default(), 0, @@ -2685,6 +2722,7 @@ mod tests { matches!( subscriber.start_announce( path.clone(), + None, fresh, crate::origin::Cost::default(), 0, @@ -2723,6 +2761,7 @@ mod tests { !subscriber .start_announce( path.clone(), + None, reflected, crate::origin::Cost::default(), 0, @@ -2739,6 +2778,7 @@ mod tests { let err = subscriber .start_announce( path.clone(), + None, hops, crate::origin::Cost::default(), 0, @@ -2779,6 +2819,7 @@ mod tests { let accepted = subscriber .start_announce( Path::new("room/host").to_owned(), + None, hops, crate::origin::Cost::default(), 0, @@ -2843,6 +2884,7 @@ mod tests { let accepted = subscriber .start_announce( Path::new("room/host").to_owned(), + None, hops, crate::origin::Cost::default(), 0, @@ -2901,6 +2943,7 @@ mod tests { let accepted = subscriber .start_announce( Path::new("room/host").to_owned(), + None, crate::Hops::new(), crate::origin::Cost::UNKNOWN, 0, @@ -2948,6 +2991,7 @@ mod tests { subscriber .start_announce( Path::new("room/host").to_owned(), + None, hops, crate::origin::Cost::UNKNOWN, 0, @@ -2978,6 +3022,7 @@ mod tests { first .start_announce( Path::new("room/host").to_owned(), + None, crate::Hops::new(), crate::origin::Cost::UNKNOWN, 0, @@ -3035,6 +3080,7 @@ mod tests { subscriber .start_announce( path.clone(), + None, crate::Hops::new(), crate::origin::Cost::UNKNOWN, 0, @@ -3090,6 +3136,7 @@ mod tests { subscriber .start_announce( path.clone(), + None, hops, crate::origin::Cost::default(), 1, @@ -3111,6 +3158,7 @@ mod tests { subscriber .start_announce( path.clone(), + None, hops, crate::origin::Cost::default(), 1, @@ -3132,6 +3180,7 @@ mod tests { async fn the_count_lands_at_the_last_initial_start() { const VERSION: Version = Version::Lite06; let start = |suffix| lite::AnnounceBroadcast::Active { + epoch: None, suffix: lite::PathRef::literal(Path::new(suffix)), hops: lite::HopsRef::literal(crate::Hops::new()), cost: crate::origin::Cost::default(), @@ -3273,6 +3322,9 @@ enum Sub { #[derive(Default)] struct Announced { routes: HashMap>, + /// The epoch each advertisement named, kept while it stands even when declined, + /// so a restart that attaches it later serves the same broadcast. + epochs: HashMap, /// Attached routes whose request queue woke since the last serve pass. The /// driver wakes for every group the session carries, so a pass must cost what /// was requested, not every route the peer announced. @@ -3310,11 +3362,19 @@ impl Announced { /// the peer still holds. /// Only valid on a prefix the peer does not already hold, which the caller establishes /// with [`Self::contains`]. Overwriting an attached route is [`Self::declined`]'s job. - fn reserve(&mut self, path: PathOwned) { + fn reserve(&mut self, path: PathOwned, epoch: Option) { debug_assert!(!self.routes.contains_key(&path), "reserved a prefix already advertised"); + if let Some(epoch) = epoch { + self.epochs.insert(path.clone(), epoch); + } self.routes.insert(path, None); } + /// The epoch the advertisement at `path` named, if any. + fn epoch(&self, path: &PathOwned) -> Option { + self.epochs.get(path).cloned() + } + fn attached(&mut self, path: &PathOwned) -> Option<&mut AnnouncedRoute> { self.routes.get_mut(path)?.as_mut() } @@ -3322,6 +3382,7 @@ impl Announced { /// Retire this session's advertisement without invalidating another live /// session from the same peer. Dropping its sources closes their requests. fn withdraw(&mut self, path: &PathOwned) { + self.epochs.remove(path); if let Some(Some(entry)) = self.routes.remove(path) { entry.dynamic.withdrawn(); } @@ -3350,7 +3411,9 @@ impl Announced { }; let path = path.to_owned(); let source = subscriber.origin.create_source(&path); - let _ = subscriber.sources.try_push((path.clone(), source.dynamic())); + let _ = subscriber + .sources + .try_push((path.clone(), entry.route.epoch.clone(), source.dynamic())); request.accept(&source); entry .sources @@ -3460,6 +3523,8 @@ enum ServeEnd { struct TrackServe { subscriber: Subscriber, path: PathOwned, + /// The publisher instance to ask for; the peer refuses a request for another. + epoch: Option, name: String, } @@ -3770,6 +3835,7 @@ impl Establish { let bounds = WireBounds::new(self.subscription.start, self.subscription.end); let msg = lite::Subscribe { + epoch: self.serve.epoch.clone(), id: self.id, broadcast: self.serve.path.as_path(), track: self.serve.name.as_str().into(), @@ -4009,6 +4075,7 @@ impl TrackInfoFetch { let mut stream = ready!(Stream::poll_open(&mut self.session, serve.subscriber.version, &mut cx))?; stream.writer.buffer(&lite::ControlType::Track)?; stream.writer.buffer(&lite::Track { + epoch: serve.epoch.clone(), broadcast: serve.path.as_path(), track: serve.name.as_str().into(), })?; @@ -4505,6 +4572,7 @@ impl kio::Task for FetchServeRun { }; let msg = lite::Fetch { + epoch: self.serve.epoch.clone(), broadcast: self.serve.path.as_path(), track: self.serve.name.as_str().into(), priority: request.priority(), diff --git a/rs/moq-net/src/lite/track.rs b/rs/moq-net/src/lite/track.rs index 0c96058ac9..0b6baa2b7a 100644 --- a/rs/moq-net/src/lite/track.rs +++ b/rs/moq-net/src/lite/track.rs @@ -17,6 +17,9 @@ const LEGACY_UNLIMITED: u64 = (1u64 << 53) - 1; #[derive(Clone, Debug)] pub struct Track<'a> { pub broadcast: Path<'a>, + /// The publisher instance the subscriber expects; see [`crate::origin::Route::epoch`]. + /// Lite07+ only. + pub epoch: Option, pub track: Cow<'a, str>, } @@ -27,9 +30,14 @@ impl Message for Track<'_> { } let broadcast = Path::decode(r, version)?; + let epoch = super::epoch::decode_epoch(r, version)?; let track = Cow::Owned(r.string()?); - Ok(Self { broadcast, track }) + Ok(Self { + broadcast, + epoch, + track, + }) } fn encode_msg(&self, w: &mut Encoder<'_>, version: Version) -> Result<(), EncodeError> { @@ -38,6 +46,7 @@ impl Message for Track<'_> { } self.broadcast.encode(w, version)?; + super::epoch::encode_epoch(w, version, self.epoch.as_ref())?; w.string(&self.track)?; Ok(()) } @@ -269,6 +278,7 @@ mod test { #[test] fn track_request_roundtrips_on_lite05() { let msg = Track { + epoch: None, broadcast: Path::new("room").to_owned(), track: Cow::Borrowed("video"), }; @@ -284,6 +294,7 @@ mod test { #[test] fn track_request_errors_before_lite05() { let msg = Track { + epoch: None, broadcast: Path::new("room").to_owned(), track: Cow::Borrowed("video"), }; diff --git a/rs/moq-net/src/lite/version.rs b/rs/moq-net/src/lite/version.rs index cd33b28d1b..727035b7a1 100644 --- a/rs/moq-net/src/lite/version.rs +++ b/rs/moq-net/src/lite/version.rs @@ -227,6 +227,19 @@ impl Version { } } + /// Whether announcements and track requests carry the publisher epoch, the + /// identity that says two routes serve the same bytes. Added in lite-07. + /// Older versions carry nothing, so a received route has no epoch and is + /// never resumed through another route. + #[allow(clippy::match_like_matches_macro)] + pub fn has_epoch(self) -> bool { + // Match form so future versions default forward (AGENTS.md convention). + match self { + Self::Lite01 | Self::Lite02 | Self::Lite03 | Self::Lite04 | Self::Lite05 | Self::Lite06 => false, + _ => true, + } + } + /// Whether announcements carry the route cost: the marginal cost of pulling /// the broadcast via this route, accumulated per link. Added in lite-06. /// Older versions carry nothing, so a received route stays at zero and ranks diff --git a/rs/moq-net/src/model/broadcast.rs b/rs/moq-net/src/model/broadcast.rs index d1ad79c89f..d4a97e9734 100644 --- a/rs/moq-net/src/model/broadcast.rs +++ b/rs/moq-net/src/model/broadcast.rs @@ -197,8 +197,16 @@ impl Producer { self } - /// Advertise this broadcast's exact path as a route, or re-price the standing - /// advertisement in place. + /// The route this broadcast is announced with, or `None` while it is not. + pub fn route(&self) -> Option { + self.alive.announcer.lock().as_ref()?.route() + } + + /// Advertise this broadcast's exact path as a route, or replace the standing + /// advertisement's route. The route is taken as given, epoch included: another + /// epoch, or none, announces a new broadcast. To re-price the same instance, + /// start from the current route: + /// `broadcast.announce(broadcast.route().unwrap_or_default().with_cost(cost))`. /// /// Until this is called the broadcast exists for nobody: announce cursors do /// not list it and requests for its path fail with [`Error::Unroutable`], for diff --git a/rs/moq-net/src/model/front.rs b/rs/moq-net/src/model/front.rs index b524dbfe8d..402bd98eca 100644 --- a/rs/moq-net/src/model/front.rs +++ b/rs/moq-net/src/model/front.rs @@ -9,10 +9,11 @@ //! sources, the tracks, the clock) and executes the actions; see //! `origin::run_front`. //! -//! A path names one broadcast, whoever serves it, so any route covering the path -//! may take over from another and its tracks resume where they stopped. A -//! publisher reusing a name for different content is a bug; it publishes under a -//! new name (an epoch) instead. +//! The driver only offers a front routes that serve its broadcast: those with the +//! epoch it first resolved, whose tracks resume where they stopped, or its first +//! route alone when that had no epoch, since nothing says another serves the +//! same bytes. A newer epoch supersedes the front, ending even the tracks in +//! flight, so readers re-request the new broadcast rather than stall on the old. //! //! Sources and tracks are named by ids and names, never handles, so a //! transition can be checked in a unit test by comparing the actions it emits. @@ -88,6 +89,9 @@ pub(super) enum Event { /// The driver let go of a track the machine asked it to [`Action::Forget`]. /// Not fed when a reader arrived first: [`Event::Used`] follows instead. Forgotten { track: Arc }, + /// A newer publisher instance replaced the broadcast: every track ends now, + /// even one in flight, and readers re-request the path. + Superseded, /// The origin is tearing down. Closed, } @@ -218,6 +222,11 @@ impl Front { self.excluded.retain(|route| standing(*route)); } + /// The route the attached source came through, if any. + pub(super) fn serving_route(&self) -> Option { + self.serving.map(|(_, candidate)| candidate.route) + } + /// The attached source, if any. pub(super) fn serving(&self) -> Option { self.serving.map(|(source, _)| source) @@ -275,6 +284,7 @@ impl Front { Event::Forgotten { track } => { self.tracks.remove(&track); } + Event::Superseded => self.supersede(&mut actions), Event::Closed => self.end(Error::Dropped, &mut actions), } if !self.ended { @@ -470,8 +480,9 @@ impl Front { let spliced = track.state == (TrackState::Spliced { source }); if track.draining == Some(source) { track.draining = None; - // A path is one broadcast, so a copy that completed ends the track whoever - // serves it now. One that failed leaves the track to the serving source. + // Every source of a front serves one broadcast, so a copy that completed ends + // the track whoever serves it now. One that failed leaves the track to the + // serving source. if result.is_ok() && !track.ended { track.state = TrackState::Idle; track.ended = true; @@ -620,6 +631,21 @@ impl Front { .min() } + /// Abort every track still going, then end: unlike a retraction, a newer + /// broadcast does not leave readers on the old one's copies. + fn supersede(&mut self, actions: &mut Vec) { + for (name, track) in &mut self.tracks { + if !track.ended { + track.ended = true; + actions.push(Action::Abort { + track: name.clone(), + err: Error::Unroutable, + }); + } + } + self.end(Error::Unroutable, actions); + } + fn end(&mut self, err: Error, actions: &mut Vec) { if self.ended { return; @@ -1556,8 +1582,8 @@ mod tests { } /// The serving source closed and the front reselected, but the pump still reads the - /// closed source's copy. That copy ending cleanly ends the track: a path is one - /// broadcast, whoever serves it. + /// closed source's copy. That copy ending cleanly ends the track: every source of + /// the front serves one broadcast. #[test] fn a_closed_sources_clean_end_finishes_the_track() { let mut front = serving(remote(1), 100); diff --git a/rs/moq-net/src/model/origin.rs b/rs/moq-net/src/model/origin.rs index 18d005b85d..f296af2738 100644 --- a/rs/moq-net/src/model/origin.rs +++ b/rs/moq-net/src/model/origin.rs @@ -472,6 +472,13 @@ impl From for Cost { #[derive(Clone, Debug, PartialEq, Eq)] #[non_exhaustive] pub struct Route { + /// The publisher instance the route serves, if known: routes with the same + /// epoch serve the same bytes, so a subscription resumes from one to another. + /// Among routes at one prefix the newest epoch wins, and a route without one + /// ranks last and is never resumed through another route. Changing it on a + /// standing advertisement (including dropping it) retracts and announces afresh. + pub epoch: Option, + /// The chain of origins the route has traversed, oldest first. Each relay /// appends its own [`crate::Hop`] when forwarding; used for loop detection /// and as the selection tie-break. A 0 entry is the anonymous mark and @@ -499,6 +506,7 @@ pub struct Route { impl Default for Route { fn default() -> Self { Self { + epoch: None, hops: Hops::new(), cost: Cost::default(), via: Hop::UNKNOWN, @@ -523,6 +531,12 @@ pub enum Source { } impl Route { + /// Set the publisher instance the route serves. + pub fn with_epoch(mut self, epoch: crate::Epoch) -> Self { + self.epoch = Some(epoch); + self + } + /// Replace the hop chain. pub fn with_hops(mut self, hops: Hops) -> Self { self.hops = hops; @@ -603,8 +617,9 @@ fn fnv_key(name: &str, origins: impl IntoIterator) -> u64 { hash } -/// Ordering key for a route entry resolving `path`. Lower wins: an identified -/// chain (no 0) outranks an anonymous one regardless of cost, then the cheapest +/// Ordering key for a route entry resolving `path`. Lower wins: the newest epoch +/// first, since an older one is a publisher that was replaced, and a route with no +/// epoch last. Then an identified chain (no 0) outranks an anonymous one regardless of cost, then the cheapest /// cost, then a broadcast published on this origin (it serves what is here, not a /// claim that has to ask), then the shortest hop chain, then a deterministic hash /// of `path` and the chain so every node converges on the same winner, and finally @@ -615,8 +630,21 @@ fn fnv_key(name: &str, origins: impl IntoIterator) -> u64 { /// itself for an advertisement. Keying the hash on the requested path is what /// spreads equal-cost advertisers of one prefix: each path picks its own winner /// from the pool, rather than every path under the prefix hashing alike. -fn route_order(path: &Path, entry: &RouteEntry) -> (bool, Cost, bool, usize, u64, Reverse) { +#[allow(clippy::type_complexity)] +fn route_order<'a>( + path: &Path, + entry: &'a RouteEntry, +) -> ( + Reverse>, + bool, + Cost, + bool, + usize, + u64, + Reverse, +) { ( + Reverse(entry.epoch.as_ref()), entry.is_anonymous(), entry.cost, !entry.local, @@ -626,8 +654,8 @@ fn route_order(path: &Path, entry: &RouteEntry) -> (bool, Cost, bool, usize, u64 ) } -/// The `(hops, cost, source)` metadata an announce cursor delivers alongside a prefix. -type RouteMeta = (Hops, Cost, Source); +/// The `(hops, cost, source, epoch)` metadata an announce cursor delivers alongside a prefix. +type RouteMeta = (Hops, Cost, Source, Option); /// A pending update's hold, keyed by prefix in its [`AnnounceConsumer`]. /// @@ -828,6 +856,7 @@ impl OriginConsumerState { cost: meta.1, via: Hop::UNKNOWN, source: meta.2, + epoch: meta.3, }, })) } @@ -836,6 +865,8 @@ impl OriginConsumerState { /// One announced route in the origin's table, absolute prefix. struct RouteEntry { id: u64, + /// The publisher instance this route serves; see [`Route::epoch`]. + epoch: Option, prefix: PathOwned, /// The absolute patterns the announcing producer may serve. The prefix is /// only the wire-visible covering claim; this scope remains authoritative. @@ -1123,7 +1154,7 @@ impl TableCursor { fn update(&mut self, presented: &PathOwned, best: Option<&RouteEntry>) { match best { Some(entry) => { - let meta = (entry.hops.clone(), entry.cost, entry.entered()); + let meta = (entry.hops.clone(), entry.cost, entry.entered(), entry.epoch.clone()); let served = entry.server.is_some(); let captures = self.captures(&entry.prefix); let previous = self @@ -1138,9 +1169,10 @@ impl TableCursor { // it would park that waiter forever. Some((_, prev, prev_served, prev_captures)) if prev == meta && prev_served == served && prev_captures == captures => {} - // Captures are consumer identity, not route metadata. Replace the - // old identity explicitly so capture-keyed consumers can remove it. - Some((_, prev, _, prev_captures)) if prev_captures != captures => { + // Captures are consumer identity, not route metadata, and an epoch is + // another broadcast. Replace the old identity explicitly so + // consumers keyed on it remove it. + Some((_, prev, _, prev_captures)) if prev_captures != captures || prev.3 != meta.3 => { if let Ok(mut state) = self.state.write() { state.apply_unannounce(self.under.join(presented), prev, prev_captures); state.apply_announce(self.under.join(presented), meta, captures); @@ -1558,11 +1590,11 @@ impl Producer { /// Create a broadcast at `path`, fed through the returned producer. /// /// This is how local content enters an origin. The returned - /// [`broadcast::Producer`] is a source: the origin owns the broadcast - /// consumers actually see, and feeds its tracks from every source created at the - /// same path, preferring the newest. When the serving source changes, tracks - /// resume from the replacement where they left off; consumers never observe the - /// swap. + /// [`broadcast::Producer`] is a source. Announce it with a [`Route::epoch`] + /// ([`Epoch::mint`](crate::Epoch::mint) per run, or a replica's shared one) so a + /// restart replaces it rather than resuming into different content, and so + /// routes with that epoch resume across each other. Without one it never + /// resumes elsewhere. /// /// The broadcast exists for nobody until [`broadcast::Producer::announce`]: /// until then no announce cursor lists it and a request for its path fails @@ -1933,6 +1965,7 @@ impl Announcing { let stale = shared.withdrawn_through(prefix, &route.hops); shared.routes.insert(RouteEntry { id, + epoch: route.epoch.clone(), prefix: prefix.clone(), scope: self.scope.clone(), hops: route.hops.clone(), @@ -2002,6 +2035,11 @@ impl Announcer { self.entry.withdraw(); self.entry.guard = None; } + + /// The route the broadcast's path is advertised with, if it is. + pub(crate) fn route(&self) -> Option { + self.entry.route() + } } /// The write half of an advertisement: a live claim that paths under a @@ -2021,12 +2059,10 @@ pub(crate) struct AnnounceProducer { } impl AnnounceProducer { - /// Re-price the route in place: replace its hops and cost. + /// Replace the route in place, epoch included; see [`Dynamic::update`]. /// - /// Consumers observe another active update for the same prefix; sessions - /// forward it as a restart, so route churn never looks like new content. The - /// prefix is fixed at announce time and a [`Route`] cannot name one: to move - /// an advertisement, drop this and announce again. Fails with + /// The prefix is fixed at announce time and a [`Route`] cannot name one: to + /// move an advertisement, drop this and announce again. Fails with /// [`Error::Closed`] once the origin's [`Driver`] has been dropped. pub fn update(&self, route: Route) -> Result<(), Error> { let mut shared = self.shared.lock(); @@ -2041,6 +2077,7 @@ impl AnnounceProducer { return Err(Error::Closed); }; entry.hops = route.hops.clone(); + entry.epoch = route.epoch.clone(); entry.stale = stale; entry.cost = route.cost; entry.via = route.via; @@ -2052,6 +2089,20 @@ impl AnnounceProducer { Ok(()) } + /// The route as last given, while advertised: every entry carries the same one. + fn route(&self) -> Option { + let shared = self.shared.lock(); + let (prefix, id) = self.entries.first()?; + let entry = shared.routes.entry(prefix, *id).filter(|entry| entry.advertised)?; + Some(Route { + epoch: entry.epoch.clone(), + hops: entry.hops.clone(), + cost: entry.cost, + via: entry.via, + source: entry.entered(), + }) + } + /// Hide the entries from everyone, local and remote alike: cursors see a /// retraction and requests stop resolving through them. The entries stay, /// so announcing again restores the same route. What @@ -2371,6 +2422,50 @@ async fn run_front(task: FrontTask, origin: TasksWeak) { .await; } +/// What a front may take from the table, given the broadcast it resolved. +enum Pick<'a> { + /// The route to serve from, or none: the broadcast is no longer routed here. + Route(Option<&'a RouteEntry>), + /// A newer publisher instance won the path. + Superseded, +} + +/// The route a front may serve from. `resolved` is the epoch of the route the front +/// first resolved through, once it has: routes with that epoch serve the same bytes, +/// so any of them may take over, and a newer epoch supersedes the front. A front +/// resolved without one stays on its first route, which is the only one known to +/// serve its bytes, until that route goes or a route with an epoch wins. +fn pick<'a>( + table: &'a OriginState, + path: &Path, + horizon: Horizon, + front: &Front, + resolved: Option<&Option>, +) -> Pick<'a> { + let best = table.best_route(path, horizon, front.excluded_routes()); + match resolved { + None => Pick::Route(best), + Some(Some(epoch)) => match best { + Some(entry) if entry.epoch.as_ref() == Some(epoch) => Pick::Route(Some(entry)), + Some(entry) if entry.epoch.as_ref() > Some(epoch) => Pick::Superseded, + // Every route with the epoch is gone: a retraction, not a replacement. + _ => Pick::Route(None), + }, + Some(None) => { + if best.is_some_and(|entry| entry.epoch.is_some()) { + return Pick::Superseded; + } + let serving = front.serving_route().and_then(|route| { + table + .routes + .covering(path) + .find(|entry| entry.id == route && entry.live() && horizon.admits(entry)) + }); + Pick::Route(serving) + } + } +} + /// Feeds the world's events to a [`Front`] and performs the actions it returns, /// until the front ends; returns the tracks it ended in flight. The decisions live /// in the machine; this only waits and executes, so nothing here decides anything @@ -2410,11 +2505,17 @@ async fn serve_front(task: FrontTask) -> Vec { let mut deadline = crate::time::Deadline::new(&timers); // The watch generation the last selection saw. let mut seen = 0; + // The epoch of the route the front first resolved through; see [`pick`]. + let mut resolved: Option> = None; let mut events: VecDeque = VecDeque::new(); // Read the table for the machine: the best route and whether // the serving source is on its way out. Also what the watch wakes for. - let select = |front: &mut Front, sources: &HashMap, seen: &mut u64| -> Event { + let select = |front: &mut Front, + sources: &HashMap, + seen: &mut u64, + resolved: &Option>| + -> Event { let table = shared.read(); if table.closed { return Event::Closed; @@ -2422,12 +2523,13 @@ async fn serve_front(task: FrontTask) -> Vec { // Read alongside the decision, under the lock a poke takes first. *seen = watch.seen(); front.retain_routes(|route| table.routes.covers(&path.as_path(), route)); - let best = table - .best_route(&path.as_path(), horizon, front.excluded_routes()) - .map(|entry| Candidate { + let best = match pick(&table, &path.as_path(), horizon, front, resolved.as_ref()) { + Pick::Route(best) => best.map(|entry| Candidate { route: entry.id, local: entry.local, - }); + }), + Pick::Superseded => return Event::Superseded, + }; let serving_closing = front .serving() .and_then(|id| sources.get(&id)) @@ -2438,14 +2540,27 @@ async fn serve_front(task: FrontTask) -> Vec { // Whether a route's refusal is the answer: only the current winner speaks for the // path. A retraction resolves like a rejection, and a route beaten while its // request was pending speaks for nobody, so either one re-selects instead. - let standing = |front: &Front, route: u64| { - shared - .read() - .best_route(&path.as_path(), horizon, front.excluded_routes()) - .is_some_and(|best| best.id == route) + let standing = |front: &Front, route: u64, resolved: &Option>| { + let table = shared.read(); + matches!( + pick(&table, &path.as_path(), horizon, front, resolved.as_ref()), + Pick::Route(Some(best)) if best.id == route + ) }; - events.push_back(select(&mut front, &sources, &mut seen)); + // Whether resolving through `route` would serve an epoch a newer one already beat. + let superseded = |front: &Front, route: u64| { + let table = shared.read(); + let Some(entry) = table.routes.covering(&path.as_path()).find(|entry| entry.id == route) else { + return false; + }; + matches!( + pick(&table, &path.as_path(), horizon, front, Some(&entry.epoch)), + Pick::Superseded + ) + }; + + events.push_back(select(&mut front, &sources, &mut seen, &resolved)); loop { while let Some(event) = events.pop_front() { @@ -2455,7 +2570,7 @@ async fn serve_front(task: FrontTask) -> Vec { }; for action in front.step(event) { match action { - Action::Reselect => events.push_back(select(&mut front, &sources, &mut seen)), + Action::Reselect => events.push_back(select(&mut front, &sources, &mut seen, &resolved)), Action::Request { route } => { // The entry and what it serves. let found = { @@ -2488,7 +2603,7 @@ async fn serve_front(task: FrontTask) -> Vec { route, result: Err(Refusal { err: Error::Unroutable, - standing: standing(&front, route), + standing: standing(&front, route, &resolved), }), }); continue; @@ -2504,7 +2619,7 @@ async fn serve_front(task: FrontTask) -> Vec { route, result: Err(Refusal { err: Error::Unroutable, - standing: standing(&front, route), + standing: standing(&front, route, &resolved), }), }); continue; @@ -2534,7 +2649,7 @@ async fn serve_front(task: FrontTask) -> Vec { route, result: Err(Refusal { err: Error::Unroutable, - standing: standing(&front, route), + standing: standing(&front, route, &resolved), }), }); continue; @@ -2559,8 +2674,25 @@ async fn serve_front(task: FrontTask) -> Vec { } } Action::Resolve => { - if let Ok(mut pending) = request.write() { - pending.resolved.get_or_insert(Ok(broadcast.consume())); + // The broadcast is whatever its first route serves. + let table = shared.read(); + let epoch = front + .serving_route() + .and_then(|route| { + table + .routes + .covering(&path.as_path()) + .find(|entry| entry.id == route) + .map(|entry| entry.epoch.clone()) + }) + .flatten(); + drop(table); + resolved = Some(epoch.clone()); + if let Ok(mut pending) = request.write() + && pending.resolved.is_none() + { + pending.resolved = Some(Ok(broadcast.consume())); + pending.epoch = epoch; } } Action::Query { track: name, source } => { @@ -2787,6 +2919,16 @@ async fn serve_front(task: FrontTask) -> Vec { Step::Resolved(route, result) => { upstream = None; match result { + // A newer epoch won the path while the first request was in flight: + // the route was beaten, so ask the winner rather than resolve + // requesters onto a broadcast that is already replaced. + Ok(_) if resolved.is_none() && superseded(&front, route) => Event::Resolved { + route, + result: Err(Refusal { + err: Error::Unroutable, + standing: false, + }), + }, Ok(source) => { let id = next_source; next_source += 1; @@ -2797,7 +2939,7 @@ async fn serve_front(task: FrontTask) -> Vec { route, result: Err(Refusal { err, - standing: standing(&front, route), + standing: standing(&front, route, &resolved), }), }, } @@ -2870,7 +3012,7 @@ async fn serve_front(task: FrontTask) -> Vec { deadline.set(None); Event::Deadline { now: timers.now() } } - Step::Table => select(&mut front, &sources, &mut seen), + Step::Table => select(&mut front, &sources, &mut seen, &resolved), }; events.push_back(event); } @@ -3110,6 +3252,15 @@ impl RouteTable { node.entries.push(entry); } + /// The route `id` announced at `prefix`. + fn entry(&self, prefix: &Path, id: u64) -> Option<&RouteEntry> { + let mut node = &self.root; + for part in prefix.parts() { + node = node.children.get(part)?; + } + node.entries.iter().find(|entry| entry.id == id) + } + /// The route `id` announced at `prefix`, for a re-price in place. fn entry_mut(&mut self, prefix: &Path, id: u64) -> Option<&mut RouteEntry> { let mut node = &mut self.root; @@ -3547,6 +3698,9 @@ impl Quiet { #[derive(Default)] struct PendingBroadcast { resolved: Option>, + /// For a front, the epoch of the route it resolved through, written with + /// `resolved`: a request joins the front only while that epoch still wins. + epoch: Option, } /// A served route, from [`Producer::dynamic`]: advertises a path prefix and @@ -3581,12 +3735,23 @@ impl Dynamic { self.announcement.retract(true); } - /// Re-price the route in place: replace its hops and cost. + /// The route this handle advertises, as last given. + pub fn route(&self) -> Route { + self.announcement + .route() + .expect("a dynamic route stands until its handle drops") + } + + /// Replace the route in place, taken as given. + /// + /// At the same [`Route::epoch`], consumers observe another active update for + /// the same prefix and sessions forward it as a restart, so re-pricing never + /// looks like new content. Another epoch, or none, names another publisher + /// instance: consumers see the old broadcast end and a new one start, and the + /// broadcasts served through the old one end. To re-price, start from the + /// current route: `dynamic.update(dynamic.route().with_cost(cost))`. /// - /// Consumers observe another active update for the same prefix; sessions - /// forward it as a restart, so route churn never looks like new content. - /// Broadcasts already served through the route keep serving whatever its hops - /// now say. The prefix is fixed at announce time: to move a route, drop this and call + /// The prefix is fixed at announce time: to move a route, drop this and call /// [`Producer::dynamic`] again. Fails with [`Error::Closed`] once the origin's /// [`Driver`] has been dropped. pub fn update(&self, route: Route) -> Result<(), Error> { @@ -3749,6 +3914,9 @@ pub struct Requesting { // Egress scope applied to the resolved broadcast, so its reads are attributed. // Empty (no-op) for an untagged consumer. stats: stats::Scope, + // The publisher instance the requester named, if any: the front may resolve on + // another one when the route changes while the request is in flight. + epoch: Option, } enum RequestState { @@ -3785,9 +3953,16 @@ impl Requesting { inner, path: PathOwned::default(), stats: stats::Scope::default(), + epoch: None, } } + /// Refuse a resolution on any epoch but `epoch`. + fn with_epoch(mut self, epoch: Option) -> Self { + self.epoch = epoch; + self + } + fn with_path(mut self, path: PathOwned) -> Self { self.path = path; self @@ -3810,10 +3985,12 @@ impl Requesting { RequestState::Failed(error) => Poll::Ready(Err(error.clone())), RequestState::Pending(consumer) => Poll::Ready( match ready!(consumer.poll(waiter, |state| match &state.resolved { - Some(result) => Poll::Ready(result.clone()), + Some(result) => Poll::Ready((result.clone(), state.epoch.clone())), None => Poll::Pending, })) { - Ok(result) => result.map(|broadcast| self.hand_out(broadcast)), + // Another instance than the one named: never hand it out. + Ok((Ok(_), epoch)) if self.epoch.is_some() && epoch != self.epoch => Err(Error::Unroutable), + Ok((result, _)) => result.map(|broadcast| self.hand_out(broadcast)), // Every handler dropped without resolving: nobody could route it. Err(_closed) => Err(Error::Unroutable), }, @@ -4302,13 +4479,17 @@ impl Consumer { /// path resolves through a front the origin's [`Driver`] runs: the request /// mints one or joins the one already serving the path, and the front picks /// the best announced route covering it (the most specific prefix, then the - /// cheapest, a broadcast published on this origin winning ties) and - /// materializes it, from the broadcast itself or from the peer that - /// announced the route. When its serving source dies or a better route - /// appears, the front switches to the best remaining route, invisibly to - /// subscribers: a path names one broadcast, whoever serves it. Its route - /// retracting with no replacement ends the broadcast, and the next request - /// re-serves the path. Tracks already in flight carry on to their own end. + /// newest [`Route::epoch`], then the cheapest, a broadcast published on this + /// origin winning ties) and materializes it, from the broadcast itself or from + /// the peer that announced the route. + /// + /// When its serving source dies or a better route with the same epoch appears, + /// the front switches to it, invisibly to subscribers: the epoch says both serve + /// the same bytes. A route without an epoch is never swapped for another. Its + /// route retracting with no replacement ends the broadcast, and the next request + /// re-serves the path; tracks already in flight carry on to their own end. A + /// newer epoch winning the path ends even those with [`Error::Unroutable`], so + /// readers request the new broadcast instead of stalling on the old one. /// /// The returned future fails with [`Error::Unroutable`] at once when no /// announced route covers the path, including a broadcast created on this @@ -4317,8 +4498,13 @@ impl Consumer { /// succeeds optimistically, and a path that names nothing surfaces as /// [`Error::NotFound`] on its tracks instead. pub fn request_broadcast(&self, path: impl AsPath) -> kio::Pending { - let path = path.as_path(); + self.request(path.as_path(), None) + } + /// [`Self::request_broadcast`], refused with [`Error::Unroutable`] unless the + /// winning route serves `epoch`: a peer asking for one publisher instance must + /// not be handed another. + pub(crate) fn request(&self, path: Path<'_>, epoch: Option<&crate::Epoch>) -> kio::Pending { // The path as this handle names it, absolute: what its scope authorizes and // what its egress is counted under, so a read through a mount bills where // the reader asked. @@ -4352,23 +4538,34 @@ impl Consumer { // Nothing serves the path: no announced broadcast and no served route. // Checked before joining a front, so a front still draining after its // route retracted takes no newcomers. - if state - .best_route(&absolute.as_path(), horizon, &HashSet::new()) - .is_none() - { - return kio::Pending::new(Requesting::failed(Error::Unroutable)); - } + let best = match state.best_route(&absolute.as_path(), horizon, &HashSet::new()) { + Some(best) if epoch.is_none_or(|epoch| best.epoch.as_ref() == Some(epoch)) => best.epoch.clone(), + _ => return kio::Pending::new(Requesting::failed(Error::Unroutable)), + }; // Join the live front for this path and horizon, if any: its watcher // resolves (or already resolved) the request channel with the front's - // broadcast, so repeat requests share one upstream subscription. Whichever - // route serves the path, it is the same broadcast. + // broadcast, so repeat requests share one upstream subscription. A front + // resolved on another epoch is ending (superseded or retracted) even if its + // watcher has not woken yet, so it is replaced rather than joined. let key = (absolute.clone(), horizon); if let Some(front) = state.fronts.get(&key) { - let pending = Requesting::queued(front.request.consume()) - .with_path(requested) - .with_stats(scope); - return kio::Pending::new(pending); + let current = { + let pending = front.request.read(); + match &pending.resolved { + None => true, + Some(Ok(_)) => pending.epoch == best, + Some(Err(_)) => false, + } + }; + if current { + let pending = Requesting::queued(front.request.consume()) + .with_path(requested) + .with_stats(scope) + .with_epoch(epoch.cloned()); + return kio::Pending::new(pending); + } + state.fronts.remove(&key); } // A route covers the path: mint the front and hand its watcher the @@ -4405,7 +4602,12 @@ impl Consumer { }, self.tasks.clone(), )); - kio::Pending::new(Requesting::queued(consumer).with_path(requested).with_stats(scope)) + kio::Pending::new( + Requesting::queued(consumer) + .with_path(requested) + .with_stats(scope) + .with_epoch(epoch.cloned()), + ) } /// Returns the prefix that is automatically stripped from all paths. @@ -4780,6 +4982,11 @@ mod tests { Hop::new(id).unwrap() } + /// One publisher instance, shared by every route in a test that resumes across them. + fn epoch() -> crate::Epoch { + "01900000-0000-7000-8000-000000000001".parse().unwrap() + } + fn hops(ids: &[u64]) -> Hops { let mut list = Hops::new(); for &id in ids { @@ -5703,6 +5910,34 @@ mod tests { assert_eq!(route.cost, Cost::new(9)); } + /// Re-pricing from the current route keeps the epoch, so it stays one broadcast. + #[test] + fn reprice_from_the_current_route_keeps_the_epoch() { + let producer = origin(1).produce(); + let mut announced = producer.consume().announced(); + + let dynamic = producer + .dynamic("room", Route::default().with_epoch(epoch()).with_cost(5)) + .unwrap(); + announced.assert_next_active("room"); + dynamic.update(dynamic.route().with_cost(1)).unwrap(); + let route = announced.assert_next_active("room"); + assert_eq!((route.epoch, route.cost), (Some(epoch()), Cost::new(1))); + assert_eq!(dynamic.route().cost, Cost::new(1)); + + let broadcast = producer.create_broadcast("lobby").unwrap(); + assert!(broadcast.route().is_none(), "not announced yet"); + broadcast + .announce(Route::default().with_epoch(epoch()).with_cost(5)) + .unwrap(); + announced.assert_next_active("lobby"); + broadcast.announce(broadcast.route().unwrap().with_cost(1)).unwrap(); + let route = announced.assert_next_active("lobby"); + assert_eq!((route.epoch, route.cost), (Some(epoch()), Cost::new(1))); + broadcast.unannounce(); + assert!(broadcast.route().is_none(), "withdrawn"); + } + #[test] fn retract_after_undelivered_reprice_still_delivered() { let producer = origin(1).produce(); @@ -5800,11 +6035,16 @@ mod tests { let consumer = producer.consume(); let mut announced = consumer.announced(); - let _local = producer.publish("room/alice", Route::default().with_cost(5)).unwrap(); + let _local = producer + .publish("room/alice", Route::default().with_epoch(epoch()).with_cost(5)) + .unwrap(); assert_eq!(announced.assert_next_active("room/alice").cost, Cost::new(5)); let server = producer - .dynamic("room/alice", Route::default().with_hops(hops(&[10])).with_cost(1)) + .dynamic( + "room/alice", + Route::default().with_epoch(epoch()).with_hops(hops(&[10])).with_cost(1), + ) .unwrap(); let route = announced.assert_next_active("room/alice"); assert_eq!(route.cost, Cost::new(1)); @@ -5818,21 +6058,26 @@ mod tests { pending.await.expect("resolves through the cheaper route"); } - /// A cheaper route that appears after a front was minted takes the front over: the - /// path is one broadcast whoever serves it, so a newcomer joins the front. + /// A cheaper route that appears after a front was minted takes the front over: it + /// carries the same epoch, so a newcomer joins the front. #[moq_net_sim::test] async fn cheaper_route_after_a_front_takes_it_over() { let producer = origin(1).produce(); let consumer = producer.consume(); - let _local = producer.publish("room/alice", Route::default().with_cost(5)).unwrap(); + let _local = producer + .publish("room/alice", Route::default().with_epoch(epoch()).with_cost(5)) + .unwrap(); let first = consumer .request_broadcast("room/alice") .await .expect("resolves locally"); let server = producer - .dynamic("room/alice", Route::default().with_hops(hops(&[10])).with_cost(1)) + .dynamic( + "room/alice", + Route::default().with_epoch(epoch()).with_hops(hops(&[10])).with_cost(1), + ) .unwrap(); let request = queued(&server).await; let upstream = broadcast::Info::new().produce(); @@ -5905,10 +6150,15 @@ mod tests { let mut announced = consumer.announced(); let server = producer - .dynamic("room/alice", Route::default().with_hops(hops(&[10])).with_cost(2)) + .dynamic( + "room/alice", + Route::default().with_epoch(epoch()).with_hops(hops(&[10])).with_cost(2), + ) .unwrap(); announced.assert_next_active("room/alice"); - let _local = producer.publish("room/alice", Route::default().with_cost(2)).unwrap(); + let _local = producer + .publish("room/alice", Route::default().with_epoch(epoch()).with_cost(2)) + .unwrap(); assert!(announced.assert_next_active("room/alice").hops.is_empty()); let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); @@ -6136,7 +6386,7 @@ mod tests { let peer = producer.consume().excluding(origin(7)); let mut view = peer.announced(); let incumbent = producer - .dynamic("room", Route::default().with_hops(hops(&[10]))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(&[10]))) .unwrap(); view.assert_next_active("room"); @@ -6147,7 +6397,7 @@ mod tests { // The same publisher, now also reached through the peer, outlives the incumbent. let echo = producer - .dynamic("room", Route::default().with_hops(hops(&[10, 7]))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(&[10, 7]))) .unwrap(); drop(incumbent); drop(source); @@ -6174,7 +6424,10 @@ mod tests { async fn takeover_keeps_the_open_group_head(dies_first: bool) { let producer = origin(1).produce(); let first_server = producer - .dynamic("room/alice", Route::default().with_hops(hops(&[10])).with_cost(5)) + .dynamic( + "room/alice", + Route::default().with_epoch(epoch()).with_hops(hops(&[10])).with_cost(5), + ) .unwrap(); let pending = producer.consume().request_broadcast("room/alice"); let upstream = broadcast::Info::new().produce(); @@ -6221,7 +6474,13 @@ mod tests { } let cost = if dies_first { 5 } else { 0 }; let replacement_server = producer - .dynamic("room/alice", Route::default().with_hops(hops(&[10])).with_cost(cost)) + .dynamic( + "room/alice", + Route::default() + .with_epoch(epoch()) + .with_hops(hops(&[10])) + .with_cost(cost), + ) .unwrap(); let replacement = broadcast::Info::new().produce(); let mut replacement_dynamic = replacement.dynamic(); @@ -7540,7 +7799,7 @@ mod tests { let consumer = producer.consume(); let server = producer - .dynamic("room", Route::default().with_hops(hops(first))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(first))) .unwrap(); let pending = consumer.request_broadcast("room/alice"); @@ -7583,7 +7842,7 @@ mod tests { /// back its handle, ready to answer the front's re-request. fn standby(&self, first: &[u64]) -> Dynamic { self.producer - .dynamic("room", Route::default().with_hops(hops(first))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(first))) .unwrap() } } @@ -7617,6 +7876,192 @@ mod tests { assert_eq!(&frame.payload[..], b"resumed"); } + /// Write one single-frame group and read it back through `subscription`. + async fn deliver(track: &track::Producer, subscription: &mut track::Subscriber, payload: &'static [u8]) { + let mut group = track.append_group().unwrap(); + group.write_frame(crate::Timestamp::ZERO, payload).unwrap(); + group.finish().unwrap(); + let mut group = next_group(subscription).await.unwrap().expect("track ended early"); + assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], payload); + } + + /// Without an epoch nothing says another route serves the same bytes: a better + /// route does not move the subscription, and when its own route goes it ends + /// rather than resuming elsewhere. + #[moq_net_sim::test] + async fn a_route_without_an_epoch_never_resumes_elsewhere() { + let producer = origin(1).produce(); + let consumer = producer.consume(); + let first = producer + .dynamic("room", Route::default().with_hops(hops(&[10])).with_cost(5)) + .unwrap(); + let pending = consumer.request_broadcast("room/alice"); + let source = broadcast::Info::new().produce(); + let track = source.create_track("video", None).unwrap(); + queued(&first).await.accept(&source); + let resolved = pending.await.expect("resolves"); + let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); + deliver(&track, &mut subscription, b"first").await; + + let second = producer + .dynamic("room", Route::default().with_hops(hops(&[11])).with_cost(1)) + .unwrap(); + deliver(&track, &mut subscription, b"pinned").await; + + drop(first); + drop(track); + drop(source); + assert!( + next_group(&mut subscription).await.is_err(), + "the track ends with its route" + ); + settle(|| resolved.is_closed()).await; + assert!( + second.poll_requested_broadcast(&kio::Waiter::noop()).is_pending(), + "another route without an epoch is never asked to resume" + ); + } + + /// A newer publisher at the path wins even when it costs more, and ends the + /// subscriptions on the old one at once: they re-request rather than stall on a + /// broadcast that was replaced. + #[moq_net_sim::test] + async fn a_newer_epoch_ends_the_broadcast_in_flight() { + let producer = origin(1).produce(); + let consumer = producer.consume(); + let old = producer + .publish( + "room/alice", + Route::default().with_epoch(crate::Epoch::mint()).with_cost(1), + ) + .unwrap(); + let old_track = old.create_track("video", None).unwrap(); + let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); + let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); + deliver(&old_track, &mut subscription, b"old").await; + + let new = producer + .publish( + "room/alice", + Route::default().with_epoch(crate::Epoch::mint()).with_cost(9), + ) + .unwrap(); + let new_track = new.create_track("video", None).unwrap(); + assert!(matches!(next_group(&mut subscription).await, Err(Error::Unroutable))); + + let replaced = consumer.request_broadcast("room/alice").await.expect("resolves"); + assert!(!replaced.is_clone(&resolved)); + let mut subscription = replaced.track("video").unwrap().subscribe(None).await.unwrap(); + deliver(&new_track, &mut subscription, b"new").await; + } + + /// A request in the same tick as a newer epoch's announcement is never handed + /// the old front: its watcher has not noticed the replacement yet, but the + /// request mints a front on the new epoch instead of joining one that is ending. + #[moq_net_sim::test] + async fn a_request_racing_a_newer_epoch_skips_the_old_front() { + let producer = origin(1).produce(); + let consumer = producer.consume(); + let old = producer + .publish("room/alice", Route::default().with_epoch(crate::Epoch::mint())) + .unwrap(); + let _old_track = old.create_track("video", None).unwrap(); + let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); + + let epoch = crate::Epoch::mint(); + let new = producer + .publish("room/alice", Route::default().with_epoch(epoch.clone())) + .unwrap(); + let new_track = new.create_track("video", None).unwrap(); + let named = consumer.request(Path::new("room/alice"), Some(&epoch)); + let bare = consumer.request_broadcast("room/alice"); + + let named = named.await.expect("the new epoch resolves"); + let bare = bare.await.expect("the new epoch resolves"); + assert!( + !named.is_clone(&resolved), + "a request naming the new epoch got the old one" + ); + assert!(!bare.is_clone(&resolved), "a bare request joined the superseded front"); + assert!(named.is_clone(&bare), "both share the new front"); + let mut subscription = named.track("video").unwrap().subscribe(None).await.unwrap(); + deliver(&new_track, &mut subscription, b"new").await; + } + + /// A newer epoch winning while the front's first request is in flight beats that + /// route: its late answer is not what requesters resolve to. + #[moq_net_sim::test] + async fn an_answer_racing_a_newer_epoch_resolves_the_winner() { + let producer = origin(1).produce(); + let consumer = producer.consume(); + let old = producer + .dynamic("room", Route::default().with_epoch(crate::Epoch::mint())) + .unwrap(); + let pending = consumer.request_broadcast("room/alice"); + let request = queued(&old).await; + + // The answer and the newer epoch land in the same tick. + let new = producer + .publish("room/alice", Route::default().with_epoch(crate::Epoch::mint())) + .unwrap(); + let new_track = new.create_track("video", None).unwrap(); + let stale = broadcast::Info::new().produce(); + let _stale_track = stale.create_track("video", None).unwrap(); + request.accept(&stale); + + let resolved = pending.await.expect("resolves the winner"); + let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); + deliver(&new_track, &mut subscription, b"new").await; + } + + /// Announce cursors follow the newest epoch over a cheaper one, and deliver a + /// change of epoch as an end and a fresh start: a different broadcast. + #[moq_net_sim::test] + async fn the_newest_epoch_is_announced_as_a_new_broadcast() { + let producer = origin(1).produce(); + let mut announced = producer.consume().announced(); + let _old = producer + .publish( + "room/alice", + Route::default().with_epoch(crate::Epoch::mint()).with_cost(1), + ) + .unwrap(); + let old = announced.assert_next_active("room/alice").epoch.expect("announced"); + + let new = producer + .publish( + "room/alice", + Route::default().with_epoch(crate::Epoch::mint()).with_cost(9), + ) + .unwrap(); + announced.assert_next_ended("room/alice"); + let newest = announced.assert_next_active("room/alice").epoch.expect("announced"); + assert!(newest > old); + + // The older one still stands, so it comes back once the newer one goes. + drop(new); + announced.assert_next_ended("room/alice"); + assert_eq!(announced.assert_next_active("room/alice").epoch, Some(old)); + } + + /// A peer naming one publisher instance is never handed another. + #[moq_net_sim::test] + async fn a_request_for_another_epoch_is_refused() { + let producer = origin(1).produce(); + let consumer = producer.consume(); + let _broadcast = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); + + let other: crate::Epoch = "01900000-0000-7000-8000-000000000002".parse().unwrap(); + let refused = consumer.request(Path::new("room/alice"), Some(&other)).await; + assert!(matches!(refused, Err(Error::Unroutable))); + consumer + .request(Path::new("room/alice"), Some(&epoch())) + .await + .expect("the named epoch resolves"); + } + /// The driver's completion contract: it resolves once every producer handle /// drops, however many read handles remain. #[moq_net_sim::test] @@ -7670,9 +8115,9 @@ mod tests { } } - /// A path is one broadcast whoever publishes it: when the serving route dies, the - /// subscription resumes through another route, whatever its first hop, or with - /// none at all. + /// Routes with one epoch serve one broadcast: when the serving route dies, the + /// subscription resumes through another with that epoch, whatever its first hop, + /// or with none at all. #[moq_net_sim::test] async fn any_route_at_the_path_resumes_the_subscription() { let cases = [ @@ -7733,11 +8178,11 @@ mod tests { None => producer.consume(), }; let incumbent = producer - .dynamic("room", Route::default().with_hops(hops(&[10]))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(&[10]))) .unwrap(); // The same publisher, reached through the requesting peer. let echo = producer - .dynamic("room", Route::default().with_hops(hops(&[10, 7]))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(&[10, 7]))) .unwrap(); let pending = consumer.request_broadcast("room/alice"); @@ -7929,7 +8374,12 @@ mod tests { // keeps serving. The front migrates to the standby without waiting for // the death. incumbent - .update(Route::default().with_hops(hops(&[10])).with_cost(Cost::DRAIN)) + .update( + Route::default() + .with_epoch(epoch()) + .with_hops(hops(&[10])) + .with_cost(Cost::DRAIN), + ) .unwrap(); assert_resumes(&mut rig, &standby_server).await; @@ -8313,7 +8763,9 @@ mod tests { let producer = origin(1).produce(); let consumer = producer.consume(); - let first = producer.publish("room/alice", Route::default()).unwrap(); + let first = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let track = first.create_track("video", None).unwrap(); let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); let mut subscription = resolved @@ -8329,7 +8781,9 @@ mod tests { assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], b"before"); // The newest source is dispatched the track, and refused for its metadata. - let second = producer.publish("room/alice", Route::default()).unwrap(); + let second = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let _incompatible = second .create_track("video", track::Info::default().with_timescale(crate::Timescale::MICRO)) .unwrap(); @@ -8361,7 +8815,7 @@ mod tests { ) { let producer = origin(1).produce(); let server = producer - .dynamic("room", Route::default().with_hops(hops(&[10]))) + .dynamic("room", Route::default().with_epoch(epoch()).with_hops(hops(&[10]))) .unwrap(); let pending = producer.consume().request_broadcast("room/alice"); let upstream = broadcast::Info::new().produce(); @@ -8386,7 +8840,7 @@ mod tests { let newcomer = producer.create_broadcast("room/alice").unwrap(); let mut handler = newcomer.dynamic(); - newcomer.announce(Route::default()).unwrap(); + newcomer.announce(Route::default().with_epoch(epoch())).unwrap(); let _unanswered = moq_net_sim::timeout(Duration::from_secs(1), handler.requested_track()) .await .expect("the front asked the newcomer") @@ -8409,7 +8863,9 @@ mod tests { let resolved = producer.consume().request_broadcast("room/alice").await.unwrap(); // No handler and no track: refuses "video" with NotFound. - let _second = producer.publish("room/alice", Route::default()).unwrap(); + let _second = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); for _ in 0..10 { moq_net_sim::yield_now().await; } @@ -8426,7 +8882,9 @@ mod tests { .expect("parked") .expect("open"); - let _third = producer.publish("room/alice", Route::default()).unwrap(); + let _third = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); for _ in 0..10 { moq_net_sim::yield_now().await; } @@ -8459,7 +8917,9 @@ mod tests { async fn a_refused_copy_is_not_kept_subscribed() { let producer = origin(1).produce(); let consumer = producer.consume(); - let first = producer.publish("room/alice", Route::default()).unwrap(); + let first = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let track = first.create_track("video", None).unwrap(); let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); @@ -8469,7 +8929,9 @@ mod tests { let mut group = subscription.recv_group().await.unwrap().unwrap(); assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], b"before"); - let second = producer.publish("room/alice", Route::default()).unwrap(); + let second = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let incompatible = second .create_track("video", track::Info::default().with_timescale(crate::Timescale::MICRO)) .unwrap(); @@ -8491,7 +8953,9 @@ mod tests { async fn the_end_never_feeds_a_refused_copy() { let producer = origin(1).produce(); let consumer = producer.consume(); - let first = producer.publish("room/alice", Route::default()).unwrap(); + let first = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let track = first.create_track("video", None).unwrap(); let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); @@ -8502,7 +8966,9 @@ mod tests { assert_eq!(&group.read_frame().await.unwrap().unwrap().payload[..], b"before"); drop(group); - let second = producer.publish("room/alice", Route::default()).unwrap(); + let second = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let incompatible = second .create_track("video", track::Info::default().with_timescale(crate::Timescale::MICRO)) .unwrap(); @@ -8542,7 +9008,9 @@ mod tests { async fn the_end_keeps_the_copy_still_serving() { let producer = origin(1).produce(); let consumer = producer.consume(); - let first = producer.publish("room/alice", Route::default()).unwrap(); + let first = producer + .publish("room/alice", Route::default().with_epoch(epoch())) + .unwrap(); let track = first.create_track("video", None).unwrap(); let resolved = consumer.request_broadcast("room/alice").await.expect("resolves"); let mut subscription = resolved.track("video").unwrap().subscribe(None).await.unwrap(); @@ -8555,7 +9023,7 @@ mod tests { let second = producer.create_broadcast("room/alice").unwrap(); let mut handler = second.dynamic(); - second.announce(Route::default()).unwrap(); + second.announce(Route::default().with_epoch(epoch())).unwrap(); let request = moq_net_sim::timeout(Duration::from_secs(1), handler.requested_track()) .await .expect("the front asked the newcomer") diff --git a/rs/moq-net/src/model/resume.rs b/rs/moq-net/src/model/resume.rs index 4ea64199d3..dc342792ef 100644 --- a/rs/moq-net/src/model/resume.rs +++ b/rs/moq-net/src/model/resume.rs @@ -1,8 +1,8 @@ //! Serve a front's logical track straight from its routes' copies. //! //! A front serves each track through one route at a time and switches when that route -//! dies, withdraws, or is beaten. A path names one broadcast whoever serves it, so every -//! route's copy of a track holds the same groups and frames. A reader of the logical +//! dies, withdraws, or is beaten by another with the same epoch. Routes with one epoch +//! serve one broadcast, so every route's copy of a track holds the same groups and frames. A reader of the logical //! track therefore reads the serving route's copy directly, through its own //! [`Subscriber`]: nothing is copied, and with one route it is a passthrough. //! diff --git a/rs/moq-net/src/path/epoch.json b/rs/moq-net/src/path/epoch.json deleted file mode 100644 index fe51a8e82b..0000000000 --- a/rs/moq-net/src/path/epoch.json +++ /dev/null @@ -1,125 +0,0 @@ -{ - "valid": [ - { - "text": "00000000-0000-7000-8000-000000000000", - "unix_ms": 0 - }, - { - "text": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "unix_ms": 1759727860778 - }, - { - "text": "0199b7f4-3c2a-7d1e-af0b-2b6c1a9d8e7f", - "unix_ms": 1759727860778 - }, - { - "text": "0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a", - "unix_ms": 1759727885840 - }, - { - "text": "ffffffff-ffff-7fff-bfff-ffffffffffff", - "unix_ms": 281474976710655 - } - ], - "invalid": [ - "", - "alice", - "2026/09", - "0199B7F4-3C2A-7D1E-9F0B-2B6C1A9D8E7F", - "0199b7f43c2a7d1e9f0b2b6c1a9d8e7f", - "@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "{0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f}", - "urn:uuid:0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-4d1e-9f0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-7d1e-cf0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-7d1e-7f0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f\n", - " 0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f ", - "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7g" - ], - "ordered": [ - "00000000-0000-7000-8000-000000000000", - "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "0199b7f4-3c2a-7d1e-af0b-2b6c1a9d8e7f", - "0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a", - "ffffffff-ffff-7fff-bfff-ffffffffffff" - ], - "paths": [ - { - "path": "", - "name": "", - "epoch": null - }, - { - "path": "demo/BBB.hang/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "name": "demo/BBB.hang", - "epoch": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f" - }, - { - "path": "@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "name": "", - "epoch": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f" - }, - { - "path": ".private/demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "name": ".private/demo", - "epoch": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f" - }, - { - "path": "demo/.private/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "name": "demo/.private", - "epoch": "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f" - }, - { - "path": "demo/@alice", - "name": "demo/@alice", - "epoch": null - }, - { - "path": "demo/0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "name": "demo/0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "epoch": null - }, - { - "path": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f/track", - "name": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f/track", - "epoch": null - }, - { - "path": "demo/@0199B7F4-3C2A-7D1E-9F0B-2B6C1A9D8E7F", - "name": "demo/@0199B7F4-3C2A-7D1E-9F0B-2B6C1A9D8E7F", - "epoch": null - }, - { - "path": "demo/@0199b7f4-3c2a-7d1e-cf0b-2b6c1a9d8e7f", - "name": "demo/@0199b7f4-3c2a-7d1e-cf0b-2b6c1a9d8e7f", - "epoch": null - }, - { - "path": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f/@0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a", - "name": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f", - "epoch": "0199b7f4-9e10-7f42-8a3c-5d7e2b1c0f9a" - }, - { - "path": "demo/@0199b7f4-3c2a-4d1e-9f0b-2b6c1a9d8e7f", - "name": "demo/@0199b7f4-3c2a-4d1e-9f0b-2b6c1a9d8e7f", - "epoch": null - }, - { - "path": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f.hang", - "name": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f.hang", - "epoch": null - }, - { - "path": "demo/@", - "name": "demo/@", - "epoch": null - }, - { - "path": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f\n", - "name": "demo/@0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f\n", - "epoch": null - } - ] -} diff --git a/rs/moq-net/src/path/mod.rs b/rs/moq-net/src/path/mod.rs index e08f225956..1a9b402b33 100644 --- a/rs/moq-net/src/path/mod.rs +++ b/rs/moq-net/src/path/mod.rs @@ -353,25 +353,6 @@ impl<'a> Path<'a> { } } - /// Split off a final `@` segment, leaving all other paths unchanged. - pub fn split_epoch(&self) -> (Path<'_>, Option) { - let text = self.as_str(); - let (name, segment) = text.rsplit_once('/').unwrap_or(("", text)); - if let Some(epoch) = segment.strip_prefix('@').and_then(|text| text.parse().ok()) { - (Path::new(name), Some(epoch)) - } else { - (self.borrow(), None) - } - } - - /// Append an optional epoch as a final `@` segment. - pub fn join_epoch(&self, epoch: Option<&crate::Epoch>) -> PathOwned { - match epoch { - Some(epoch) => self.join(format!("@{epoch}")), - None => self.to_owned(), - } - } - /// Resolve a [`Relative`] against this path. /// /// A non-empty reference replaces the last segment of the base, matching relative URL @@ -780,49 +761,6 @@ impl<'de> serde::Deserialize<'de> for Relative<'static> { mod tests { use super::*; - #[test] - fn epoch_vectors() { - let vectors: serde_json::Value = serde_json::from_str(include_str!("epoch.json")).unwrap(); - for row in vectors["valid"].as_array().unwrap() { - let text = row["text"].as_str().unwrap(); - let epoch: crate::Epoch = text.parse().unwrap(); - assert_eq!(epoch.as_str(), text); - assert_eq!( - epoch.time().duration_since(std::time::UNIX_EPOCH).unwrap().as_millis(), - row["unix_ms"].as_u64().unwrap() as u128 - ); - } - for row in vectors["invalid"].as_array().unwrap() { - assert!(row.as_str().unwrap().parse::().is_err(), "{row}"); - } - let ordered: Vec = vectors["ordered"] - .as_array() - .unwrap() - .iter() - .map(|row| row.as_str().unwrap().parse().unwrap()) - .collect(); - assert!(ordered.windows(2).all(|pair| pair[0] < pair[1])); - for row in vectors["paths"].as_array().unwrap() { - let path = Path::new(row["path"].as_str().unwrap()); - let (name, epoch) = path.split_epoch(); - assert_eq!(name.as_str(), row["name"].as_str().unwrap()); - assert_eq!(epoch.as_ref().map(|epoch| epoch.as_str()), row["epoch"].as_str()); - assert_eq!(name.join_epoch(epoch.as_ref()), path); - assert_eq!(path.is_hidden(), name.is_hidden()); - } - } - - #[test] - fn epoch_segments_are_literal_pattern_components() { - let epoch: crate::Epoch = "0199b7f4-3c2a-7d1e-9f0b-2b6c1a9d8e7f".parse().unwrap(); - let path = Path::new("demo/video").join_epoch(Some(&epoch)); - assert!("demo/**".parse::().unwrap().matches(path.as_str())); - assert!("demo/video/@*".parse::().unwrap().matches(path.as_str())); - assert!(!"demo/video".parse::().unwrap().matches(path.as_str())); - assert!(path.as_str().parse::().unwrap().matches(path.as_str())); - assert!(!path.is_hidden()); - } - #[test] fn test_has_prefix() { let path = Path::new("foo/bar/baz"); diff --git a/rs/moq-net/tests/route_change.rs b/rs/moq-net/tests/route_change.rs index b115882baa..f52ecdf850 100644 --- a/rs/moq-net/tests/route_change.rs +++ b/rs/moq-net/tests/route_change.rs @@ -1,10 +1,13 @@ //! A subscription survives its route changing, end to end over real sessions. //! //! A publisher `P` is pulled by two relays `A` and `B`, both of which re-advertise -//! it to the subscribing relay `R`. A path is one broadcast whoever serves it, so `R` -//! may resume a subscription served through one onto the other. The reader on `R` -//! must see every frame exactly once, in order, whether the route changes between -//! groups or in the middle of one, and however it changes. +//! it to the subscribing relay `R`. Both routes carry `P`'s epoch, so `R` may resume +//! a subscription served through one onto the other. The reader on `R` must see +//! every frame exactly once, in order, whether the route changes between groups or +//! in the middle of one, and however it changes. +//! +//! Older wire versions cannot carry the epoch, so `R` cannot tell the two routes +//! serve the same bytes: its subscription stays on its route and ends with it. mod support; @@ -88,7 +91,9 @@ impl Topology { let broadcast = publisher.create_broadcast("live").unwrap(); let track = broadcast.create_track("video", None).unwrap(); - broadcast.announce(Default::default()).unwrap(); + broadcast + .announce(origin::Route::default().with_epoch(moq_net::Epoch::mint())) + .unwrap(); let p_to_a = link(version, &publisher, &relay_a).await; let p_to_b = link(version, &publisher, &relay_b).await; @@ -284,6 +289,37 @@ async fn route_flaps(version: &str) { assert!(rx.try_recv().is_err(), "{version}: trailing delivery"); } +/// A wire without epochs: the subscription stays on `A` while `B` stands by, and +/// ends when `A`'s route goes instead of resuming through `B`. +async fn route_dies_without_an_epoch(version: &str, trigger: Trigger) { + let version: Version = version.parse().unwrap(); + let (mut topology, sub) = Topology::new(version).await; + let mut rx = read(sub); + topology.standby().await; + + let mut group = topology.track.append_group().unwrap(); + for frame in 0..FRAMES { + group.write_frame(Timestamp::ZERO, payload(0, frame)).unwrap(); + assert_eq!(next(&mut rx).await, (0, payload(0, frame)), "{version}"); + } + group.finish().unwrap(); + + topology.trigger(trigger).await; + let mut group = topology.track.append_group().unwrap(); + group.write_frame(Timestamp::ZERO, payload(1, 0)).unwrap(); + settle().await; + match moq_net_sim::timeout(Duration::from_secs(10), rx.next()) + .await + .expect("reader hung") + { + None | Some((_, Err(_))) => {} + Some((group, Ok(frame))) => panic!( + "{version} {trigger:?}: resumed through another route at {group}:{}", + String::from_utf8_lossy(&frame) + ), + } +} + macro_rules! route_change_tests { ($($name:ident: $version:literal,)*) => { $( @@ -333,6 +369,43 @@ macro_rules! route_change_tests { route_change_tests! { lite_07: "moq-lite-07-wip", +} + +macro_rules! pinned_tests { + ($($name:ident: $version:literal,)*) => { + $( + mod $name { + use super::*; + + async fn run(trigger: Trigger) { + moq_net_sim::timeout(TEST_TIMEOUT, route_dies_without_an_epoch($version, trigger)) + .await + .expect("timed out"); + } + + #[moq_net_sim::test] + async fn disconnect_ends_the_subscription() { + run(Trigger::Disconnect).await; + } + + #[moq_net_sim::test] + async fn unannounce_ends_the_subscription() { + run(Trigger::Unannounce).await; + } + + /// A better route without an epoch does not move the subscription either. + #[moq_net_sim::test] + async fn better_route_keeps_the_incumbent() { + moq_net_sim::timeout(TEST_TIMEOUT, route_change($version, Trigger::BetterRoute, Position::MidGroup)) + .await + .expect("timed out"); + } + } + )* + }; +} + +pinned_tests! { lite_06: "moq-lite-06", lite_05: "moq-lite-05", lite_04: "moq-lite-04", @@ -373,7 +446,9 @@ async fn lagging_route_dies(version: Version) -> mpsc::UnboundedReceiver<(u64, m ); let broadcast = p.create_broadcast("live").unwrap(); let track = broadcast.create_track("video", None).unwrap(); - broadcast.announce(Default::default()).unwrap(); + broadcast + .announce(origin::Route::default().with_epoch(moq_net::Epoch::mint())) + .unwrap(); let p_a = lagged(version, &p, &a, Duration::from_millis(300)).await; let p_b = lagged(version, &p, &b, Duration::ZERO).await; let a_r = lagged(version, &a, &r, Duration::ZERO).await; @@ -418,28 +493,13 @@ async fn lagging_route_dies(version: Version) -> mpsc::UnboundedReceiver<(u64, m /// arrive in order. #[moq_net_sim::test] async fn lite_resumes_after_a_lagging_route_dies() { - for version in ["moq-lite-05", "moq-lite-06"] { - let version: Version = version.parse().unwrap(); - let mut rx = lagging_route_dies(version).await; - assert_eq!(next(&mut rx).await, (1, payload(1, 2)), "{version}"); - assert_eq!(next(&mut rx).await, (1, payload(1, 3)), "{version}"); - assert_eq!(next(&mut rx).await, (2, payload(2, 0)), "{version}"); - } -} - -/// Before draft 20 the resume rides a joining FETCH from group 1 through Largest, which -/// a moq-rs publisher refuses for spanning several groups, so the subscription joins -/// live instead. The rest of group 1 is still fetched on its own. -#[moq_net_sim::test] -async fn an_older_ietf_draft_still_resumes_the_open_group() { - let version: Version = "moq-transport-19".parse().unwrap(); + let version: Version = "moq-lite-07-wip".parse().unwrap(); let mut rx = lagging_route_dies(version).await; assert_eq!(next(&mut rx).await, (1, payload(1, 2))); assert_eq!(next(&mut rx).await, (1, payload(1, 3))); + assert_eq!(next(&mut rx).await, (2, payload(2, 0))); } -/// moq-transport 22 is left out: its relay fills a group's missing frames with a FETCH, -/// which draft 20+ does not serve yet (see `quest/m1/ietf-fetch-location.md`). macro_rules! route_flap_tests { ($($name:ident: $version:literal,)*) => { $( @@ -455,9 +515,4 @@ macro_rules! route_flap_tests { route_flap_tests! { flaps_lite_07: "moq-lite-07-wip", - flaps_lite_06: "moq-lite-06", - flaps_lite_05: "moq-lite-05", - flaps_lite_04: "moq-lite-04", - flaps_ietf_19: "moq-transport-19", - flaps_ietf_22: "moq-transport-22", } diff --git a/rs/moq-relay/tests/goaway_cluster.rs b/rs/moq-relay/tests/goaway_cluster.rs index 2bacd26e84..a63dc7d6d4 100644 --- a/rs/moq-relay/tests/goaway_cluster.rs +++ b/rs/moq-relay/tests/goaway_cluster.rs @@ -55,9 +55,15 @@ where /// A stream-only moq listener on an ephemeral loopback TCP port, already /// accepting, so a dial can follow immediately. Returns the port and listener. +const LITE_07: &str = "moq-lite-07-wip"; + async fn listen_tcp() -> (u16, moq_tokio::Listener) { let mut config = moq_tokio::listen::Config::default(); config.tcp.bind = Some("127.0.0.1:0".parse().expect("parse addr")); + // lite-07 alongside the defaults, so a dialer that opts in carries route epochs. + config.version = std::iter::once(LITE_07.parse().unwrap()) + .chain(moq_net::Versions::all().iter().copied()) + .collect(); let server = config .init(Default::default()) .expect("server init") @@ -726,14 +732,17 @@ async fn collect_group(sub: &mut moq_net::track::Subscriber, seen: &mut BTreeSet } /// An empty-URI GOAWAY ("reconnect to me") makes the cluster redial the same -/// endpoint. Both sessions' routes name the same first hop, so the subscription -/// through the drained session re-splices onto the redial and keeps delivering. +/// endpoint. Both sessions' routes carry the publisher's epoch on lite-07, so the +/// subscription through the drained session re-splices onto the redial and keeps +/// delivering. async fn cluster_reconnects_on_empty_uri_goaway_inner() { let _ = rustls::crypto::aws_lc_rs::default_provider().install_default(); let upstream_origin = moq_tokio::origin::spawn(); let broadcast = upstream_origin.create_broadcast("cam").expect("create broadcast"); - broadcast.announce(Default::default()).expect("create broadcast"); + broadcast + .announce(moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint())) + .expect("create broadcast"); let track = broadcast.create_track("video", None).expect("create track"); let (port, mut accepted, _handle) = spawn_upstream(upstream_origin.clone()).await; @@ -742,6 +751,8 @@ async fn cluster_reconnects_on_empty_uri_goaway_inner() { client_config.tls.insecure = Some(true); // Short handover so the test observes the old session close quickly. client_config.goaway.handover = Duration::from_secs(2); + // Only lite-07 carries the epoch that lets the redial resume the subscription. + client_config.version = vec![LITE_07.parse().unwrap()]; let client = client_config.init(Default::default()).expect("client init"); let mut cluster_config = cluster::Config::default(); @@ -798,7 +809,7 @@ async fn cluster_reconnects_on_empty_uri_goaway_inner() { within("old session drains", first_dial.closed()).await; - // The redialed session's route names the same first hop, so the broadcast + // The redialed session's route carries the same epoch, so the broadcast // re-splices onto it: the SAME subscription delivers the next group, with // nothing re-delivered and no visible end. let mut g = track.append_group().expect("append group"); diff --git a/rs/moq-tokio/tests/broadcast.rs b/rs/moq-tokio/tests/broadcast.rs index 3d5a8d0400..3f56624d12 100644 --- a/rs/moq-tokio/tests/broadcast.rs +++ b/rs/moq-tokio/tests/broadcast.rs @@ -991,14 +991,17 @@ async fn read_payloads(sub: &mut moq_net::track::Subscriber, count: usize) -> Ve /// The client connects to two servers announcing the same route. The preferred /// (cheaper) route serves the track; when that session dies, the broadcast /// re-splices through the standby at a group boundary, without the path ever -/// being retracted: both routes name the same first hop, so they are the same -/// origin reached different ways and the subscription rides the failover. +/// being retracted: both routes carry the same epoch over lite-07, so they serve +/// the same bytes and the subscription rides the failover. #[tracing_test::traced_test] #[tokio::test] async fn broadcast_route_migration() { use moq_net::Timestamp; let publisher = Hop::new(0x42).unwrap(); + // Replicas of one publisher instance, and the only version that carries it. + let epoch = moq_net::Epoch::mint(); + let lite_07: moq_net::Version = "moq-lite-07-wip".parse().unwrap(); // ── publisher A: the preferred route (cheaper) ────────────────── let origin_a = moq_tokio::origin::spawn(); @@ -1006,7 +1009,12 @@ async fn broadcast_route_migration() { hops_a.push(publisher).unwrap(); let broadcast_a = origin_a.create_broadcast("test").expect("create broadcast"); broadcast_a - .announce(moq_net::origin::Route::default().with_hops(hops_a).with_cost(1)) + .announce( + moq_net::origin::Route::default() + .with_epoch(epoch.clone()) + .with_hops(hops_a) + .with_cost(1), + ) .expect("announce"); let track_a = broadcast_a.create_track("video", None).expect("create track"); for sequence in 0..2u64 { @@ -1026,7 +1034,12 @@ async fn broadcast_route_migration() { hops_b.push(Hop::new(0x1234).unwrap()).unwrap(); let broadcast_b = origin_b.create_broadcast("test").expect("create broadcast"); broadcast_b - .announce(moq_net::origin::Route::default().with_hops(hops_b).with_cost(2)) + .announce( + moq_net::origin::Route::default() + .with_epoch(epoch.clone()) + .with_hops(hops_b) + .with_cost(2), + ) .expect("announce"); let track_b = broadcast_b.create_track("video", None).expect("create track"); // A clone to keep producing from the test body once the task owns the rest. @@ -1043,12 +1056,14 @@ async fn broadcast_route_migration() { } let server_a = { let mut config = moq_tokio::listen::Config::default(); + config.version = vec![lite_07]; config.bind = Some("[::]:0".parse().unwrap()); config.tls.generate = vec!["localhost".into()]; config.init(Default::default()).expect("init server a") }; let server_b = { let mut config = moq_tokio::listen::Config::default(); + config.version = vec![lite_07]; config.bind = Some("[::]:0".parse().unwrap()); config.tls.generate = vec!["localhost".into()]; config.init(Default::default()).expect("init server b") @@ -1082,6 +1097,7 @@ async fn broadcast_route_migration() { let connect = |port: u16, sub: moq_net::origin::Producer| { let mut config = moq_tokio::connect::Config::default(); + config.version = vec![lite_07]; config.tls.insecure = Some(true); let client = config.init(Default::default()).expect("init client"); let url: url::Url = format!("moqt://localhost:{port}").parse().unwrap();