diff --git a/doc/setup/upgrade.md b/doc/setup/upgrade.md index 5957b50ab9..231454d9d8 100644 --- a/doc/setup/upgrade.md +++ b/doc/setup/upgrade.md @@ -40,6 +40,16 @@ These land with the next breaking release, not the 2026-09-23 train. the same two flags. In Rust, moq-mux's `ts::Export::resume` is `ts::Export::follow`, `Source::returned` is gone, and `ts::Follower` follows a path's announcements the way both exports do. +- **moq-mux's `Source` is built from the resolved catalog broadcast.** + `Source::new(origin, path)` is `Source::new(origin, broadcast)`: resolve the + broadcast first (`origin.request_broadcast(path, None)`), and its path is + where catalog references resolve from. `Source::broadcast()` returns that + broadcast without awaiting. Every later request for a path, siblings + included, stays on the broadcast the source first held there and fails once + it ends, rather than reaching a replacement; `bind` included. Build a new + `Source` to follow a new instance. moq-hls's `Broadcaster::new` no longer + awaits or fails, and a rendition whose sibling broadcast ends now ends its + playlist instead of rebinding. - **fMP4 export of Annex-B H.264 and H.265 inits from the catalog.** When the catalog codec string and dimensions are enough, `moq export fmp4` writes an `avc3` or `hev1` init segment before the first keyframe and leaves SPS, PPS, diff --git a/quest/m0/broadcast-epoch/README.md b/quest/m0/broadcast-epoch/README.md index c26fd1ded7..93990ba0bc 100644 --- a/quest/m0/broadcast-epoch/README.md +++ b/quest/m0/broadcast-epoch/README.md @@ -125,7 +125,6 @@ End+Start. The child quests own their focused regressions. - [Retired requests](/quest/m0/broadcast-epoch/retired-requests.md) - a held broadcast or track cannot open unpinned wire work against its replacement - [Resume duplicates](/quest/m0/broadcast-epoch/resume-duplicates.md) - a delayed group is delivered once even beyond the old 1024-entry history -- [Source pin](/quest/m0/broadcast-epoch/source-pin.md) - `Source` is built from the resolved catalog broadcast and every later request stays on it, so an epochless replacement never splices into the old program - [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - players and exporters pin siblings consistently, with optional reference epochs and documented legacy best effort - [Publish catalog restart](/quest/m0/broadcast-epoch/publish-catalog-restart.md) - `@moq/publish` never reuses catalog group numbers under one name and epoch after a re-announce - [Bindings](/quest/m0/broadcast-epoch/bindings.md) - moq-ffi and every wrapper expose the epoch and let a publisher announce one diff --git a/quest/m0/broadcast-epoch/catalog-references.md b/quest/m0/broadcast-epoch/catalog-references.md index 27a24e16ba..c21c596b80 100644 --- a/quest/m0/broadcast-epoch/catalog-references.md +++ b/quest/m0/broadcast-epoch/catalog-references.md @@ -68,14 +68,10 @@ upgrade notes inline. The [identity guide](/quest/m1/identity-guide.md) consolidates the behavior separately. Explain both the strong epoch-aware guarantee and the selected legacy best-effort behavior. -## Required - -- [Source pin](/quest/m0/broadcast-epoch/source-pin.md) - exporters share their resolved catalog handle - ## Related +- [Source pin](https://github.com/moq-dev/moq/pull/5252) - exporters share their resolved catalog handle - [JS consume identity](https://github.com/moq-dev/moq/pull/5254) - JS resolves the actual serving identity - - [Retired requests](/quest/m0/broadcast-epoch/retired-requests.md) - held handles cannot emit unpinned requests after retirement - [Catalog rendition IDs](/quest/m1/catalog-track-id.md) - the adjacent optional wire-track field - [Catalog track identity](/quest/m1/catalog-tracks.md) - catalog updates cannot redefine a track's content diff --git a/quest/m0/broadcast-epoch/retired-requests.md b/quest/m0/broadcast-epoch/retired-requests.md index 3938286421..38e725752b 100644 --- a/quest/m0/broadcast-epoch/retired-requests.md +++ b/quest/m0/broadcast-epoch/retired-requests.md @@ -58,7 +58,7 @@ Update existing identity and request-lifetime documentation inline. ## Related -- [Source pin](/quest/m0/broadcast-epoch/source-pin.md) - the exporter holds the exact catalog broadcast +- [Source pin](https://github.com/moq-dev/moq/pull/5252) - the exporter holds the exact catalog broadcast - [JS consume identity](https://github.com/moq-dev/moq/pull/5254) - fresh consumers select the serving announcement's identity - [JS restart keeps the request](https://github.com/moq-dev/moq/pull/5199) - an existing resolved request stays sticky - [IETF epochs](/quest/m1/ietf-epochs.md) - explicit wire identity removes the epochless ambiguity diff --git a/quest/m0/broadcast-epoch/source-pin.md b/quest/m0/broadcast-epoch/source-pin.md deleted file mode 100644 index e6f1656822..0000000000 --- a/quest/m0/broadcast-epoch/source-pin.md +++ /dev/null @@ -1,70 +0,0 @@ -# [M] An export's later requests stay on the broadcast it resolved - -## Goal - -`moq_mux::Source` is built from the catalog broadcast its caller resolved, -keeps the first `broadcast::Consumer` it resolves for each cross-broadcast -reference, and serves every later request for a path from that handle. A -late rendition subscribe or SI repoint never lands on a replacement, epoch -or not. On a replaced instance it fails loudly instead of splicing the new -publisher's data into the old program. - -## Plan - -Found by Codex on #5147 (2026-10-10), planned the same day. `Source` pins -its own path by epoch (`epoch` in `rs/moq-mux/src/source.rs`, set through -`pinned` from `ts::Export`). An epochless route (every default lite-06 -session) has `None`, so each later request re-resolves the path, and if a -replacement won in the meantime, it serves that request even without -`--stitch`. - -- Decided: pin by handle, replacing the epoch field. One mechanism covers - epoch and epochless routes. A replaced instance's sticky front refuses new - tracks, so a late request on it errors, and the Follower decides what - happens next (`Replaced`, or a stitch). -- Decided: pin every resolved path, not only the source's own, so a replaced - sibling (`broadcast: ./source`) cannot splice in either. -- Decided (2026-10-10, amending the lazy own-path cache): `Source` is built - from the resolved catalog broadcast, which seeds its own-path handle. Most - callers (e.g. moq-cli, moq-ffi, moq-c, moq-hls, moq-rtc, moq-rtmp, and the - fmp4, flv, and codec exporters) subscribe the catalog outside `Source`, so - a lazy first resolve could still land on a replacement. Construction closes - that for every exporter. moq-hls's own catalog hold (`export/upstream.rs`) - should then be redundant, once `Source` returns the seeded handle for every - self-reference; delete it if so. Callers that let the export resolve today - (the TS export from `moq publish`) resolve first, or the export seeds the - handle from the exact broadcast it resolved, never a second resolve; pick - whichever reads simpler. -- The pin lifecycle follows the export's instance. A stitch builds a new - `Source` whose own-path entry is seeded with the broadcast `follow` was - given (sibling entries start empty), so its first track request can't - resolve a newer replacement. A same-epoch return replaces the own-path - handle with the returned broadcast, since the old one's routes are gone. -- A sibling's guarantee starts at its first resolution. A sibling the catalog - first references after a replacement resolves whatever serves it then. - The export never held another instance of that path, so that isn't a - splice. -- Tests, with mocked time, on an epochless route: an export resolves `live`, - a replacement wins `live`, and then a rendition added to the old catalog - (and an SI repoint) fails rather than reading the replacement. The same - for a sibling reference already resolved. A `Source` built from `live` - after a replacement already won still reads the broadcast it was given on - its first track request. A stitch whose replacement is - itself replaced before the first track request still reads the broadcast - it followed. A same-epoch return reads the returned handle. Today's - epoch-pin test (`source.rs`) moves to the handle. - -Public API: breaking. `Source::new` takes the resolved catalog broadcast, -and every caller in the repo moves with it (follow the Cross-Package Sync -table for moq-ffi and moq-c). A `Source` reused across a route replacement -returns its pinned handle for a path it already holds (`broadcast`, -`catalog`, and track requests), or fails on the replaced instance, instead -of resolving the current route. Document it on `Source`, fix stale -comments inline, and add a line for the `Source::new` break to the -Unreleased section of `doc/setup/upgrade.md`; no new doc page (decided -2026-10-10). Wire: none. - -## Related - -- [Retired requests](/quest/m0/broadcast-epoch/retired-requests.md) - held track FETCH and JS wire operations also need instance validity after retirement; handle pinning alone does not fix them -- [Catalog references](/quest/m0/broadcast-epoch/catalog-references.md) - players adopt the same sibling policy, and optional catalog epochs constrain initial resolution where supported diff --git a/rs/moq-c/src/consume.rs b/rs/moq-c/src/consume.rs index aed2e64588..1ce547a219 100644 --- a/rs/moq-c/src/consume.rs +++ b/rs/moq-c/src/consume.rs @@ -62,7 +62,7 @@ async fn resolve( }; let origin = origin.ok_or_else(|| Error::UnresolvableBroadcast(reference.as_str().to_string()))?; - let source = moq_mux::Source::new(origin, &broadcast.info().path); + let source = moq_mux::Source::new(origin, broadcast); Ok(source.resolve(Some(&reference.borrow())).await?) } diff --git a/rs/moq-cli/src/main.rs b/rs/moq-cli/src/main.rs index aa3f29d199..2c3c9042d7 100644 --- a/rs/moq-cli/src/main.rs +++ b/rs/moq-cli/src/main.rs @@ -810,14 +810,14 @@ async fn run_stdout(consumer: moq_net::origin::Consumer, name: String, args: Sub let catalog = args.catalog_format(&name); // Confirm the broadcast is reachable and wait for it to be announced; `Subscribe` then - // resolves it (and any sibling broadcast a rendition's `broadcast` field references, - // e.g. "./source") through the origin. + // resolves it, and any sibling broadcast a rendition's `broadcast` field references + // (e.g. "./source") through the same origin. consumer .routed(&name) .await .ok_or_else(|| anyhow::anyhow!("origin closed before broadcast `{name}` was announced"))?; - Subscribe::new(consumer, &name, catalog, args).run().await + Subscribe::new(consumer, &name, catalog, args).await?.run().await } /// Run every endpoint until the first finishes (stdin EOF, SIGINT, SIGTERM, or an diff --git a/rs/moq-cli/src/play/media.rs b/rs/moq-cli/src/play/media.rs index e66f1d88cd..07577fd732 100644 --- a/rs/moq-cli/src/play/media.rs +++ b/rs/moq-cli/src/play/media.rs @@ -69,7 +69,6 @@ impl Media { /// update is the same instance, which its subscriptions already ride out. async fn play(self) -> anyhow::Result<()> { let mut follow = self.origin.follow(&self.broadcast)?; - let source = moq_mux::Source::new(self.origin.clone(), &self.broadcast); let mut playing = None; loop { @@ -97,7 +96,7 @@ impl Media { self.video.lock().unwrap().clear(); self.drained.notify_one(); self.output.send(Event::Wake); - playing = Some(Box::pin(self.play_broadcast(source.clone()))); + playing = Some(Box::pin(self.play_broadcast())); } result = async { playing.as_mut().expect("guarded").await }, if playing.is_some() => { playing = None; @@ -111,13 +110,15 @@ impl Media { } } - /// Play the broadcast at the path until its catalog and every track it + /// Play the broadcast now at the path until its catalog and every track it /// started end. - async fn play_broadcast(&self, source: moq_mux::Source) -> anyhow::Result<()> { - let broadcast = source - .broadcast() + async fn play_broadcast(&self) -> anyhow::Result<()> { + let broadcast = self + .origin + .request_broadcast(&self.broadcast, None) .await .context("failed to subscribe to the broadcast")?; + let source = moq_mux::Source::new(self.origin.clone(), broadcast.clone()); let catalog = catalog::Consumer::<()>::new(&broadcast, self.args.catalog_format(&self.broadcast)) .await .context("failed to subscribe to the catalog")?; diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index 446684f680..03b414ba48 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -749,7 +749,7 @@ pub(crate) mod tests { // `catalog`, the producers, and `import` stay alive: the exporter subscribes to // the retained tracks. drain( - Export::with_ts(moq_mux::Source::new(origin.consume(), "cli"), CatalogFormat::Hang) + Export::with_ts(resolved(&origin, "cli").await, CatalogFormat::Hang) .await .unwrap() .with_delay(RECORDING_MAX_AGE), @@ -757,6 +757,13 @@ pub(crate) mod tests { .await } + /// A source for the broadcast at `path`, resolved through `origin`. + async fn resolved(origin: &moq_net::origin::Producer, path: &str) -> moq_mux::Source { + let consumer = origin.consume(); + let broadcast = consumer.request_broadcast(path, None).await.unwrap(); + moq_mux::Source::new(consumer, broadcast) + } + /// The export's delay, long enough to reach the ancillary tracks: these tests publish a /// whole feed before exporting it, and the media tracks start at their newest group. const RECORDING_MAX_AGE: std::time::Duration = Duration::from_secs(30); @@ -800,7 +807,7 @@ pub(crate) mod tests { // Subscribe side: the same `with_ts` call `run_ts` makes, re-emitting the // ancillary streams verbatim. let output = drain( - Export::with_ts(moq_mux::Source::new(origin.consume(), "cli"), format) + Export::with_ts(resolved(&origin, "cli").await, format) .await .unwrap() .with_delay(RECORDING_MAX_AGE), diff --git a/rs/moq-cli/src/subscribe.rs b/rs/moq-cli/src/subscribe.rs index 05cb099711..23b37952c3 100644 --- a/rs/moq-cli/src/subscribe.rs +++ b/rs/moq-cli/src/subscribe.rs @@ -253,18 +253,27 @@ pub struct Subscribe { } impl Subscribe { - /// Export the broadcast at `path` on `origin` with the resolved settings; [`run`](Self::run) - /// drives it. - pub fn new(origin: moq_net::origin::Consumer, path: &str, catalog: CatalogFormat, args: SubscribeArgs) -> Self { + /// Resolve the broadcast at `path` on `origin` to export it with the resolved settings; + /// [`run`](Self::run) drives it. + pub async fn new( + origin: moq_net::origin::Consumer, + path: &str, + catalog: CatalogFormat, + args: SubscribeArgs, + ) -> anyhow::Result { let path = moq_net::Path::new(path).to_owned(); - let source = moq_mux::Source::new(origin.clone(), &path); - Self { + let broadcast = origin + .request_broadcast(&path, None) + .await + .with_context(|| format!("failed to resolve broadcast `{path}`"))?; + let source = moq_mux::Source::new(origin.clone(), broadcast); + Ok(Self { origin, path, source, catalog, args, - } + }) } /// Build the catalog stream, narrowed by the rendition selection flags. The diff --git a/rs/moq-ffi/src/consumer.rs b/rs/moq-ffi/src/consumer.rs index d6d09a5154..381c6096aa 100644 --- a/rs/moq-ffi/src/consumer.rs +++ b/rs/moq-ffi/src/consumer.rs @@ -124,7 +124,7 @@ impl MoqBroadcastConsumer { .clone() .ok_or_else(|| MoqError::UnresolvableBroadcast(reference.as_str().to_string()))?; - let source = moq_mux::Source::new(origin, &self.inner.info().path); + let source = moq_mux::Source::new(origin, self.inner.clone()); Ok(source.resolve(Some(&reference)).await?) } } diff --git a/rs/moq-hls/src/export/archive_tests.rs b/rs/moq-hls/src/export/archive_tests.rs index 6a0db647df..a422ace659 100644 --- a/rs/moq-hls/src/export/archive_tests.rs +++ b/rs/moq-hls/src/export/archive_tests.rs @@ -316,8 +316,7 @@ impl Replay { tokio::task::yield_now().await; } - let source = moq_mux::Source::new(origin.consume(), "rec"); - let broadcaster = Broadcaster::new(source, config).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "rec").await, config); tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()) .await .expect("a rendition becomes playable"); diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 2d0d85b34d..467ff30fc5 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -34,7 +34,6 @@ mod mpd; mod playlist; mod rendition; mod spans; -mod upstream; pub mod renditions; pub mod segments; @@ -48,7 +47,6 @@ use rand::RngExt; pub(crate) use playlist::render_media; pub use rendition::{Kind, Rendition}; use segments::Reference; -pub(crate) use upstream::Upstream; /// Backoff bounds for the initial catalog subscription. /// @@ -124,13 +122,9 @@ pub struct Broadcaster { } impl Broadcaster { - /// Resolve `source`'s catalog broadcast and start tracking its renditions. - pub async fn new(source: moq_mux::Source, config: Config) -> crate::Result> { - let broadcast = source.broadcast().await?; - let upstream = Upstream { - source, - broadcast: broadcast.clone(), - }; + /// Start tracking the renditions of `source`'s catalog broadcast. + pub fn new(source: moq_mux::Source, config: Config) -> Arc { + let broadcast = source.broadcast(); let renditions = renditions::Producer::new(config.window); let timeline_watcher = Arc::new(Mutex::new(None)); let broadcaster = Arc::new(Self { @@ -141,9 +135,9 @@ impl Broadcaster { }); // The watcher owns its own producer clone; the `Broadcaster`'s `Drop` aborts it so the // standing catalog subscription stops when nobody's serving from this broadcaster. - let watcher = tokio::spawn(watch_catalog(upstream, renditions, timeline_watcher, config.history)); + let watcher = tokio::spawn(watch_catalog(source, renditions, timeline_watcher, config.history)); *broadcaster.watcher.lock().unwrap() = Some(watcher); - Ok(broadcaster) + broadcaster } /// Whether the source broadcast has closed (ended or dropped). @@ -356,15 +350,14 @@ impl Drop for Broadcaster { } async fn watch_catalog( - upstream: Upstream, + source: moq_mux::Source, renditions: renditions::Producer, timeline_watcher: Arc>>>, history: bool, ) { - // The already-resolved handle rather than a fresh lookup through `source`, so the catalog, - // the timeline subscription, and the closed check below all refer to the same resolution. A - // same-name replacement between two lookups would otherwise split them. - let broadcast = &upstream.broadcast; + // The source's catalog broadcast, so the catalog, the timeline subscription, and the closed + // check below all refer to the one broadcast it was built from. + let broadcast = &source.broadcast(); let mut delay = CATALOG_RETRY_MIN; let mut consumer = loop { match catalog::Consumer::<()>::new(broadcast, CatalogFormat::Hang).await { @@ -394,7 +387,7 @@ async fn watch_catalog( unbound = true; renditions.fanout().unbound(); } - renditions.sync(&upstream, &catalog); + renditions.sync(&source, &catalog); // Keep the reference while it stays. Replacing it renumbers every segment. let next = self::reference(&catalog, reference.as_ref()); @@ -703,11 +696,7 @@ mod tests { let _broadcast = origin.create_broadcast("a/pub").expect("publish allowed"); _broadcast.announce(Default::default()).expect("publish allowed"); settle().await; - let source = moq_mux::Source::new(origin.consume(), "a/pub"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "a/pub").await; let mut config = hang::catalog::VideoConfig::new(hang::catalog::VideoCodec::VP8); config.broadcast = Some(moq_net::path::Relative::new("../../source").to_owned()); let mut catalog = moq_mux::catalog::hang::Catalog::default(); @@ -719,13 +708,9 @@ mod tests { assert!(renditions.get(Kind::Video, "video").is_none()); } - /// An upstream over an empty published broadcast, for the `sync` reconciliation tests. - async fn empty_upstream(origin: &moq_net::origin::Producer, path: &str) -> Upstream { - let source = moq_mux::Source::new(origin.consume(), path); - Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - } + /// A source over an empty published broadcast, for the `sync` reconciliation tests. + async fn empty_upstream(origin: &moq_net::origin::Producer, path: &str) -> moq_mux::Source { + crate::resolved(origin, path).await } /// A catalog with one video and one audio rendition, both servable. @@ -994,8 +979,7 @@ mod tests { ) .unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let rendition = tokio::time::timeout(Duration::from_secs(5), async { loop { if let Some(rendition) = broadcaster.rendition(Kind::Video, "video0") { @@ -1082,8 +1066,7 @@ mod tests { .unwrap(); broadcast.announce(moq_net::origin::Route::default()).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); broadcaster.ready().await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); let mut cursor = rendition.segments(); @@ -1114,8 +1097,7 @@ mod tests { .unwrap(); broadcast.announce(moq_net::origin::Route::default()).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); broadcaster.ready().await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); @@ -1150,8 +1132,7 @@ mod tests { settle().await; let (_catalog, mut registration, _media) = publish_vp8(&mut broadcast); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let old = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); let old_playlist = old @@ -1195,8 +1176,7 @@ mod tests { settle().await; let (_catalog, _registration, mut media) = publish_vp8(&mut broadcast); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); let unversioned = rendition.media_playlist(None).expect("playable"); @@ -1268,8 +1248,7 @@ mod tests { media.write(vp8_frame(micros, true)).unwrap(); } - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); let _ = tokio::time::timeout(Duration::from_secs(5), rendition.playable()).await; @@ -1325,8 +1304,7 @@ mod tests { media.write(vp8_frame(3_000_000, false)).unwrap(); media.write(vp8_frame(4_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster .rendition(Kind::Video, "video0") @@ -1406,8 +1384,7 @@ mod tests { media.write(vp8_frame(2_000_000, true)).unwrap(); media.finish().unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster .rendition(Kind::Video, "video0") @@ -1470,8 +1447,7 @@ mod tests { video.write(frame(4_000_000, true)).unwrap(); // live edge audio_write(4_000_000); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.playable()).await; let video_rendition = broadcaster.rendition(Kind::Video, "video0").expect("video discovered"); @@ -1551,8 +1527,7 @@ mod tests { media.write(frame(2_000_000, true)).unwrap(); media.write(frame(4_000_000, true)).unwrap(); // live edge - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("video discovered"); let _ = tokio::time::timeout(Duration::from_secs(5), rendition.playable()).await; @@ -1608,8 +1583,7 @@ mod tests { media.write(frame(2_000_000, true)).unwrap(); media.write(frame(4_000_000, true)).unwrap(); // live edge - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("video discovered"); let _ = tokio::time::timeout(Duration::from_secs(5), rendition.playable()).await; @@ -1651,8 +1625,7 @@ mod tests { // Watch before the first fragment, as a live viewer would, so a catalog the moov // published on a provisional clock would stick. - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); assert!( next_event(&mut broadcaster.renditions()).await.is_none(), "the moov alone lists no rendition" @@ -1711,8 +1684,7 @@ mod tests { media.write(frame(0, true)).unwrap(); media.write(frame(2_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.playable()).await; @@ -1788,8 +1760,7 @@ mod tests { producers.push(media); } - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let listed = |name: &'static str, count: usize| { let broadcaster = broadcaster.clone(); async move { @@ -1874,8 +1845,7 @@ mod tests { audio.cut(None).unwrap(); } - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Audio, "audio0").expect("audio discovered"); let _ = tokio::time::timeout(Duration::from_secs(5), rendition.playable()).await; @@ -1922,8 +1892,7 @@ mod tests { media.write(frame(3_000_000, true)).unwrap(); media.write(frame(6_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster.rendition(Kind::Video, "video0").expect("rendition"); let _ = tokio::time::timeout(Duration::from_secs(5), rendition.playable()).await; @@ -1985,8 +1954,7 @@ mod tests { video.write(frame(4_000_000, true)).unwrap(); // segment 2 (live edge) audio_write(4_000_000); // audio group 8 opens its slice of segment 2 - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let video_rendition = broadcaster.rendition(Kind::Video, "video0").expect("video discovered"); let audio_rendition = broadcaster.rendition(Kind::Audio, "audio0").expect("audio discovered"); @@ -2056,8 +2024,7 @@ mod tests { media.write(frame(0, true)).unwrap(); media.write(frame(2_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let _ = tokio::time::timeout(Duration::from_secs(5), broadcaster.ready()).await; let rendition = broadcaster .rendition(Kind::Video, "video0") @@ -2097,7 +2064,7 @@ mod tests { /// Reconcile a one-video-rendition catalog snapshot by hand and drive the timeline, which is /// what `watch_catalog` does with each snapshot. fn export( - upstream: &Upstream, + upstream: &moq_mux::Source, config: &hang::catalog::VideoConfig, ) -> (Arc, tokio::task::JoinHandle<()>) { let mut catalog = moq_mux::catalog::hang::Catalog::default(); @@ -2109,7 +2076,7 @@ mod tests { renditions.sync(upstream, &catalog); let reference = Arc::new((Kind::Video, "video0".to_string())); let watcher = tokio::spawn(watch_timeline( - upstream.broadcast.clone(), + upstream.broadcast(), archive, reference, renditions.fanout(), @@ -2163,11 +2130,7 @@ mod tests { settle().await; // The export binds to the broadcast it reads the catalog from. - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; // The publisher goes away and comes back before the catalog snapshot is reconciled. That // gap is wide in practice: `watch_catalog` retries a not-yet-written catalog track with @@ -2197,11 +2160,7 @@ mod tests { // Control: an export started after the takeover does serve the new publisher's media, so // the assertion above is about which epoch a rendition is bound to, not about a broadcast // that happens to be unreadable. - let source = moq_mux::Source::new(origin.consume(), "live"); - let fresh = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let fresh = crate::resolved(&origin, "live").await; let (rendition, watcher) = export(&fresh, &config); tokio::time::timeout(Duration::from_secs(5), rendition.playable()) .await @@ -2268,11 +2227,7 @@ mod tests { let old_media = publish_media(&origin, OLD, Some(recorder)); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; // A relative reference replaces the base's last segment, so "media" is a sibling of the // catalog broadcast "live". @@ -2310,11 +2265,7 @@ mod tests { // Control: an export started after the takeover does serve the new publisher's media, so // the assertion above is about which epoch the rendition is bound to, not about a // broadcast that happens to be unreadable. - let source = moq_mux::Source::new(origin.consume(), "live"); - let fresh = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let fresh = crate::resolved(&origin, "live").await; let (rendition, watcher) = export(&fresh, &config); tokio::time::timeout(Duration::from_secs(5), rendition.playable()) .await @@ -2391,12 +2342,11 @@ mod tests { } } - // A bound sibling whose route goes away ends with Dropped, and a new publisher may serve - // the path afterwards. The exporter must drop rows listed for the old publisher, re-bind, - // and list only rows that arrive after the new bind resolves: fetching an old row from the - // replacement would serve restarted groups under the previous publisher's segment numbers. + // A bound sibling that ends ends the rendition: its rows go, its playlist ends, and a + // publisher serving the path afterwards is never bound. A broadcast end carries no cause, so + // a clean end and a replacement end alike; a new instance only comes with a new catalog. #[tokio::test] - async fn a_replaced_sibling_drops_old_rows_and_rebinds() { + async fn a_replaced_sibling_ends_the_rendition() { tokio::time::pause(); const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; const NEW: &[u8] = b"NEWNEWNEWNEWNEWN"; @@ -2412,11 +2362,7 @@ mod tests { let old_server = origin.dynamic("media", sibling_route(10)).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; let mut config = video_config(); config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); @@ -2428,12 +2374,7 @@ mod tests { renditions.sync(&upstream, &snapshot); let fanout = renditions.fanout(); let reference = Arc::new((Kind::Video, "video0".to_string())); - let watcher = tokio::spawn(watch_timeline( - upstream.broadcast.clone(), - archive, - reference, - fanout.clone(), - )); + let watcher = tokio::spawn(watch_timeline(upstream.broadcast(), archive, reference, fanout.clone())); let rendition = renditions.get(Kind::Video, "video0").unwrap(); accept_sibling(&old_server, &old_media).await; tokio::time::timeout(Duration::from_secs(5), rendition.playable()) @@ -2453,72 +2394,28 @@ mod tests { until_empty(&rendition).await; assert!( rendition.listed_segment(0).await.unwrap().is_none(), - "a row listed for the old publisher answers 404 after replacement" + "a row listed for the old publisher answers 404" ); + assert!(rendition.snapshot().finished, "the playlist ends with its sibling"); + assert!(early.next().await.unwrap().is_none(), "segment cursors end"); - // A publisher still routed when the incumbent goes would resume its broadcast - // instead: a path is one broadcast, whoever serves it. The bind failed while nothing - // served the path, so the next poll re-issues it. + // A publisher serving the path afterwards, with rows recorded for it, is never bound. let new_server = origin.dynamic("media", sibling_route(11)).unwrap(); let _ = rendition.snapshot(); - - let mut new_media = moq_net::broadcast::Info::new().produce(); - accept_sibling(&new_server, &new_media).await; - // The rendition skips timeline rows until this bind resolves. Joining the - // same front waits for attach, so writes below land after that. - tokio::time::timeout( - Duration::from_secs(5), - origin.consume().request_broadcast("media", None), - ) - .await - .expect("the rebound sibling resolves") - .expect("the replacement is routable"); + assert!( + tokio::time::timeout(Duration::from_secs(1), new_server.requested_broadcast()) + .await + .is_err(), + "the replacement is never requested" + ); let recorder = catalog.enroll_test("video0").unwrap(); - // Model source records lost during the outage: every rendition must carry the - // same new timeline sequence, including a recorder that starts after this rebind. fanout.skip(); + let mut new_media = moq_net::broadcast::Info::new().produce(); let _new_track = write_routed_media(&mut new_media, NEW, recorder, 6_000_000); - let deadline = tokio::time::Instant::now() + Duration::from_secs(5); - loop { - if !rendition.snapshot().segments.is_empty() { - break; - } - assert!( - tokio::time::Instant::now() < deadline, - "rows produced after the new bind were not listed" - ); - tokio::task::yield_now().await; - } - let listed = rendition - .snapshot() - .segments - .into_iter() - .find(|segment| !segment.gap) - .expect("a non-gap row after the new bind") - .segment; - let served = rendition - .listed_segment(listed) - .await - .unwrap() - .expect("the rebound sibling is servable"); - assert!( - contains(&served, NEW), - "rows after the new bind come from the replacement" - ); - assert!(!contains(&served, OLD)); - let mut late = rendition.segments(); - let early = early.next().await.unwrap().unwrap(); - let late = late.next().await.unwrap().unwrap(); - assert_eq!(early.segment, late.segment); - assert!(early.discontinuity > 0, "the replacement starts after a timeline break"); - assert_eq!( - late.discontinuity, early.discontinuity, - "cursor creation cannot reset the baseline" - ); - assert!( - rendition.listed_segment(0).await.unwrap().is_none(), - "a row listed for the old publisher is not served from the replacement" - ); + tokio::time::sleep(Duration::from_secs(1)).await; + let snapshot = rendition.snapshot(); + assert!(snapshot.segments.is_empty(), "nothing of the replacement is listed"); + assert!(snapshot.finished); watcher.abort(); drop((new_media, _new_track, new_server, catalog, live)); } @@ -2542,11 +2439,7 @@ mod tests { let server = origin.dynamic("media", sibling_route(10)).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; let mut config = hang::catalog::VideoConfig::new(hang::catalog::VideoCodec::VP8); config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); @@ -2558,12 +2451,7 @@ mod tests { renditions.sync(&upstream, &snapshot); let fanout = renditions.fanout(); let reference = Arc::new((Kind::Video, "video0".to_string())); - let watcher = tokio::spawn(watch_timeline( - upstream.broadcast.clone(), - archive, - reference, - fanout.clone(), - )); + let watcher = tokio::spawn(watch_timeline(upstream.broadcast(), archive, reference, fanout.clone())); let rendition = renditions.get(Kind::Video, "video0").unwrap(); tokio::time::timeout(Duration::from_secs(5), rendition.playable()) .await @@ -2608,11 +2496,7 @@ mod tests { let server = origin.dynamic("media", sibling_route(11)).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; let mut config = video_config(); config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); @@ -2631,11 +2515,12 @@ mod tests { drop((media, _track, server, catalog, live)); } - // The replacement is often announced after the incumbent is already gone. The rebind - // then resolves Unroutable; Binding never retries, so the next poll must issue a new - // request or the rendition stays empty forever. + // The replacement is often announced after the incumbent is already gone. Nothing serves + // the path in between, but a sibling that resolved once is never looked up again, so the + // rendition stays ended rather than recovering onto the replacement. #[tokio::test] - async fn a_replaced_sibling_recovers_after_an_unroutable_gap() { + async fn a_replaced_sibling_stays_ended_across_an_unroutable_gap() { + tokio::time::pause(); const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; const NEW: &[u8] = b"NEWNEWNEWNEWNEWN"; @@ -2650,11 +2535,7 @@ mod tests { let old_server = origin.dynamic("media", sibling_route(10)).unwrap(); settle().await; - let source = moq_mux::Source::new(origin.consume(), "live"); - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; let mut config = video_config(); config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); @@ -2674,44 +2555,107 @@ mod tests { let new_server = origin.dynamic("media", sibling_route(11)).unwrap(); let mut new_media = moq_net::broadcast::Info::new().produce(); - let _ = rendition.snapshot(); - accept_sibling(&new_server, &new_media).await; - tokio::time::timeout( - Duration::from_secs(5), - origin.consume().request_broadcast("media", None), - ) - .await - .expect("the rebound sibling resolves") - .expect("the replacement is routable"); let recorder = catalog.enroll_test("video0").unwrap(); let _new_track = write_routed_media(&mut new_media, NEW, recorder, 6_000_000); - let deadline = tokio::time::Instant::now() + Duration::from_secs(5); + let _ = rendition.snapshot(); + assert!( + tokio::time::timeout(Duration::from_secs(1), new_server.requested_broadcast()) + .await + .is_err(), + "the replacement is never requested" + ); + let snapshot = rendition.snapshot(); + assert!(snapshot.segments.is_empty(), "nothing of the replacement is listed"); + assert!(snapshot.finished, "the playlist ended with its sibling"); + watcher.abort(); + drop((new_media, _new_track, new_server, catalog, live)); + } + + // A recorder waiting at the live edge ends when the bound sibling ends, with nothing else + // (a playlist request, a timeline update) touching the rendition. + #[tokio::test] + async fn a_cursor_at_the_edge_ends_when_its_sibling_ends() { + tokio::time::pause(); + const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; + + let origin = produce_origin(); + let mut live = origin.create_broadcast("live").expect("publish allowed"); + live.announce(Default::default()).expect("publish allowed"); + let catalog = moq_mux::catalog::Producer::new(&mut live, moq_mux::catalog::Config::default()).unwrap(); + let recorder = catalog.enroll_test("video0").unwrap(); + let mut media = moq_net::broadcast::Info::new().produce(); + let track = write_routed_media(&mut media, OLD, recorder, 0); + let server = origin.dynamic("media", sibling_route(10)).unwrap(); + settle().await; + + let upstream = crate::resolved(&origin, "live").await; + let mut config = video_config(); + config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); + let (rendition, watcher) = export(&upstream, &config); + accept_sibling(&server, &media).await; + tokio::time::timeout(Duration::from_secs(5), rendition.playable()) + .await + .expect("the timeline reaches the rendition"); + + let mut cursor = rendition.segments(); loop { - if !rendition.snapshot().segments.is_empty() { - break; + match tokio::time::timeout(Duration::from_secs(1), cursor.next()).await { + Ok(Ok(Some(_))) => continue, + Err(_) => break, + other => panic!( + "the cursor waits at the live edge: {:?}", + other.map(|r| r.map(|s| s.is_some())) + ), } - assert!( - tokio::time::Instant::now() < deadline, - "rows produced after the new bind were not listed" - ); - tokio::task::yield_now().await; } - let listed = rendition - .snapshot() - .segments - .into_iter() - .find(|segment| !segment.gap) - .expect("a non-gap row after the new bind") - .segment; + + // Only the sibling broadcast goes: its recorder stays, so the timeline stays quiet. + drop((server, media)); + let end = tokio::time::timeout(Duration::from_secs(5), cursor.next()) + .await + .expect("the cursor wakes when its sibling ends") + .unwrap(); + assert!(end.is_none(), "the cursor ends with its sibling"); + watcher.abort(); + drop((track, catalog, live)); + } + + // A sibling not yet announced when the rendition binds it fails that first lookup. The + // catalog's rows are still listed, and the bind is issued again until the sibling resolves. + #[tokio::test] + async fn a_sibling_announced_after_export_start_is_bound() { + tokio::time::pause(); + const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; + + let origin = produce_origin(); + let mut live = origin.create_broadcast("live").expect("publish allowed"); + live.announce(Default::default()).expect("publish allowed"); + let catalog = moq_mux::catalog::Producer::new(&mut live, moq_mux::catalog::Config::default()).unwrap(); + let recorder = catalog.enroll_test("video0").unwrap(); + let mut media = moq_net::broadcast::Info::new().produce(); + let _track = write_routed_media(&mut media, OLD, recorder, 0); + settle().await; + + let upstream = crate::resolved(&origin, "live").await; + let mut config = video_config(); + config.broadcast = Some(moq_net::path::Relative::new("media").to_owned()); + + let (rendition, watcher) = export(&upstream, &config); + tokio::time::timeout(Duration::from_secs(5), rendition.playable()) + .await + .expect("the catalog's rows are listed before the sibling resolves"); + + let server = origin.dynamic("media", sibling_route(10)).unwrap(); + let _ = rendition.snapshot(); + accept_sibling(&server, &media).await; let served = rendition - .listed_segment(listed) + .listed_segment(0) .await .unwrap() - .expect("the rebound sibling is servable after an unroutable gap"); - assert!(contains(&served, NEW)); - assert!(rendition.listed_segment(0).await.unwrap().is_none()); + .expect("the late sibling is servable"); + assert!(contains(&served, OLD)); watcher.abort(); - drop((new_media, _new_track, new_server, catalog, live)); + drop((media, _track, server, catalog, live)); } // A rendition the catalog drops must end its segment cursors. The cursor holds an @@ -2722,12 +2666,8 @@ mod tests { let origin = produce_origin(); let broadcast = origin.create_broadcast("live").expect("publish allowed"); broadcast.announce(Default::default()).expect("publish allowed"); - let source = moq_mux::Source::new(origin.consume(), "live"); settle().await; - let upstream = Upstream { - broadcast: source.broadcast().await.unwrap(), - source, - }; + let upstream = crate::resolved(&origin, "live").await; // Drive the producer directly so the catalog can be reconciled synchronously. let renditions = renditions::Producer::new(Config::default().window); @@ -2781,8 +2721,7 @@ mod tests { media.write(frame(0, true)).unwrap(); media.write(frame(2_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let mut renditions = broadcaster.renditions(); let added = tokio::time::timeout(Duration::from_secs(5), renditions.next()) @@ -2842,8 +2781,7 @@ mod tests { media.write(frame(2_000_000, true)).unwrap(); media.write(frame(4_000_000, true)).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "live"); - let broadcaster = Broadcaster::new(source, Config::default()).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); let mut renditions = broadcaster.renditions(); let rendition = match tokio::time::timeout(Duration::from_secs(5), renditions.next()) @@ -2950,14 +2888,11 @@ mod tests { broadcast: &moq_net::broadcast::Producer, catalog: &moq_mux::catalog::hang::Catalog, ) -> (renditions::Producer, tokio::task::JoinHandle<()>) { - let upstream = Upstream { - source: moq_mux::Source::new(produce_origin().consume(), "live"), - broadcast: broadcast.consume(), - }; + let upstream = moq_mux::Source::new(produce_origin().consume(), broadcast.consume()); let renditions = renditions::Producer::new(Config::default().window); renditions.sync(&upstream, catalog); let watcher = tokio::spawn(watch_timeline( - upstream.broadcast.clone(), + upstream.broadcast(), catalog.archive.clone().unwrap(), reference(catalog, None).unwrap(), renditions.fanout(), @@ -3156,12 +3091,11 @@ mod tests { media.write(payload_frame(micros, true, OLD)).unwrap(); } - let source = moq_mux::Source::new(origin.consume(), "live"); let config = Config { window: Duration::from_secs(6), ..Config::default() }; - let broadcaster = Broadcaster::new(source, config).await.unwrap(); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "live").await, config); let sequence_of = |playlist: &str| -> u64 { between(playlist, "#EXT-X-MEDIA-SEQUENCE:", "\n") .parse() diff --git a/rs/moq-hls/src/export/rendition.rs b/rs/moq-hls/src/export/rendition.rs index 42be9c16d3..15b3b4f6a0 100644 --- a/rs/moq-hls/src/export/rendition.rs +++ b/rs/moq-hls/src/export/rendition.rs @@ -13,7 +13,6 @@ use sha2::{Digest, Sha256}; use super::playlist::{Segment, Snapshot}; use super::spans::{Content, Spans}; -use super::upstream::Upstream; use super::{mpd, segments}; use crate::Result; @@ -99,7 +98,7 @@ struct Run { /// The window [`resets`](segments::Producer::resets) during which a listed segment started /// where a player can, so the master keeps advertising the rendition even once a GOP past /// [`MAX_SEGMENTS`](segments::MAX_SEGMENTS) records evicts that segment. Stale once the window - /// is cleared (a new generation, a timeline skip or rewind, a replaced sibling publisher). + /// is cleared (a new generation, a timeline skip or rewind, an ended sibling). startable: Option, } @@ -109,84 +108,91 @@ enum Config { Audio(AudioConfig), } -/// The broadcast serving this rendition's media, rebound when a sibling publisher is replaced. +/// The broadcast serving this rendition's media. A sibling that ends ends the rendition: it is +/// never rebound onto whatever serves the path next. struct Media { handle: Mutex, - /// When set, a Dropped sibling is rebound through this source rather than keeping the - /// replaced publisher's rows listed. + /// The source and reference a sibling is bound through, to re-issue a bind that failed + /// before it ever resolved. sibling: Option<(moq_mux::Source, moq_net::path::RelativeOwned)>, } struct Handle { binding: Arc, - /// Skip incoming timeline rows until the current bind resolves (set when a sibling is rebound). - waiting: bool, + /// The bound sibling ended, so the rendition lists nothing more. + ended: bool, } impl Media { - fn bind(upstream: &Upstream, rel: Option<&moq_net::path::RelativeOwned>) -> moq_mux::Result { + fn bind(source: &moq_mux::Source, rel: Option<&moq_net::path::RelativeOwned>) -> moq_mux::Result { Ok(Self { handle: Mutex::new(Handle { - binding: Arc::new(upstream.bind(rel)?), - waiting: false, + binding: Arc::new(source.bind(rel)?), + ended: false, }), - sibling: sibling(upstream, rel), + sibling: sibling(source, rel), }) } - /// Drop listed rows and rebind when a sibling publisher has been replaced; return whether - /// incoming timeline rows should be listed. + /// Whether incoming timeline rows should be listed: not once the bound sibling ended. fn admits(&self, window: &segments::Producer) -> bool { self.sync(window); - !self.handle.lock().expect("media lock poisoned").waiting + !self.handle.lock().expect("media lock poisoned").ended } - /// Drop listed rows and rebind when a sibling publisher has been replaced; return the - /// binding those rows belong to. Fetch from this handle, not a later `sync`: a replacement - /// that lands between a range lookup and `track` would otherwise serve the old groups. + /// End the window when a bound sibling has ended; return the binding listed rows belong to. fn sync(&self, window: &segments::Producer) -> Arc { + self.poll_sync(&kio::Waiter::noop(), window) + } + + /// [`sync`](Self::sync), registering `waiter` to wake when the sibling resolves or ends, so a + /// waiter on the window learns of an end that nothing else touches the rendition for. + fn poll_sync(&self, waiter: &kio::Waiter, window: &segments::Producer) -> Arc { let Some((source, rel)) = &self.sibling else { return self.handle.lock().expect("media lock poisoned").binding.clone(); }; let mut handle = self.handle.lock().expect("media lock poisoned"); - match handle.binding.poll_broadcast(&kio::Waiter::noop()) { - Poll::Ready(Ok(broadcast)) if broadcast.is_closed() => { - // The bound sibling ended, and a broadcast end carries no cause, so a rival - // publisher looks the same as a clean VOD end. Rows listed for it must not - // be served from the replacement. - window.clear(); - if let Ok(next) = source.bind(Some(rel)) { - handle.binding = Arc::new(next); + let mut reissued = false; + loop { + match handle.binding.poll_broadcast(waiter) { + Poll::Ready(Ok(broadcast)) if !handle.ended && broadcast.poll_closed(waiter).is_ready() => { + // A broadcast end carries no cause, so a clean end and a replacement look the + // same. Either way its rows can no longer be fetched, and a new instance only + // comes with a new catalog instance, so the playlist ends here. + tracing::error!(broadcast = %rel, "rendition's sibling broadcast ended; ending its playlist"); + handle.ended = true; + window.clear(); + window.end(); } - handle.waiting = true; - } - Poll::Ready(Ok(_)) => handle.waiting = false, - Poll::Ready(Err(_)) => { - // Binding never retries a failed request. Re-issue so a publisher that - // appears after this Unroutable can still recover the rendition. Do not - // set waiting: an initial Unroutable (announce still in flight) must - // still list the catalog's rows. - if let Ok(next) = source.bind(Some(rel)) { - handle.binding = Arc::new(next); + // Binding never retries a failed request. Re-issue it, once per poll, so a sibling + // still being announced binds once it is, and poll the new one. Only a bind that + // never resolved fails, and the source pins every one that did, so this never + // re-resolves a sibling. + Poll::Ready(Err(_)) if !reissued => { + reissued = true; + if let Ok(next) = source.bind(Some(rel)) { + handle.binding = Arc::new(next); + continue; + } } + _ => {} } - Poll::Pending => {} + return handle.binding.clone(); } - handle.binding.clone() } } /// A catalog `broadcast` reference that names a different path than the catalog itself. fn sibling( - upstream: &Upstream, + source: &moq_mux::Source, rel: Option<&moq_net::path::RelativeOwned>, ) -> Option<(moq_mux::Source, moq_net::path::RelativeOwned)> { - let target = upstream.source.resolve_reference(rel)?; - if upstream.source.resolve_reference(None).as_ref() == Some(&target) { + let target = source.resolve_reference(rel)?; + if source.resolve_reference(None).as_ref() == Some(&target) { return None; } Some(( - upstream.source.clone(), + source.clone(), rel.cloned().unwrap_or_else(moq_net::path::Relative::empty), )) } @@ -257,7 +263,7 @@ pub struct Rendition { /// Feeds [`spans`](Self::spans); aborted when the rendition drops. watcher: Mutex>>, /// The broadcast serving this rendition's media. A sibling is bound when the rendition is - /// created and rebound if that publisher is replaced (see [`Upstream::bind`]). + /// created, and ends the rendition when it ends. media: Media, /// The current publisher run: the embedder's label every segment URL carries (see /// [`Broadcaster::set_generation`](super::Broadcaster::set_generation)) and the init, built @@ -324,7 +330,7 @@ impl Rendition { pub(crate) fn video( name: String, config: &VideoConfig, - upstream: &Upstream, + source: &moq_mux::Source, section: Archive, clock: Option, ) -> moq_mux::Result { @@ -341,9 +347,9 @@ impl Rendition { clock, live: Arc::new(segments::Producer::new()), spans: Arc::new(Spans::new()), - catalog: upstream.broadcast.clone(), + catalog: source.broadcast(), watcher: Mutex::new(None), - media: Media::bind(upstream, config.broadcast.as_ref())?, + media: Media::bind(source, config.broadcast.as_ref())?, run: Mutex::default(), building: tokio::sync::Mutex::new(()), }) @@ -354,7 +360,7 @@ impl Rendition { pub(crate) fn audio( name: String, config: &AudioConfig, - upstream: &Upstream, + source: &moq_mux::Source, section: Archive, clock: Option, ) -> moq_mux::Result { @@ -371,9 +377,9 @@ impl Rendition { clock, live: Arc::new(segments::Producer::new()), spans: Arc::new(Spans::new()), - catalog: upstream.broadcast.clone(), + catalog: source.broadcast(), watcher: Mutex::new(None), - media: Media::bind(upstream, config.broadcast.as_ref())?, + media: Media::bind(source, config.broadcast.as_ref())?, run: Mutex::default(), building: tokio::sync::Mutex::new(()), }) @@ -535,10 +541,16 @@ impl Rendition { /// Poll until this rendition has a renderable media playlist. pub(crate) fn poll_playable(&self, waiter: &kio::Waiter) -> Poll<()> { - self.media.sync(&self.live); + self.poll_media(waiter); self.live.poll_playable(waiter) } + /// End the window if the bound sibling ended, registering `waiter` to wake when it does, and + /// return the binding this rendition's media is fetched through now. + pub(crate) fn poll_media(&self, waiter: &kio::Waiter) -> Arc { + self.media.poll_sync(waiter, &self.live) + } + /// Render this rendition's media playlist from the current timeline window, or `None` when /// there is nothing to serve yet (no segment, and the broadcast has not ended) -- a /// playlist with no segments confuses players, so a server should treat `None` as "not @@ -667,7 +679,7 @@ impl Rendition { if self.kind == Kind::Audio { return self.poll_playable(waiter); } - self.media.sync(&self.live); + self.poll_media(waiter); if self.run().startable == Some(self.live.resets()) { return self.poll_playable(waiter); } @@ -799,11 +811,6 @@ impl Rendition { }) } - /// The binding this rendition's media is fetched through now. - pub(crate) fn binding(&self) -> Arc { - self.media.sync(&self.live) - } - /// Subscribe to the media track on `binding` once it resolves: `None` when it never will. pub(crate) fn poll_subscribe( &self, diff --git a/rs/moq-hls/src/export/renditions.rs b/rs/moq-hls/src/export/renditions.rs index 867f9d383a..eb2caed6d7 100644 --- a/rs/moq-hls/src/export/renditions.rs +++ b/rs/moq-hls/src/export/renditions.rs @@ -18,7 +18,6 @@ use std::time::{Duration, SystemTime}; use moq_mux::catalog::hang::Catalog; use moq_mux::timeline::Entry; -use super::Upstream; use super::rendition::{Kind, Rendition}; use super::segments::{self, Discontinuities, Reference}; @@ -400,11 +399,11 @@ impl Producer { /// entry): without one there is nothing to render playlists from, so the whole catalog is /// skipped with a warning. /// - /// `upstream` carries the broadcast the snapshot was read from, so each rendition serves media + /// `source` holds the broadcast the snapshot was read from, so each rendition serves media /// from that same broadcast rather than whatever is at the path by the time it is asked. A /// rendition whose `broadcast` reference escapes above the origin root names no broadcast at /// all, so it is dropped with a warning rather than served. - pub fn sync(&self, upstream: &Upstream, catalog: &Catalog) { + pub fn sync(&self, source: &moq_mux::Source, catalog: &Catalog) { let Some(archive) = catalog.archive.clone() else { if !catalog.video.renditions.is_empty() || !catalog.audio.renditions.is_empty() { tracing::warn!("catalog advertises no archive; its renditions can't be served as HLS"); @@ -451,7 +450,7 @@ impl Producer { rendition.refresh(video.bitrate, video.framerate); continue; } - let rendition = match Rendition::video(name.clone(), video, upstream, section.clone(), clock) { + let rendition = match Rendition::video(name.clone(), video, source, section.clone(), clock) { Ok(rendition) => Arc::new(rendition), Err(err) => { tracing::warn!(rendition = name, %err, "ignoring unservable video rendition"); @@ -467,7 +466,7 @@ impl Producer { rendition.refresh(audio.bitrate, None); continue; } - let rendition = match Rendition::audio(name.clone(), audio, upstream, section.clone(), clock) { + let rendition = match Rendition::audio(name.clone(), audio, source, section.clone(), clock) { Ok(rendition) => Arc::new(rendition), Err(err) => { tracing::warn!(rendition = name, %err, "ignoring unservable audio rendition"); diff --git a/rs/moq-hls/src/export/segments.rs b/rs/moq-hls/src/export/segments.rs index c7b32ee763..050074a1ce 100644 --- a/rs/moq-hls/src/export/segments.rs +++ b/rs/moq-hls/src/export/segments.rs @@ -52,7 +52,7 @@ struct State { /// Numbers the broadcast's content timeline: the sequence bumps wherever the timeline breaks. /// /// Held once by the timeline fanout rather than per rendition, so every rendition stamps a -/// record with the same sequence even when one isn't listing rows (waiting to rebind) or +/// record with the same sequence even when one isn't listing rows (its sibling ended) or /// cleared its own window. #[derive(Default)] pub(crate) struct Discontinuities { @@ -445,7 +445,7 @@ pub struct Consumer { /// The live subscription a [`Consumer`] holds on its rendition's media track. struct Feed { - /// What it subscribed through: a rebound sibling publisher gets a fresh feed. + /// What it subscribed through: a re-issued sibling bind gets a fresh feed. binding: Arc, state: FeedState, } @@ -516,7 +516,8 @@ impl Consumer { /// Hold a live subscription on the media track the rendition is bound to now. fn poll_feed(&mut self, waiter: &kio::Waiter) { - let binding = self.rendition.binding(); + // Also ends the window on an ended sibling, which nothing else may wake this cursor for. + let binding = self.rendition.poll_media(waiter); let feed = match &mut self.feed { Some(feed) if Arc::ptr_eq(&feed.binding, &binding) => feed, feed => feed.insert(Feed { diff --git a/rs/moq-hls/src/export/upstream.rs b/rs/moq-hls/src/export/upstream.rs deleted file mode 100644 index e3e836e28c..0000000000 --- a/rs/moq-hls/src/export/upstream.rs +++ /dev/null @@ -1,67 +0,0 @@ -//! The broadcast an export is bound to, held rather than looked up again by path. - -/// The one broadcast an export is bound to: the already-resolved catalog consumer, plus the -/// [`moq_mux::Source`] it came from so a rendition can still follow a cross-broadcast reference. -/// -/// The two travel together because resolving the catalog path by name is not idempotent. Once -/// a broadcast ends, its path can be published again as a brand new broadcast, such as a -/// publisher reconnecting after its old route is gone, with its group numbering restarted. A -/// rendition that looked its media up by path would then serve the replacement's groups under -/// the manifest, segment numbering, and `PROGRAM-DATE-TIME` of the broadcast it replaced. -#[derive(Clone)] -pub(crate) struct Upstream { - /// Origin plus catalog path, for resolving a rendition's sibling `broadcast` reference. - pub source: moq_mux::Source, - /// The catalog broadcast, resolved once when the export started. - pub broadcast: moq_net::broadcast::Consumer, -} - -impl Upstream { - /// Bind a rendition whose catalog `broadcast` field is `rel` to the broadcast serving it. - /// - /// Every reference resolving to the catalog path reuses the broadcast already in hand. - /// A sibling request starts when the rendition is created, before it can list a segment, - /// and subsequent media fetches reuse that request's result. - /// - /// Fails when the reference escapes above the origin root and so names no broadcast at all. - pub fn bind(&self, rel: Option<&moq_net::path::RelativeOwned>) -> moq_mux::Result { - if self.source.resolve_reference(rel) == self.source.resolve_reference(None) { - Ok(moq_mux::Binding::new(self.broadcast.clone())) - } else { - self.source.bind(rel) - } - } -} - -#[cfg(test)] -mod tests { - use super::*; - - #[tokio::test] - async fn self_references_keep_the_catalog_broadcast_after_replacement() { - let (origin, driver) = moq_net::origin::Producer::new(moq_net::origin::Config::default()); - let driver = tokio::spawn(moq_net::time::run(driver)); - let old = origin.publish("a/live", Default::default()).unwrap(); - let source = moq_mux::Source::new(origin.consume(), "a/live"); - let upstream = Upstream { - source, - broadcast: old.consume(), - }; - drop(old); - let _replacement = origin.publish("a/live", Default::default()).unwrap(); - for _ in 0..10 { - tokio::task::yield_now().await; - } - assert!(!upstream.source.broadcast().await.unwrap().is_closed()); - - for reference in [None, Some(""), Some("live"), Some("./live"), Some("../a/live")] { - let rel = reference.map(|value| moq_net::path::Relative::new(value).to_owned()); - let bound = upstream.bind(rel.as_ref()).unwrap().broadcast().await.unwrap(); - assert!( - bound.is_closed(), - "{reference:?} must keep the original catalog broadcast" - ); - } - driver.abort(); - } -} diff --git a/rs/moq-hls/src/lib.rs b/rs/moq-hls/src/lib.rs index cb79fff34f..094928d17f 100644 --- a/rs/moq-hls/src/lib.rs +++ b/rs/moq-hls/src/lib.rs @@ -38,3 +38,14 @@ pub use server::Server; /// breaking change for this crate. #[cfg(feature = "server")] pub use axum; + +/// Test helper: a [`moq_mux::Source`] for the broadcast at `path`, resolved through `origin`. +#[cfg(test)] +pub(crate) async fn resolved(origin: &moq_net::origin::Producer, path: &str) -> moq_mux::Source { + let consumer = origin.consume(); + let broadcast = consumer + .request_broadcast(path, None) + .await + .expect("the broadcast resolves"); + moq_mux::Source::new(consumer, broadcast) +} diff --git a/rs/moq-hls/src/server/mod.rs b/rs/moq-hls/src/server/mod.rs index 2dee9c18c2..2c820f3095 100644 --- a/rs/moq-hls/src/server/mod.rs +++ b/rs/moq-hls/src/server/mod.rs @@ -111,19 +111,22 @@ impl Server { } } - // Confirm a route covers the broadcast (and it is in scope) before building a - // broadcaster; `Broadcaster::new` re-resolves it through the origin, which also - // lets a rendition's catalog `broadcast` field reference a sibling broadcast. + // Confirm a route covers the broadcast (and it is in scope) before resolving it. The + // source resolves a rendition's sibling `broadcast` reference through the same origin. tokio::time::timeout(RESOLVE_TIMEOUT, self.inner.origin.routed(name)) .await .ok() .flatten()?; - let source = moq_mux::Source::new(self.inner.origin.consume(), name); - let broadcaster = Broadcaster::new(source, self.inner.config.clone()) + let origin = self.inner.origin.consume(); + // A covering dynamic route may never answer, so the request is bounded too. + let broadcast = tokio::time::timeout(RESOLVE_TIMEOUT, origin.request_broadcast(name, None)) .await - .map_err(|err| tracing::warn!(%err, %name, "failed to resolve broadcast catalog")) + .map_err(|_| tracing::warn!(%name, "timed out resolving broadcast")) + .ok()? + .map_err(|err| tracing::warn!(%err, %name, "failed to resolve broadcast")) .ok()?; + let broadcaster = Broadcaster::new(moq_mux::Source::new(origin, broadcast), self.inner.config.clone()); let mut broadcasters = self.inner.broadcasters.lock().unwrap(); if let Some(existing) = broadcasters.get(name) { @@ -256,10 +259,7 @@ mod tests { let producer = origin.create_broadcast("gone").expect("publish allowed"); producer.announce(Default::default()).expect("publish allowed"); settle().await; - let source = moq_mux::Source::new(origin.consume(), "gone"); - let broadcaster = Broadcaster::new(source, Config::default()) - .await - .expect("catalog broadcast resolves while announced"); + let broadcaster = Broadcaster::new(crate::resolved(&origin, "gone").await, Config::default()); // Close the publisher so the resolved broadcast (and the broadcaster) reports closed. producer.close(); settle().await; @@ -296,9 +296,7 @@ mod tests { let new_producer = origin.create_broadcast("live").expect("publish allowed"); new_producer.announce(Default::default()).expect("publish allowed"); settle().await; - let new = Broadcaster::new(moq_mux::Source::new(origin.consume(), "live"), Config::default()) - .await - .expect("catalog broadcast resolves while announced"); + let new = Broadcaster::new(crate::resolved(&origin, "live").await, Config::default()); server .inner diff --git a/rs/moq-hls/src/server/routes.rs b/rs/moq-hls/src/server/routes.rs index d30f64d046..d12c56d616 100644 --- a/rs/moq-hls/src/server/routes.rs +++ b/rs/moq-hls/src/server/routes.rs @@ -712,14 +712,12 @@ mod tests { .await .expect("announce timed out") .expect("routed"); - let source = moq_mux::Source::new(origin, "live"); - let broadcaster = tokio::time::timeout( - TIMEOUT, - crate::export::Broadcaster::new(source, crate::export::Config::default()), - ) - .await - .expect("broadcaster timed out") - .expect("broadcaster"); + let remote = tokio::time::timeout(TIMEOUT, origin.request_broadcast("live", None)) + .await + .expect("request timed out") + .expect("remote broadcast"); + let source = moq_mux::Source::new(origin, remote); + let broadcaster = crate::export::Broadcaster::new(source, crate::export::Config::default()); // Recorded as soon as the catalog lists it, before any media, as a recorder does. let mut renditions = broadcaster.renditions(); let rendition = match tokio::time::timeout(TIMEOUT, renditions.next()).await { diff --git a/rs/moq-mux/src/binary.rs b/rs/moq-mux/src/binary.rs index 6e23d6b2a7..79550f3528 100644 --- a/rs/moq-mux/src/binary.rs +++ b/rs/moq-mux/src/binary.rs @@ -790,7 +790,7 @@ mod test { #[tokio::test] async fn roundtrips_through_a_catalog_consumer() { let (mut broadcast, catalog) = catalog(); - let source = crate::source::announced(&broadcast.consume()); + let source = crate::source::announced(&broadcast.consume()).await; let mut consumer = catalog.consume().unwrap(); let mut telemetry = catalog diff --git a/rs/moq-mux/src/codec/h264/export.rs b/rs/moq-mux/src/codec/h264/export.rs index f04dbc1cb5..cde920df67 100644 --- a/rs/moq-mux/src/codec/h264/export.rs +++ b/rs/moq-mux/src/codec/h264/export.rs @@ -291,7 +291,7 @@ mod tests { let consumer = broadcast.consume(); // The whole track is written before the exporter runs, so it needs a budget // wide enough to read it: the default skips everything but the live edge. - let mut export = Export::new(crate::source::announced(&consumer), Once(Some(catalog))) + let mut export = Export::new(crate::source::announced(&consumer).await, Once(Some(catalog))) .with_max_delay(std::time::Duration::from_secs(30)); let frame0 = export.next().await.unwrap().expect("first frame"); @@ -341,7 +341,7 @@ mod tests { group.finish().unwrap(); track.finish().unwrap(); let consumer = broadcast.consume(); - let mut exporter = Export::new(crate::source::announced(&consumer), Once(Some(catalog))); + let mut exporter = Export::new(crate::source::announced(&consumer).await, Once(Some(catalog))); let result = exporter.next().await; if in_band { let expected: Vec<_> = nals diff --git a/rs/moq-mux/src/codec/h265/export.rs b/rs/moq-mux/src/codec/h265/export.rs index 309104ee27..5e89e73eb8 100644 --- a/rs/moq-mux/src/codec/h265/export.rs +++ b/rs/moq-mux/src/codec/h265/export.rs @@ -280,7 +280,7 @@ mod tests { let consumer = broadcast.consume(); // The whole track is written before the exporter runs, so it needs a budget // wide enough to read it: the default skips everything but the live edge. - let mut export = Export::new(crate::source::announced(&consumer), Once(Some(catalog))) + let mut export = Export::new(crate::source::announced(&consumer).await, Once(Some(catalog))) .with_max_delay(std::time::Duration::from_secs(30)); let frame0 = export.next().await.unwrap().expect("first frame"); @@ -326,7 +326,7 @@ mod tests { group.finish().unwrap(); track.finish().unwrap(); let consumer = broadcast.consume(); - let mut exporter = Export::new(crate::source::announced(&consumer), Once(Some(catalog))); + let mut exporter = Export::new(crate::source::announced(&consumer).await, Once(Some(catalog))); let result = exporter.next().await; if in_band { let expected: Vec<_> = nals diff --git a/rs/moq-mux/src/container/flv/export_test.rs b/rs/moq-mux/src/container/flv/export_test.rs index 03532bd4b5..c245ee9e37 100644 --- a/rs/moq-mux/src/container/flv/export_test.rs +++ b/rs/moq-mux/src/container/flv/export_test.rs @@ -148,7 +148,7 @@ async fn export_roundtrips_through_import() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; // The export must be a real FLV stream. @@ -186,7 +186,7 @@ async fn export_emits_sequence_headers_and_frames() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)) + let exporter = open_export(crate::source::announced(&consumer).await) .await .with_max_delay(RECORDING_MAX_DELAY); let exported = drain_export(exporter, importer).await; @@ -268,7 +268,7 @@ async fn export_roundtrips_enhanced() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; let tags = parse_tags(&exported); @@ -332,7 +332,7 @@ async fn export_roundtrips_mp3() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; // The audio is muxed as a legacy SoundFormat 2 (MP3) tag, no sequence header. @@ -386,7 +386,7 @@ async fn export_roundtrips_av1() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; let tags = parse_tags(&exported); @@ -442,7 +442,7 @@ async fn export_roundtrips_ac3() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; let tags = parse_tags(&exported); assert!( @@ -477,7 +477,7 @@ async fn export_roundtrips_eac3() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; let tags = parse_tags(&exported); assert!( @@ -623,7 +623,7 @@ async fn drain_to_end(mut exporter: Export, keepal async fn export_multitrack_roundtrips_all_renditions() { let (consumer, descriptions, keepalive) = build_multitrack_broadcast(); - let exporter = open_export(crate::source::announced(&consumer)) + let exporter = open_export(crate::source::announced(&consumer).await) .await .with_multitrack(true); let exported = drain_to_end(exporter, keepalive).await; @@ -682,7 +682,7 @@ async fn export_multitrack_roundtrips_all_renditions() { async fn export_without_multitrack_keeps_best_rendition() { let (consumer, descriptions, keepalive) = build_multitrack_broadcast(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_to_end(exporter, keepalive).await; // No multitrack framing: the single video track is a legacy AVC tag. @@ -775,7 +775,7 @@ async fn export_without_multitrack_keeps_best_audio_rendition() { } catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_to_end(exporter, (producer, catalog, tracks)).await; let mut bcast2 = moq_net::broadcast::Info::new().produce(); @@ -825,7 +825,7 @@ async fn export_picks_the_best_selected_rendition() { let select = crate::select::Broadcast::default() .video(crate::select::Video::default().name(small)) .audio(crate::select::Audio::default()); - let source = crate::source::announced(&consumer); + let source = crate::source::announced(&consumer).await; let catalog = source .catalog::<()>(crate::catalog::CatalogFormat::Hang) .await @@ -883,7 +883,7 @@ async fn export_rebinds_to_a_better_rendition_before_the_header() { crate::catalog::hang::Container::Legacy(crate::container::Kind::Data), ); - let mut exporter = open_export(crate::source::announced(&consumer)).await; + let mut exporter = open_export(crate::source::announced(&consumer).await).await; assert!( tokio::time::timeout(Duration::from_millis(100), exporter.next()) .await @@ -1007,7 +1007,7 @@ async fn export_rebinds_to_a_better_audio_rendition_before_the_header() { crate::catalog::hang::Container::Legacy(crate::container::Kind::Data), ); - let mut exporter = open_export(crate::source::announced(&consumer)).await; + let mut exporter = open_export(crate::source::announced(&consumer).await).await; assert!( tokio::time::timeout(Duration::from_millis(100), exporter.next()) .await @@ -1097,7 +1097,7 @@ async fn export_preserves_timestamps() { importer.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_export(exporter, importer).await; let tags = parse_tags(&exported); @@ -1178,7 +1178,7 @@ async fn export_authors_dts_and_composition_time_for_reordered_avc() { audio.finish().unwrap(); catalog.finish().unwrap(); - let exporter = open_export(crate::source::announced(&consumer)).await; + let exporter = open_export(crate::source::announced(&consumer).await).await; let exported = drain_exporter_chunks(exporter, 6).await; let tags = parse_tags(&exported); @@ -1270,7 +1270,7 @@ async fn a_track_leaving_the_catalog_is_read_to_its_end() { video.write(frame(0, &[0, 0, 0, 1, 0x65], true)).unwrap(); audio.write(frame(0, &[0xde, 0xad], true)).unwrap(); - let mut exporter = open_export(crate::source::announced(&consumer)) + let mut exporter = open_export(crate::source::announced(&consumer).await) .await .with_max_delay(RECORDING_MAX_DELAY); let mut exported = Vec::new(); diff --git a/rs/moq-mux/src/container/fmp4/export_test.rs b/rs/moq-mux/src/container/fmp4/export_test.rs index c6cdeb3d63..efb7c1674a 100644 --- a/rs/moq-mux/src/container/fmp4/export_test.rs +++ b/rs/moq-mux/src/container/fmp4/export_test.rs @@ -23,7 +23,7 @@ async fn avc3_source_to_cmaf_export_roundtrip() { live.track.write(video_frame(0, true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter) .await .init() @@ -96,7 +96,7 @@ async fn legacy_aac_source_to_cmaf_export_synthesizes_esds() { live.track.write(raw_frame(0, &[0x01, 0x02, 0x03, 0x04], true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter) .await .init() @@ -161,7 +161,7 @@ async fn vp8_source_to_cmaf_export_synthesizes_vp08() { .unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter) .await .init() @@ -209,7 +209,7 @@ async fn dimensionless_video_waits_for_catalog_geometry() { // so the synthetic producer accepts it before the catalog update arrives. live.track.write(raw_frame(0, &[0x31, 0x00, 0x00], true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let pending = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()).await; assert!( pending.is_err(), @@ -259,7 +259,7 @@ async fn dimensionless_video_rejects_a_malformed_description() { catalog.modify().unwrap().video.renditions.insert(name, config); }); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let error = exporter.next().await.expect_err("malformed fixed description"); assert!(matches!( error, @@ -293,7 +293,7 @@ async fn vp9_source_to_cmaf_export_synthesizes_vp09() { live.track.write(raw_frame(0, &[0x82, 0x49, 0x83, 0x42], true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter) .await .init() @@ -358,7 +358,7 @@ async fn av1_source_to_cmaf_export_synthesizes_av01() { live.track.write(raw_frame(0, &[0x12, 0x00, 0x0a, 0x0b], true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter) .await .init() @@ -420,7 +420,7 @@ async fn cmaf_source_to_cmaf_export_passthrough() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer), catalog_stream); + let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer).await, catalog_stream); let init = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) .await @@ -480,7 +480,7 @@ async fn single_track_export_init_matches_fragment_track_id() { .await .expect("catalog consumer"); let selected = catalog_stream.select(crate::select::Broadcast::default().audio(crate::select::Audio::default())); - let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer), selected) + let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer).await, selected) .with_max_delay(RECORDING_MAX_DELAY); let init = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) @@ -542,7 +542,7 @@ async fn next_chunk_reports_segment_metadata() { live.track.finish().unwrap(); // Sub-GOP cap so GOP 0 splits into two parts (the trailing part non-independent). - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::from_millis(20)) .with_max_delay(RECORDING_MAX_DELAY); @@ -583,7 +583,7 @@ async fn chunk_data_reaches_both_variants() { .cut(Some(moq_net::Timestamp::from_micros(33_000).unwrap())) .unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::ZERO); let init = chunk_now(&mut exporter).await; @@ -601,7 +601,7 @@ async fn chunk_data_reaches_both_variants() { async fn zero_fragment_duration_waits_for_the_exact_video_endpoint() { let mut live = Live::avc3(); live.track.write(video_frame(0, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::ZERO); chunk_now(&mut exporter).await.init().expect("init"); assert!(drain_now(&mut exporter).await.is_empty(), "unknown duration must wait"); @@ -619,7 +619,7 @@ async fn zero_fragment_duration_waits_for_the_exact_video_endpoint() { async fn zero_fragment_duration_uses_the_successor_then_the_cut() { let mut live = Live::avc3(); live.track.write(video_frame(0, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::ZERO); chunk_now(&mut exporter).await.init().expect("init"); assert!(drain_now(&mut exporter).await.is_empty()); @@ -655,7 +655,7 @@ async fn ntsc_tail_uses_a_representable_catalog_cadence() { live.track.write(video_frame(0, true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); chunk_now(&mut exporter) .await .init() @@ -679,7 +679,7 @@ async fn unusable_framerate_uses_the_standard_fallback_rate() { live.track.write(raw_frame(0, &[0x82, 0x00], true)).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); chunk_now(&mut exporter) .await .init() @@ -718,7 +718,7 @@ async fn one_packet_audio_group_is_timed_by_the_catalog() { .write(raw_frame(500_000, &[0x01, 0x02, 0x03, 0x04], true)) .unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter) .await @@ -750,7 +750,7 @@ async fn audio_fragment_is_the_publisher_group() { .unwrap(); } - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter) .await @@ -766,7 +766,7 @@ async fn audio_fragment_is_the_publisher_group() { fragment.duration ); // The one-frame-per-fragment mode is the explicit way back to per-packet output. - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY) .with_fragment_duration(std::time::Duration::ZERO); chunk_now(&mut exporter).await.init().expect("init"); @@ -782,7 +782,7 @@ async fn closed_audio_group_flushes_without_a_successor() { let mut live = Live::audio(AudioConfig::new(AudioCodec::Opus, 48_000, 2)); live.track.write(raw_frame(0, &[0x08, 0xaa], true)).unwrap(); live.track.write(raw_frame(20_000, &[0x08, 0xbb], false)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); assert!( @@ -815,7 +815,7 @@ async fn uncut_audio_waits_for_the_explicit_cap() { .unwrap(); } - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); assert!( @@ -823,7 +823,7 @@ async fn uncut_audio_waits_for_the_explicit_cap() { "an open group is not a fragment yet" ); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY) .with_fragment_duration(std::time::Duration::from_millis(200)); chunk_now(&mut exporter).await.init().expect("init"); @@ -848,7 +848,7 @@ async fn opus_frame_duration_from_toc() { .write(raw_frame(20_000, &[0x08, 0xaa, 0xbb, 0xcc], true)) .unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter) .await @@ -987,8 +987,9 @@ fn a_long_gop_does_not_cost_a_stack_frame_per_sample() { } live.track.write(video_frame(600 * 16_666, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) - .with_max_delay(RECORDING_MAX_DELAY); + let mut exporter = + crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) + .with_max_delay(RECORDING_MAX_DELAY); assert!( chunk_now(&mut exporter).await.init().is_some(), "first chunk must be the init segment" @@ -1046,7 +1047,7 @@ async fn export_av(group: u64) -> (Vec<(u32, usize)>, (u32, u32)) { live.track.finish().unwrap(); audio.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); let init = chunk_now(&mut exporter).await.init().expect("init"); let ids = track_ids(&init); @@ -1113,7 +1114,7 @@ async fn video_that_ends_first_writes_its_tail_in_order() { .unwrap(); } - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); let init = chunk_now(&mut exporter).await.init().expect("init"); let (video_id, audio_id) = track_ids(&init); @@ -1356,7 +1357,7 @@ async fn video_cut_preserves_the_exact_duration_without_a_successor() { let mut live = Live::avc3(); live.track.write(video_frame(0, true)).unwrap(); live.track.write(video_frame(10_000, false)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); assert!(drain_now(&mut exporter).await.is_empty()); @@ -1414,7 +1415,7 @@ async fn audio_before_the_first_keyframe_is_kept() { // still waits for the SPS. A baseline catalog would be described immediately. wait_for_sps(&mut live); // The CLI's default max delay: a parked reader falls behind it within a second. - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(std::time::Duration::from_millis(500)); // Two seconds of audio, published live with no video yet. @@ -1457,7 +1458,7 @@ async fn returning_rendition_keeps_its_track_id() { } write_audio(&mut audio, 0, 6); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter).await.init().expect("init"); let (video_id, audio_id) = track_ids(&init); let mut fragments = drain_now(&mut exporter).await; @@ -1503,7 +1504,7 @@ fn add_opus(live: &mut Live) -> (crate::container::Producer = drain_now(&mut exporter) @@ -1925,7 +1926,7 @@ async fn an_in_band_keyframe_without_parameter_sets_fails() { live.track.write(annexb_frame(0, &[IDR])).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); let err = error_now(&mut exporter).await; @@ -1945,7 +1946,7 @@ async fn an_incompatible_return_fails() { let name = audio.name().to_string(); live.track.write(video_frame(0, true)).unwrap(); write_audio(&mut audio, 0, 6); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); chunk_now(&mut exporter).await.init().expect("init"); let mut config = live.catalog.modify().unwrap().audio.renditions.remove(&name).unwrap(); @@ -1972,7 +1973,7 @@ async fn a_replayed_return_fails() { let name = audio.name().to_string(); live.track.write(video_frame(0, true)).unwrap(); write_audio(&mut audio, 0, 11); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); drain_now(&mut exporter).await; @@ -2003,7 +2004,7 @@ async fn a_track_that_never_describes_itself_fails() { // The baseline catalog would init before this IDR, which carries no SPS. wait_for_sps(&mut live); live.track.write(raw_frame(0, IDR_ONLY, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); // Live audio, one second at a time, until the export gives up on the video. @@ -2034,7 +2035,7 @@ async fn an_uncut_audio_backlog_is_bounded() { let video = live.track.name().to_string(); wait_for_sps(&mut live); live.track.write(raw_frame(0, IDR_ONLY, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); // One audio group that never closes, one second at a time. @@ -2091,7 +2092,7 @@ async fn hevc_main10_is_described_from_its_sps() { config.container = Container::Legacy; catalog.modify().unwrap().video.renditions.insert(name, config); }); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let pending = tokio::time::timeout(std::time::Duration::from_millis(1), exporter.next_chunk()).await; assert!(pending.is_err(), "an init before the SPS: {pending:?}"); @@ -2193,7 +2194,7 @@ async fn cmaf_source_exports_at_the_frame_timestamp() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer), catalog_stream) + let mut exporter = crate::container::fmp4::Export::new(crate::source::announced(&consumer).await, catalog_stream) .with_max_delay(RECORDING_MAX_DELAY); let init = chunk_now(&mut exporter).await.init().expect("init"); let fragments = drain_now(&mut exporter).await; @@ -2223,7 +2224,7 @@ fn timescale(init: &Bytes) -> u32 { #[tokio::test(start_paused = true)] async fn annexb_init_is_written_before_the_first_keyframe() { let live = Live::avc3(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter).await.init().expect("init before any frame"); let mp4_atom::Codec::Avc3(avc3) = video_codec(&init) else { panic!("expected an avc3 sample entry"); @@ -2242,7 +2243,7 @@ async fn a_bare_keyframe_is_prefixed_with_cached_parameter_sets() { live.track.write(annexb_frame(33_000, &[IDR])).unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await) .with_max_delay(RECORDING_MAX_DELAY); chunk_now(&mut exporter).await.init().expect("init"); @@ -2264,7 +2265,7 @@ async fn a_returning_rendition_may_restart_its_sps() { let mut live = Live::avc3(); let name = live.track.name().to_string(); live.track.write(video_frame(0, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); chunk_now(&mut exporter).await.init().expect("init"); let config = live.catalog.modify().unwrap().video.renditions.remove(&name).unwrap(); @@ -2290,7 +2291,7 @@ async fn a_changed_catalog_profile_fails() { let mut live = Live::avc3(); let name = live.track.name().to_string(); live.track.write(video_frame(0, true)).unwrap(); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); chunk_now(&mut exporter).await.init().expect("init"); { @@ -2315,7 +2316,7 @@ async fn a_changed_catalog_profile_fails() { async fn high10_waits_for_its_sps() { let mut live = Live::avc3(); wait_for_sps(&mut live); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let pending = tokio::time::timeout(std::time::Duration::from_millis(1), exporter.next_chunk()).await; assert!(pending.is_err(), "an init before the SPS: {pending:?}"); @@ -2346,7 +2347,7 @@ async fn high_profile_inits_from_the_catalog() { h264.constraints = 0; h264.level = 0x1f; } - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter).await.init().expect("init before any frame"); let mp4_atom::Codec::Avc3(avc3) = video_codec(&init) else { panic!("expected an avc3 sample entry"); @@ -2376,7 +2377,7 @@ async fn annexb_without_dimensions_waits_for_the_sps() { config.framerate = Some(30.0); catalog.modify().unwrap().video.renditions.insert(name, config); }); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let pending = tokio::time::timeout(std::time::Duration::from_millis(1), exporter.next_chunk()).await; assert!(pending.is_err(), "an init before dimensions: {pending:?}"); @@ -2413,7 +2414,7 @@ async fn hevc_main_inits_from_the_catalog() { config.coded_height = Some(240); catalog.modify().unwrap().video.renditions.insert(name, config); }); - let mut exporter = crate::container::fmp4::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::fmp4::Export::new(live.source().await, live.catalog_stream().await); let init = chunk_now(&mut exporter).await.init().expect("init before any frame"); let mp4_atom::Codec::Hev1(hev1) = video_codec(&init) else { panic!("expected an hev1 sample entry"); diff --git a/rs/moq-mux/src/container/mkv/export_test.rs b/rs/moq-mux/src/container/mkv/export_test.rs index 9e29297e98..ed46c8f3fc 100644 --- a/rs/moq-mux/src/container/mkv/export_test.rs +++ b/rs/moq-mux/src/container/mkv/export_test.rs @@ -42,7 +42,7 @@ async fn export_header_roundtrip_vp9_opus() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer), catalog_stream); + let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer).await, catalog_stream); // First `next()` should give us the header (EBML + Segment-start + Info + Tracks). let header = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) @@ -270,7 +270,7 @@ async fn export_header_roundtrip_mp3() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer), catalog_stream); + let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer).await, catalog_stream); let header = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) .await @@ -349,7 +349,7 @@ async fn export_waits_for_catalog_before_header() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .unwrap(); - let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer), catalog_stream); + let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer).await, catalog_stream); // next() must remain pending (timing out), not surface a "no catalog // snapshot" error from a vacuously-ready empty track set. @@ -385,7 +385,7 @@ async fn export_derives_video_geometry_before_header() { .unwrap(); live.track.finish().unwrap(); - let mut exporter = crate::container::mkv::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::mkv::Export::new(live.source().await, live.catalog_stream().await); let header = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) .await .expect("exporter timed out") @@ -421,7 +421,7 @@ async fn export_emits_blocks_for_each_frame() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer), catalog_stream) + let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer).await, catalog_stream) // Use per-frame clustering so each frame is observable as its own // Cluster chunk; batching is exercised in a dedicated test below. .with_fragment_duration(std::time::Duration::ZERO) @@ -501,7 +501,7 @@ async fn export_rejects_cmaf_track() { catalog.modify().unwrap().video.renditions.insert(name, config); }); - let mut exporter = crate::container::mkv::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::mkv::Export::new(live.source().await, live.catalog_stream().await); let result = tokio::time::timeout(std::time::Duration::from_secs(1), exporter.next()) .await .expect("exporter timed out"); @@ -526,7 +526,7 @@ async fn export_avc3_source_synthesizes_avcc_and_length_prefixes() { live.track.finish().unwrap(); live.catalog.finish().unwrap(); - let mut exporter = crate::container::mkv::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::mkv::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::ZERO); let mut exported: Vec = Vec::new(); @@ -662,7 +662,7 @@ async fn export_fragment_duration_batches_blocks() { let catalog_stream = crate::catalog::Consumer::<()>::new(&consumer, crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer"); - let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer), catalog_stream) + let mut exporter = crate::container::mkv::Export::new(crate::source::announced(&consumer).await, catalog_stream) .with_fragment_duration(std::time::Duration::from_secs(2)) .with_max_delay(RECORDING_MAX_DELAY); let mut exported: Vec = Vec::new(); @@ -705,7 +705,7 @@ async fn zero_fragment_duration_emits_without_successor() { let mut live = Live::avc3(); live.track.write(video_frame(0, true)).unwrap(); - let mut exporter = crate::container::mkv::Export::new(live.source(), live.catalog_stream().await) + let mut exporter = crate::container::mkv::Export::new(live.source().await, live.catalog_stream().await) .with_fragment_duration(std::time::Duration::ZERO); let header = chunk_now(&mut exporter).await; let cluster = chunk_now(&mut exporter).await; @@ -719,7 +719,7 @@ async fn reconfigured_zero_duration_drains_open_cluster() { live.track.write(video_frame(33_000, false)).unwrap(); // Default per-GOP clustering: both frames land in one open cluster. - let mut exporter = crate::container::mkv::Export::new(live.source(), live.catalog_stream().await); + let mut exporter = crate::container::mkv::Export::new(live.source().await, live.catalog_stream().await); let header = chunk_now(&mut exporter).await; tokio::time::timeout(std::time::Duration::from_millis(1), exporter.next()) .await diff --git a/rs/moq-mux/src/container/source.rs b/rs/moq-mux/src/container/source.rs index ba386ed32a..9c25239af3 100644 --- a/rs/moq-mux/src/container/source.rs +++ b/rs/moq-mux/src/container/source.rs @@ -56,7 +56,7 @@ impl VideoTransform { enum SourceState { /// Waiting for the target broadcast (the catalog broadcast, or a cross-broadcast /// reference) to resolve; the track (by name) is subscribed once it does. - Requesting(kio::Pending, String), + Requesting(crate::Binding, String), /// Waiting for the subscription to resolve (blocks on the publisher's SUBSCRIBE_OK). Subscribing(kio::Pending), /// The resolved consumer, reading frames. Boxed because it's much larger than @@ -329,7 +329,7 @@ impl ExportSource { let SourceState::Requesting(pending, name) = &self.state else { unreachable!("just matched Requesting"); }; - (ready!(pending.poll_ok(waiter))?, name.clone()) + (ready!(pending.poll_broadcast(waiter))?, name.clone()) }; // A zero budget asks only for the newest group. let max_delay = match self.live { @@ -534,7 +534,7 @@ mod tests { #[tokio::test] async fn escaping_reference_skips_the_rendition() { let live = Live::avc3(); - let source = live.source(); + let source = live.source().await; let max_delay = std::time::Duration::ZERO; let escaping = |result: Result, crate::Error>, what: &str| match result { @@ -564,7 +564,7 @@ mod tests { #[tokio::test] async fn legal_reference_keeps_the_rendition() { let live = Live::avc3(); - let source = live.source(); + let source = live.source().await; let max_delay = std::time::Duration::ZERO; for reference in [None, Some(""), Some("./source"), Some("sub"), Some(".")] { @@ -583,7 +583,7 @@ mod tests { async fn latency_is_sent_with_the_initial_subscription() { let live = Live::avc3(); let max_delay = std::time::Duration::from_secs(10); - let mut export = ExportSource::for_video(&live.source(), live.track.name(), &video(None), max_delay) + let mut export = ExportSource::for_video(&live.source().await, live.track.name(), &video(None), max_delay) .unwrap() .expect("fixture should produce a video rendition"); diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs index f577875ed4..5b4829cada 100644 --- a/rs/moq-mux/src/container/test_util.rs +++ b/rs/moq-mux/src/container/test_util.rs @@ -84,12 +84,13 @@ impl Live { }) } - pub(crate) fn source(&self) -> crate::Source { - crate::source::announced(&self.consumer) + pub(crate) async fn source(&self) -> crate::Source { + crate::source::announced(&self.consumer).await } pub(crate) async fn catalog_stream(&self) -> crate::catalog::Consumer { self.source() + .await .catalog::<()>(crate::catalog::CatalogFormat::Hang) .await .expect("catalog consumer") diff --git a/rs/moq-mux/src/container/ts/export.rs b/rs/moq-mux/src/container/ts/export.rs index 3589f167ba..d2faadae99 100644 --- a/rs/moq-mux/src/container/ts/export.rs +++ b/rs/moq-mux/src/container/ts/export.rs @@ -572,7 +572,7 @@ struct SiTrack { enum SiState { /// Waiting for the catalog broadcast to resolve; the track (by name) is /// subscribed once it does. - Requesting(kio::Pending, String), + Requesting(crate::Binding, String), /// Waiting for the subscription to resolve. Subscribing(kio::Pending), /// The resolved subscription, reading snapshot groups. @@ -628,7 +628,7 @@ impl SiTrack { let SiState::Requesting(pending, name) = &self.state else { unreachable!("just matched Requesting"); }; - match pending.poll_ok(waiter) { + match pending.poll_broadcast(waiter) { Poll::Ready(Ok(broadcast)) => Ok((broadcast, name.clone())), Poll::Ready(Err(err)) => Err(err), Poll::Pending => return, @@ -756,13 +756,12 @@ impl Export { /// Shared constructor. The public entry points each live on a concrete /// `Export` impl that pins `E`, so the extension is chosen by which one you call. async fn build(source: crate::Source, catalog_format: CatalogFormat) -> Result { - let broadcast = source.broadcast().await?; + let broadcast = source.broadcast(); let catalog = crate::catalog::Consumer::new(&broadcast, catalog_format).await?; Ok(Self::build_with(source, &broadcast, catalog_format, catalog)) } - /// Export `broadcast`, the one at `source`'s path, from its subscribed `catalog`, pinning - /// every later request for that path to its instance. + /// Export `broadcast`, the one `source` holds at its path, from its subscribed `catalog`. fn build_with( source: crate::Source, broadcast: &moq_net::broadcast::Consumer, @@ -771,7 +770,7 @@ impl Export { ) -> Self { let instance = broadcast.info().epoch.clone(); Self { - source: source.pinned(instance.clone()), + source, catalog: Some(catalog), catalog_format, cataloged: false, @@ -1506,6 +1505,12 @@ impl Export { catalog: crate::catalog::Consumer, ) -> Result<(), crate::Error> { let same = self.instance.is_some() && broadcast.info().epoch == self.instance; + // The same instance keeps its siblings, but the old broadcast's routes are gone. Another + // one has its own siblings to resolve. + match same { + true => self.source.pin(broadcast), + false => self.source = self.source.switched(broadcast), + } if !same || self.psi.is_none() { let mut next = Self::build_with(self.source.clone(), broadcast, self.catalog_format, catalog); next.replay = self.replay; diff --git a/rs/moq-mux/src/container/ts/export_test.rs b/rs/moq-mux/src/container/ts/export_test.rs index d77ca6e704..cb6f3a916d 100644 --- a/rs/moq-mux/src/container/ts/export_test.rs +++ b/rs/moq-mux/src/container/ts/export_test.rs @@ -77,7 +77,7 @@ const DRAIN: std::time::Duration = RECORDING_MAX_AGE .saturating_add(std::time::Duration::from_secs(1)); async fn drain(consumer: moq_net::broadcast::Consumer) -> BytesMut { - drain_with(Export::new(crate::source::announced(&consumer)).await.unwrap()).await + drain_with(Export::new(crate::source::announced(&consumer).await).await.unwrap()).await } /// `drain` for an exporter built with an explicit catalog extension. @@ -300,7 +300,7 @@ async fn export_lead_audio() -> BytesMut { video.finish().unwrap(); audio.finish().unwrap(); - let exporter = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let exporter = Export::new(crate::source::announced(&consumer).await).await.unwrap(); // The producers stay alive through the drain so the retained tracks are readable. drain_with(exporter).await } @@ -850,7 +850,7 @@ async fn export_pcr_wraps_below_zero_at_start() { } producer.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)) + let mut exporter = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(RECORDING_MAX_AGE); @@ -1138,9 +1138,12 @@ async fn exported_ac3(payload: Vec) -> Vec<(u64, usize)> { import.decode(&BytesMut::from(&data[..])).unwrap(); import.finish().unwrap(); let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; @@ -1234,9 +1237,12 @@ async fn export_scte35_roundtrip() { // `import`, `catalog`, and `scte_producer` stay alive: retained tracks. The // exporter must carry the extension to see the mpegts section. let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -1369,9 +1375,12 @@ async fn export_pes_verbatim_roundtrip() { // `import`, `catalog`, and `data_producer` stay alive: retained tracks. let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -1458,9 +1467,12 @@ async fn scte35_without_video_export_is_rejected() { producer.cut(None).unwrap(); producer.finish().unwrap(); - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(); let err = loop { match tokio::time::timeout(DRAIN, exporter.next()).await { Ok(Ok(Some(_))) => continue, @@ -1738,7 +1750,7 @@ async fn aac_program_config_follows_each_table() { .renditions .insert("a.aac".to_string(), cfg); let mut producer = Producer::new(track, HangContainer::Legacy(Kind::Audio)); - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -1879,7 +1891,7 @@ async fn aac_export_refuses_what_adts_cannot_label() { .unwrap(); producer.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let mut exporter = Export::new(crate::source::announced(&consumer).await).await.unwrap(); let err = loop { match tokio::time::timeout(DRAIN, exporter.next()).await { Ok(Ok(Some(_))) => continue, @@ -2178,9 +2190,12 @@ async fn scte35_fixtures_survive_roundtrip() { // Export and re-ingest. let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -2424,9 +2439,12 @@ async fn service_layer_survives_roundtrip() { // `import` and `catalog` stay alive: retained tracks the exporter subscribes to. let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -2576,9 +2594,12 @@ async fn si_pids_are_re_emitted_on_their_own_interval() { producer.finish().unwrap(); let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -2662,9 +2683,12 @@ async fn inline_si_form_is_re_emitted() { producer.finish().unwrap(); let ts = drain_with( - Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(), + Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(), ) .await; assert_packet_aligned(&ts); @@ -2751,10 +2775,13 @@ async fn export_rejects_si_pid_on_the_pmt() { } let _producer = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Data)); - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap() - .with_delay(RECORDING_MAX_AGE); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE); let err = exporter .next() .await @@ -2800,10 +2827,13 @@ async fn export_rejects_si_pid_on_an_elementary_stream() { } let _producer = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Data)); - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap() - .with_delay(RECORDING_MAX_AGE); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE); let err = exporter .next() .await @@ -2816,11 +2846,14 @@ async fn export_rejects_si_pid_on_an_elementary_stream() { /// A `mpegts`-extension exporter over an announced broadcast. async fn export_of(consumer: &moq_net::broadcast::Consumer) -> Export { - Export::with_ts(crate::source::announced(consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap() - .with_delay(RECORDING_MAX_AGE) - .with_replay() + Export::with_ts( + crate::source::announced(consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE) + .with_replay() } fn publish_sdt( @@ -3692,7 +3725,7 @@ async fn a_selected_programs_si_survives_export() { let ts = drain_with( Export::with_ts( - crate::source::announced(&rig.consumer), + crate::source::announced(&rig.consumer).await, crate::catalog::CatalogFormat::Hang, ) .await @@ -3918,9 +3951,12 @@ async fn stale_si_entry_does_not_block_output() { producer.cut(None).unwrap(); producer.finish().unwrap(); - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(); // The timeout distinguishes "produced output promptly" from "held dark"; // under paused time a wedged exporter would hit it instantly. let frame = tokio::time::timeout(DRAIN, exporter.next()) @@ -4167,7 +4203,7 @@ async fn export_opus_track(channel_count: u32, description: Option) -> Re .unwrap(); producer.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)) + let mut exporter = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -4373,7 +4409,7 @@ async fn export_twice(with_video: bool) -> (Vec, Vec) { Producer::new(track, HangContainer::Legacy(crate::container::Kind::Data)) }; - let source = crate::source::announced(&consumer); + let source = crate::source::announced(&consumer).await; let export = || Export::with_ts(source.clone(), crate::catalog::CatalogFormat::Hang); let mut a = export().await.unwrap(); let mut b = None; @@ -4533,7 +4569,7 @@ struct Interleave { } impl Interleave { - fn new() -> Self { + async fn new() -> Self { let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); let mut catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); @@ -4578,7 +4614,7 @@ impl Interleave { let audio = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Audio)); Self { - source: crate::source::announced(&consumer), + source: crate::source::announced(&consumer).await, _broadcast: broadcast, _catalog: catalog, video, @@ -4757,7 +4793,7 @@ fn assert_on_time(frames: &[Frame]) -> usize { /// Audio lagging video by up to a frame, on a live clock: each video frame lands before /// the audio that precedes it. Returns the decode order the exporter emitted. async fn lagging_audio(delay: Duration) -> Vec { - let mut rig = Interleave::new(); + let mut rig = Interleave::new().await; let mut export = rig.export(delay).await; let mut out = Vec::new(); for tick in 0..TICKS / 2 { @@ -4793,7 +4829,7 @@ async fn zero_delay_keeps_arrival_order() { /// of its audio), the other only once each tick's frames have all landed. #[tokio::test(start_paused = true)] async fn arrival_order_does_not_change_the_output() { - let mut rig = Interleave::new(); + let mut rig = Interleave::new().await; let delay = Duration::from_millis(500); let (mut eager, mut batched) = (rig.export(delay).await, rig.export(delay).await); let (mut out_eager, mut out_batched) = (Vec::new(), Vec::new()); @@ -4819,7 +4855,7 @@ async fn arrival_order_does_not_change_the_output() { #[tokio::test(start_paused = true)] async fn a_late_frame_is_dropped_and_the_rest_keep_their_order() { let delay = Duration::from_millis(100); - let mut rig = Interleave::new(); + let mut rig = Interleave::new().await; let mut export = rig.export(delay).await; let mut out = Vec::new(); for tick in 0..TICKS / 2 { @@ -4849,7 +4885,7 @@ async fn a_late_frame_is_dropped_and_the_rest_keep_their_order() { #[tokio::test(start_paused = true)] async fn tune_in_does_not_wait_on_dropped_audio() { let delay = Duration::from_millis(500); - let mut rig = Interleave::new(); + let mut rig = Interleave::new().await; let mut export = rig.export(delay).await; let mut out = Vec::new(); rig.audio_until(3 * GOP * VIDEO_US, &mut export, &mut out); @@ -4967,10 +5003,13 @@ async fn repointed_si_entry_resubscribes() { // Writes both GOPs up front and only then reads, so it needs a replay window: // the real-time default would take the live edge and skip the first. - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap() - .with_delay(RECORDING_MAX_AGE); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE); let mut before = BytesMut::new(); write_key(&mut producer, 0); write_key(&mut producer, 1); @@ -5093,9 +5132,12 @@ async fn si_revision_after_final_media_frame_is_flushed() { producer.cut(None).unwrap(); producer.finish().unwrap(); - let mut exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(); + let mut exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(); let first = loop { let frame = tokio::time::timeout(DRAIN, exporter.next()) .await @@ -5186,9 +5228,12 @@ async fn si_cadence_rig(pid: u16, table_id: u8, interval: Duration) -> SiCadence } let producer = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Data)); - let exporter = Export::with_ts(crate::source::announced(&consumer), crate::catalog::CatalogFormat::Hang) - .await - .unwrap(); + let exporter = Export::with_ts( + crate::source::announced(&consumer).await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap(); SiCadenceRig { started: false, producer, @@ -5630,7 +5675,7 @@ async fn export_cbr_video() -> Vec { } video.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)) + let mut exporter = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_mux_rate(20_000_000) @@ -5696,7 +5741,7 @@ async fn a_frame_on_a_slot_boundary_finishes_before_it_decodes() { } video.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let mut exporter = Export::new(crate::source::announced(&consumer).await).await.unwrap(); let frames = drain_frames(&mut exporter).await; let ts: Vec = frames.iter().flat_map(|f| f.payload.iter().copied()).collect(); let packets: Vec<&[u8]> = ts.chunks(188).collect(); @@ -5816,7 +5861,7 @@ async fn slices_go_out_on_the_clock() { let consumer = broadcast.consume(); let mut catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); let mut audio = aac_rendition(&mut broadcast, &mut catalog, "a.aac"); - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(Duration::from_millis(1_500)); @@ -5861,7 +5906,7 @@ async fn pcr_stays_among_the_bytes_across_reordered_tracks() { import.decode(&BytesMut::from(&data[..])).unwrap(); import.finish().unwrap(); - let mut exporter = Export::new(crate::source::announced(&consumer)) + let mut exporter = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_mux_rate(20_000_000) @@ -5981,7 +6026,7 @@ async fn a_heavy_passage_fills_the_slots_before_it() { } producer.finish().unwrap(); - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_mux_rate(2_000_000) @@ -6225,7 +6270,7 @@ async fn export_pcr_is_its_byte_position_at_the_rate() { } producer.finish().unwrap(); - let export = Export::new(crate::source::announced(&consumer)) + let export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_mux_rate(1_000_000); @@ -6282,7 +6327,7 @@ struct Reordered { } impl Reordered { - fn new(sps: &'static [u8], pps: &'static [u8], pyramid: &'static [u64]) -> Self { + async fn new(sps: &'static [u8], pps: &'static [u8], pyramid: &'static [u64]) -> Self { let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); let mut catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); @@ -6323,7 +6368,7 @@ impl Reordered { let audio = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Audio)); Self { - source: crate::source::announced(&consumer), + source: crate::source::announced(&consumer).await, _broadcast: broadcast, catalog, video, @@ -6402,7 +6447,7 @@ async fn export_reordered( rate: Option, ) -> (Vec, Vec) { let delay = Duration::from_millis(500); - let mut rig = Reordered::new(sps, pps, pyramid); + let mut rig = Reordered::new(sps, pps, pyramid).await; let padded = |export: Export| match rate { Some(rate) => export.with_mux_rate(rate), None => export, @@ -6503,7 +6548,8 @@ async fn declared_reorder_sizes_the_decode_clock_from_the_first_frame() { crate::codec::h264::fixtures::SPS_IPB, crate::codec::h264::fixtures::PPS, &[], - ); + ) + .await; let max_age = Duration::from_millis(500); // Written as fast as it is read, so the broadcast is all there at once. let (mut late, mut whole) = ( @@ -6559,7 +6605,7 @@ async fn declared_reorder_sizes_the_decode_clock_from_the_first_frame() { #[tokio::test(start_paused = true)] async fn reordered_video_loses_nothing_on_a_clean_path() { let delay = Duration::from_millis(500); - let mut rig = Reordered::new(SPS, PPS, &[0, 1, 2, 3, 4]); + let mut rig = Reordered::new(SPS, PPS, &[0, 1, 2, 3, 4]).await; let mut export = rig.export(delay).await; let start = tokio::time::Instant::now(); let mut out = Vec::new(); @@ -6681,7 +6727,7 @@ async fn a_skipped_group_keeps_the_clock() { } let delay = Duration::from_millis(500); - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(delay); @@ -6744,7 +6790,7 @@ async fn a_timeline_restarting_at_zero_fails_the_export() { .insert("a.aac".to_string(), cfg); } - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(Duration::from_millis(500)); @@ -6804,7 +6850,7 @@ async fn a_late_join_starts_at_the_live_edge() { }) .unwrap(); if k == 275 { - let joined = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let joined = Export::new(crate::source::announced(&consumer).await).await.unwrap(); export = Some(joined.with_delay(Duration::from_secs(2))); } if let Some(export) = export.as_mut() { @@ -6894,7 +6940,7 @@ async fn a_track_leaving_the_catalog_is_read_to_its_end() { let mut kept = aac_rendition(&mut broadcast, &mut catalog, "a.aac"); let mut leaving = aac_rendition(&mut broadcast, &mut catalog, "b.aac"); - let mut export = Export::new(crate::source::announced(&consumer)) + let mut export = Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -7023,13 +7069,12 @@ fn unflagged(frames: &[Frame]) -> Vec { /// gap the clock spans. Both ways a broadcast ends, a clean finish and a drop, continue alike. async fn same_instance_returns(finish: bool) { let origin = crate::source::produce_origin(); - let source = crate::Source::new(origin.consume(), "live"); let epoch = moq_net::Epoch::mint(); let route = || moq_net::origin::Route::default().with_epoch(epoch.clone()); let (mut broadcast, mut catalog) = publish_live::<()>(&origin, route()); let mut track = aac_rendition(&mut broadcast, &mut catalog, "a.aac"); - let mut export = Export::new(source.clone()) + let mut export = Export::new(crate::source::resolved(&origin, "live").await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -7097,12 +7142,11 @@ async fn the_same_instance_continues_after_a_drop() { #[tokio::test(start_paused = true)] async fn a_switch_carries_through_an_instance_that_never_built_its_tables() { let origin = crate::source::produce_origin(); - let source = crate::Source::new(origin.consume(), "live"); let route = || moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint()); let (mut old, mut old_catalog) = publish_live::<()>(&origin, route()); let mut old_audio = aac_rendition(&mut old, &mut old_catalog, "a.aac"); - let mut export = Export::new(source.clone()) + let mut export = Export::new(crate::source::resolved(&origin, "live").await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -7152,12 +7196,11 @@ async fn a_switch_carries_through_an_instance_that_never_built_its_tables() { #[tokio::test(start_paused = true)] async fn another_instance_is_a_full_program_switch() { let origin = crate::source::produce_origin(); - let source = crate::Source::new(origin.consume(), "live"); let route = || moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint()); let (mut old, mut old_catalog) = publish_live::<()>(&origin, route()); let mut old_audio = aac_rendition(&mut old, &mut old_catalog, "a.aac"); - let mut export = Export::new(source.clone()) + let mut export = Export::new(crate::source::resolved(&origin, "live").await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) @@ -7230,7 +7273,6 @@ async fn another_instance_is_a_full_program_switch() { #[tokio::test(start_paused = true)] async fn a_switch_moving_the_pmt_advances_the_pat() { let origin = crate::source::produce_origin(); - let source = crate::Source::new(origin.consume(), "live"); let route = || moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint()); let publish = |pmt_pid: u16| { let (mut broadcast, mut catalog) = publish_live::(&origin, route()); @@ -7244,11 +7286,14 @@ async fn a_switch_moving_the_pmt_advances_the_pat() { }; let (_old, _old_catalog, mut old_track) = publish(0x100); - let mut export = Export::with_ts(source.clone(), crate::catalog::CatalogFormat::Hang) - .await - .unwrap() - .with_delay(RECORDING_MAX_AGE) - .with_replay(); + let mut export = Export::with_ts( + crate::source::resolved(&origin, "live").await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE) + .with_replay(); for ms in (0..200).step_by(20) { write_aac(&mut old_track, ms); } @@ -7279,14 +7324,16 @@ async fn a_switch_moving_the_pmt_advances_the_pat() { #[tokio::test(start_paused = true)] async fn export_stats_count_only_returned_output() { let origin = crate::source::produce_origin(); - let source = crate::Source::new(origin.consume(), "live"); let epoch = moq_net::Epoch::mint(); let route = || moq_net::origin::Route::default().with_epoch(epoch.clone()); let units = |stats: stats::Export| stats.streams.values().map(|row| row.units).sum::() as usize; let (mut broadcast, mut catalog) = publish_live::<()>(&origin, route()); let mut track = aac_rendition(&mut broadcast, &mut catalog, "a.aac"); - let mut export = Export::new(source.clone()).await.unwrap().with_delay(RECORDING_MAX_AGE); + let mut export = Export::new(crate::source::resolved(&origin, "live").await) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE); for ms in (0..200).step_by(20) { write_aac(&mut track, ms); } @@ -7357,7 +7404,7 @@ async fn export_liveness(sample: u64, stop: u64) -> (stats::Export, stats::Expor aac_rendition(&mut broadcast, &mut catalog, "secondary.aac"), ]; - let mut export = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let mut export = Export::new(crate::source::announced(&consumer).await).await.unwrap(); assert!(export.stats().streams.is_empty(), "no row before the program tables"); let (mut sampled, mut frames) = (None, Vec::new()); let mut audio_index = 0; @@ -7471,7 +7518,7 @@ async fn export_stats_count_a_gap_longer_than_the_backfill() { let mut stopped = aac_rendition(&mut broadcast, &mut catalog, "a.aac"); let mut running = aac_rendition(&mut broadcast, &mut catalog, "b.aac"); - let mut export = Export::new(crate::source::announced(&consumer)).await.unwrap(); + let mut export = Export::new(crate::source::announced(&consumer).await).await.unwrap(); for ms in (0..2_000).step_by(20) { write_aac(&mut stopped, ms); write_aac(&mut running, ms); @@ -7513,7 +7560,7 @@ fn publish_bare( /// Follow the broadcast at `live` with a fresh export of it. async fn follower(origin: &moq_net::origin::Producer) -> super::Follower { - let source = crate::Source::new(origin.consume(), "live"); + let source = crate::source::resolved(origin, "live").await; super::Follower::new(Export::new(source).await.unwrap()).unwrap() } @@ -7795,8 +7842,9 @@ async fn a_follower_built_after_its_route_went_follows_the_return() { let origin = crate::source::produce_origin(); let epoch = moq_net::Epoch::mint(); let first = publish_bare(&origin, Some(&epoch)); - let source = crate::Source::new(origin.consume(), "live"); - let export = Export::new(source).await.unwrap(); + let export = Export::new(crate::source::resolved(&origin, "live").await) + .await + .unwrap(); let start = tokio::time::Instant::now(); finish(first); tokio::time::sleep(Duration::from_secs(1)).await; @@ -7844,7 +7892,7 @@ async fn a_follower_takes_its_own_instance_winning_back_for_no_replacement() { request.accept(consumer.clone()); } }); - let source = crate::Source::new(origin.consume(), "pool/job"); + let source = crate::source::resolved(&origin, "pool/job").await; let mut follower = super::Follower::new(Export::new(source).await.unwrap()).unwrap(); // A more specific route takes the path for a while, then goes. @@ -7958,7 +8006,7 @@ async fn a_follower_continues_through_a_same_epoch_handoff_after_a_gap() { let mut exact = origin.publish("pool/job", route.clone()).unwrap(); let mut catalog = crate::catalog::Producer::<()>::new(&mut exact, Default::default()).unwrap(); let mut track = aac_rendition(&mut exact, &mut catalog, "a.aac"); - let source = crate::Source::new(origin.consume(), "pool/job"); + let source = crate::source::resolved(&origin, "pool/job").await; let export = Export::new(source) .await .unwrap() @@ -8025,7 +8073,7 @@ async fn a_follower_ignores_a_covering_prefix_while_the_exact_route_serves() { let route = |epoch| moq_net::origin::Route::default().with_epoch(epoch); let mut exact = origin.publish("pool/job", route(moq_net::Epoch::mint())).unwrap(); let catalog = crate::catalog::Producer::<()>::new(&mut exact, Default::default()).unwrap(); - let source = crate::Source::new(origin.consume(), "pool/job"); + let source = crate::source::resolved(&origin, "pool/job").await; let mut follower = super::Follower::new(Export::new(source).await.unwrap()).unwrap(); let quiet = Duration::from_secs(1); @@ -8041,3 +8089,152 @@ async fn a_follower_ignores_a_covering_prefix_while_the_exact_route_serves() { let end = follower.next().await; assert!(matches!(end, Ok(None)), "no replacement: {end:?}"); } + +/// The audio PTS in `frames`. +fn audio_pts(frames: &[Frame]) -> Vec { + let bytes: Vec = frames.iter().flat_map(|f| f.payload.iter().copied()).collect(); + collect_pes_pts(&bytes).1 +} + +/// A rendition the catalog lists only after another instance took the path reads the instance +/// the export resolved, never the replacement, even on an epochless route. +#[tokio::test(start_paused = true)] +async fn a_late_rendition_never_reads_a_replacement() { + let origin = crate::source::produce_origin(); + let (mut old, mut old_catalog) = publish_live::<()>(&origin, Default::default()); + let mut export = Export::new(crate::source::resolved(&origin, "live").await) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE) + .with_replay(); + assert!(drain_frames(&mut export).await.is_empty(), "nothing listed yet"); + + // Another instance takes the path, its audio ten seconds ahead. + let (mut new, mut new_catalog) = publish_live::<()>(&origin, Default::default()); + let mut new_audio = aac_rendition(&mut new, &mut new_catalog, "a.aac"); + for ms in (10_000..10_200).step_by(20) { + write_aac(&mut new_audio, ms); + } + + let mut old_audio = aac_rendition(&mut old, &mut old_catalog, "a.aac"); + for ms in (0..200).step_by(20) { + write_aac(&mut old_audio, ms); + } + let pts = audio_pts(&drain_frames(&mut export).await); + assert_eq!(pts.len(), 10, "the old instance's audio went out: {pts:?}"); + assert!( + pts.iter().all(|&pts| pts < 60_200 * 90), + "nothing of the replacement: {pts:?}" + ); +} + +/// An SI entry repointed after another instance took the path reads the instance the export +/// resolved, never the replacement, even on an epochless route. +#[tokio::test(start_paused = true)] +async fn a_repointed_si_entry_never_reads_a_replacement() { + let sdt_a = make_long_section(0x42, 1, 0, 0, 0, &[0xaa; 8]); + let sdt_b = make_long_section(0x42, 1, 1, 0, 0, &[0xbb; 8]); + let sdt_new = make_long_section(0x42, 1, 2, 0, 0, &[0xcc; 8]); + let entry = |track: &str| tscat::SiEntry { + track: track.to_string(), + interval: Some(Duration::from_secs(1)), + ..Default::default() + }; + let si_track = |broadcast: &mut moq_net::broadcast::Producer, name: &str, sdt: &[u8]| { + let mut track = broadcast.create_track(name, None).unwrap(); + track.write_frame(Timestamp::ZERO, Bytes::copy_from_slice(sdt)).unwrap(); + track + }; + + let origin = crate::source::produce_origin(); + let (mut old, mut catalog) = publish_live::(&origin, Default::default()); + let mut video = h264_rendition(&mut old, &mut catalog, "video.avc1"); + let _a = si_track(&mut old, "a.si", &sdt_a); + let _b = si_track(&mut old, "b.si", &sdt_b); + catalog + .modify() + .unwrap() + .ext + .mpegts + .si + .entry(0x0011) + .or_default() + .insert(0x42, entry("a.si")); + let mut export = Export::with_ts( + crate::source::resolved(&origin, "live").await, + crate::catalog::CatalogFormat::Hang, + ) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE) + .with_replay(); + for ms in (0..1_000).step_by(40) { + write_h264(&mut video, ms); + } + let before: Vec = drain_frames(&mut export) + .await + .iter() + .flat_map(|f| f.payload.iter().copied()) + .collect(); + + // Another instance takes the path with its own `b.si`, and only then is the entry repointed. + let mut new = origin.publish("live", Default::default()).unwrap(); + let _new_b = si_track(&mut new, "b.si", &sdt_new); + catalog + .modify() + .unwrap() + .ext + .mpegts + .si + .get_mut(&0x0011) + .unwrap() + .insert(0x42, entry("b.si")); + for ms in (1_000..3_000).step_by(40) { + write_h264(&mut video, ms); + } + let after: Vec = drain_frames(&mut export) + .await + .iter() + .flat_map(|f| f.payload.iter().copied()) + .collect(); + + let contains = |haystack: &[u8], needle: &[u8]| haystack.windows(needle.len()).any(|w| w == needle); + assert!(contains(&before, &sdt_a), "the original entry went out (control)"); + assert!(contains(&after, &sdt_b), "the repointed entry reads the old instance"); + assert!(!contains(&after, &sdt_new), "nothing of the replacement"); +} + +/// A switch onto an instance that is itself replaced before the switched export requests a +/// track still reads the instance it switched to. +#[tokio::test(start_paused = true)] +async fn a_switch_reads_the_instance_it_followed() { + let origin = crate::source::produce_origin(); + let (_old, _old_catalog) = publish_live::<()>(&origin, Default::default()); + let export = Export::new(crate::source::resolved(&origin, "live").await) + .await + .unwrap() + .with_delay(RECORDING_MAX_AGE) + .with_replay(); + + let (mut followed, mut followed_catalog) = publish_live::<()>(&origin, Default::default()); + let mut followed_audio = aac_rendition(&mut followed, &mut followed_catalog, "a.aac"); + for ms in (0..200).step_by(20) { + write_aac(&mut followed_audio, ms); + } + let broadcast = origin.consume().request_broadcast("live", None).await.unwrap(); + + // Replaced again, ten seconds ahead, before the switched export requests its track. + let (mut newer, mut newer_catalog) = publish_live::<()>(&origin, Default::default()); + let mut newer_audio = aac_rendition(&mut newer, &mut newer_catalog, "a.aac"); + for ms in (10_000..10_200).step_by(20) { + write_aac(&mut newer_audio, ms); + } + + let mut export = export.follow(broadcast).await.unwrap(); + let pts = audio_pts(&drain_frames(&mut export).await); + assert_eq!(pts.len(), 10, "the followed instance's audio went out: {pts:?}"); + assert!( + pts.iter().all(|&pts| pts < 60_200 * 90), + "nothing of the newer instance: {pts:?}" + ); +} diff --git a/rs/moq-mux/src/container/ts/export_timing_test.rs b/rs/moq-mux/src/container/ts/export_timing_test.rs index 4eb000e3c5..7993581106 100644 --- a/rs/moq-mux/src/container/ts/export_timing_test.rs +++ b/rs/moq-mux/src/container/ts/export_timing_test.rs @@ -83,7 +83,7 @@ struct Live { } impl Live { - fn new(scale: f64, audio_lag: i64) -> Self { + async fn new(scale: f64, audio_lag: i64) -> Self { let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); let mut catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); @@ -128,7 +128,7 @@ impl Live { let audio = Producer::new(track, HangContainer::Legacy(crate::container::Kind::Audio)); Self { - source: crate::source::announced(&consumer), + source: crate::source::announced(&consumer).await, _broadcast: broadcast, catalog, video, @@ -508,7 +508,7 @@ const CLOCK_SLACK: f64 = 2.0 * SLOT_MS; /// Join mid-group with `lag`, render 7.2 s with the video ticks `stall` accepts stalled, and /// return the leg's units. async fn join(lag: i64, stall: impl Fn(u64) -> bool) -> Vec { - let mut live = Live::new(1.0, lag); + let mut live = Live::new(1.0, lag).await; live.run(JOIN, &mut [], |_| false).await; let mut leg = live.join(Duration::ZERO).await; live.run(24 * GOP, &mut [&mut leg], stall).await; @@ -573,7 +573,7 @@ async fn a_track_sent_later_than_the_delay_loses_nothing() { let mut lost = Vec::new(); for lag in [LONG_LAG, -LONG_LAG] { for join in [3 * GOP, JOIN] { - let mut live = Live::new(1.0, lag); + let mut live = Live::new(1.0, lag).await; live.run(join, &mut [], |_| false).await; let mut leg = live.join(Duration::ZERO).await; live.run(join + 20 * GOP, &mut [&mut leg], |_| false).await; @@ -611,7 +611,7 @@ async fn frames_held_through_a_silence_drop_as_late() { // The frames presented within the jitter of the last one read before the silence. let held = jitter.as_micros() as u64 / VIDEO_US + 1; for lag in [0, LAGS[0], LAGS[1]] { - let mut live = Live::new(1.0, lag); + let mut live = Live::new(1.0, lag).await; live.declare_video_jitter(jitter); let mut leg = live.join(Duration::ZERO).await; live.run(8 * GOP + 5, &mut [&mut leg], |_| false).await; @@ -647,7 +647,7 @@ async fn frames_held_through_a_silence_drop_as_late() { #[tokio::test(start_paused = true)] async fn video_held_for_most_of_the_delay_drops_nothing() { for lag in [0, LAGS[0], LAGS[1]] { - let mut live = Live::new(1.0, lag); + let mut live = Live::new(1.0, lag).await; live.declare_video_jitter(DELAY * 4 / 5); let mut leg = live.join(Duration::ZERO).await; live.run(20 * GOP, &mut [&mut leg], |_| false).await; @@ -661,7 +661,7 @@ async fn video_held_for_most_of_the_delay_drops_nothing() { #[tokio::test(start_paused = true)] async fn the_video_held_at_the_end_goes_out() { for lag in [0, LAGS[0], LAGS[1]] { - let mut live = Live::new(1.0, lag); + let mut live = Live::new(1.0, lag).await; live.declare_video_jitter(Duration::from_millis(300)); let mut leg = live.join(Duration::ZERO).await; let last = 8 * GOP + 5; @@ -693,7 +693,7 @@ const DRIFT_TICKS: u64 = 300_000_000 / VIDEO_US; /// The spread of (slot sent − source's clock at the slot) after the first 10 s over `ticks` /// of media, the drops, and the export's stats at the end. async fn on_a_scaled_clock(scale: f64, ticks: u64) -> (Duration, u64, Stats) { - let mut live = Live::new(scale, 0); + let mut live = Live::new(scale, 0).await; let mut leg = live.join(Duration::ZERO).await; live.run(ticks, &mut [&mut leg], |_| false).await; @@ -781,7 +781,7 @@ async fn pair(scale: f64, lag: i64, ticks: u64, stall: impl Fn(u64) -> bool + Co } async fn pair_at(rate: u64, scale: f64, lag: i64, ticks: u64, stall: impl Fn(u64) -> bool + Copy) -> (Leg, Leg, u64) { - let mut live = Live::new(scale, lag); + let mut live = Live::new(scale, lag).await; live.rate = rate; let mut a = live.join(Duration::ZERO).await; live.run(JOIN, &mut [&mut a], stall).await; @@ -838,7 +838,7 @@ async fn two_legs_render_a_drifting_source_the_same_way() { #[tokio::test(start_paused = true)] async fn a_joiner_keeps_the_system_clock_in_tolerance() { const WINDOW: u64 = 30_000_000; - let mut live = Live::new(1.0, 0); + let mut live = Live::new(1.0, 0).await; live.run(JOIN, &mut [], |_| false).await; let mut leg = live.join(Duration::ZERO).await; live.run(JOIN + 300_000_000 / VIDEO_US, &mut [&mut leg], |_| false) @@ -885,7 +885,7 @@ async fn a_joiner_keeps_the_system_clock_in_tolerance() { /// joined on was. #[tokio::test(start_paused = true)] async fn a_joiner_runs_at_the_delay_from_its_first_output() { - let mut live = Live::new(1.0, 0); + let mut live = Live::new(1.0, 0).await; live.run(JOIN, &mut [], |_| false).await; let mut leg = live.join(Duration::ZERO).await; live.run(JOIN + 10_000_000 / VIDEO_US, &mut [&mut leg], |_| false).await; diff --git a/rs/moq-mux/src/container/ts/import.rs b/rs/moq-mux/src/container/ts/import.rs index 3dd62ef1e8..e64d1e6e80 100644 --- a/rs/moq-mux/src/container/ts/import.rs +++ b/rs/moq-mux/src/container/ts/import.rs @@ -4006,7 +4006,7 @@ pub(super) mod test { import.finish().unwrap(); let exporter = crate::container::ts::Export::with_ts( - crate::source::announced(&consumer), + crate::source::announced(&consumer).await, crate::catalog::CatalogFormat::Hang, ) .await @@ -7106,7 +7106,7 @@ pub(super) mod test { "the first group ends at the break, one frame after its last picture" ); - let mut exporter = super::super::Export::new(crate::source::announced(&consumer)) + let mut exporter = super::super::Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(Duration::from_secs(30)) @@ -7513,7 +7513,7 @@ pub(super) mod test { // and the boundary with them. The paused clock runs out the delay and every gap in // the media without a real wait. let delay = std::time::Duration::from_secs(3600); - let mut exporter = crate::container::ts::Export::new(crate::source::announced(&consumer)) + let mut exporter = crate::container::ts::Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(delay) diff --git a/rs/moq-mux/src/container/ts/import_test.rs b/rs/moq-mux/src/container/ts/import_test.rs index 8b3f0662b3..03a6217b0a 100644 --- a/rs/moq-mux/src/container/ts/import_test.rs +++ b/rs/moq-mux/src/container/ts/import_test.rs @@ -474,7 +474,7 @@ async fn import_export_import_roundtrip() { // Re-export to TS. `import` and `catalog` stay alive so the exporter can // subscribe to the finished, retained tracks. - let mut exporter = crate::container::ts::Export::new(crate::source::announced(&consumer)) + let mut exporter = crate::container::ts::Export::new(crate::source::announced(&consumer).await) .await .unwrap() .with_delay(RECORDING_MAX_AGE) diff --git a/rs/moq-mux/src/container/ts/programs.rs b/rs/moq-mux/src/container/ts/programs.rs index 291c8c49f0..49ab1ffda5 100644 --- a/rs/moq-mux/src/container/ts/programs.rs +++ b/rs/moq-mux/src/container/ts/programs.rs @@ -229,10 +229,7 @@ mod test { programs.finish().unwrap(); assert_eq!(programs.programs.len(), 50); - let consumer = crate::Source::new(origin.consume(), "event/50.hang") - .broadcast() - .await - .unwrap(); + let consumer = origin.consume().request_broadcast("event/50.hang", None).await.unwrap(); let catalog = hang::catalog::Catalog::<()>::subscribe(&consumer) .await .unwrap() @@ -268,7 +265,7 @@ mod test { /// How many audio renditions the catalog `path` publishes lists. async fn renditions(origin: &moq_net::origin::Producer, path: &str) -> usize { - let consumer = crate::Source::new(origin.consume(), path).broadcast().await.unwrap(); + let consumer = origin.consume().request_broadcast(path, None).await.unwrap(); let catalog = hang::catalog::Catalog::<()>::subscribe(&consumer) .await .unwrap() @@ -366,7 +363,7 @@ mod test { let after = std::time::SystemTime::now(); for (path, fills) in [("event/1.hang", [0xAA, 0xBB]), ("event/2.hang", [0xCC, 0xDD])] { - let consumer = crate::Source::new(origin.consume(), path).broadcast().await.unwrap(); + let consumer = origin.consume().request_broadcast(path, None).await.unwrap(); let catalog = hang::catalog::Catalog::<()>::subscribe(&consumer) .await .unwrap() diff --git a/rs/moq-mux/src/json.rs b/rs/moq-mux/src/json.rs index 602f1a3036..fb2ccd3dcf 100644 --- a/rs/moq-mux/src/json.rs +++ b/rs/moq-mux/src/json.rs @@ -768,7 +768,7 @@ mod test { async fn an_entry_subscribes_and_reads() { let mut broadcast = moq_net::broadcast::Info::new().produce(); let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); - let source = crate::source::announced(&broadcast.consume()); + let source = crate::source::announced(&broadcast.consume()).await; let mut chat = catalog .json_stream::(track(&mut broadcast, "chat"), Config::default()) diff --git a/rs/moq-mux/src/source.rs b/rs/moq-mux/src/source.rs index f128b6beec..18bf590625 100644 --- a/rs/moq-mux/src/source.rs +++ b/rs/moq-mux/src/source.rs @@ -1,55 +1,76 @@ -//! Export input: an origin plus the path of the broadcast whose catalog drives the export. +//! Export input: an origin plus the catalog broadcast that drives the export. //! //! A hang catalog rendition may reference a track published in *another* //! broadcast via its `broadcast` field (a path relative to the catalog's //! broadcast, e.g. `./source`). Resolving that reference needs the catalog //! broadcast's own path and an [`moq_net::origin::Consumer`] to fetch the -//! referenced broadcast from. [`Source`] bundles the two, and resolves both the -//! catalog broadcast and any referenced broadcast through the same origin so +//! referenced broadcast from. [`Source`] bundles the two, and resolves every +//! referenced broadcast through the same origin so //! [`request_broadcast`](moq_net::origin::Consumer::request_broadcast) deduplicates //! shared subscriptions. +use std::collections::HashMap; +use std::sync::{Arc, Mutex}; use std::task::{Poll, ready}; -use moq_net::AsPath; +/// The broadcast held for each path, shared by a source's clones and its requests. +type Pins = Arc>>; -/// The subscription side of an export: an origin and the path of the broadcast -/// whose catalog drives it. +/// The subscription side of an export: an origin and the catalog broadcast that drives it. /// -/// The catalog broadcast and every rendition (including ones whose catalog -/// `broadcast` field references a sibling broadcast) resolve against `origin`, -/// so a source can always follow a cross-broadcast reference. Build one with -/// [`Source::new`]. +/// Every rendition (including ones whose catalog `broadcast` field references a sibling +/// broadcast) resolves against `origin`, so a source can always follow a cross-broadcast +/// reference. Build one with [`Source::new`]. +/// +/// A source serves its own path from the broadcast it was built from, and keeps the first +/// broadcast it resolves for each sibling. Every later request for a path is served from that +/// handle ([`broadcast`](Self::broadcast), [`catalog`](Self::catalog), +/// [`resolve`](Self::resolve), [`bind`](Self::bind), [`subscribe_track`](Self::subscribe_track), +/// and an export's track requests). A route replaced after that is never resolved, epoch or +/// not, so two publishers never splice into one export; a request on a held broadcast that has +/// ended fails instead. Clones share the held broadcasts. #[derive(Clone)] pub struct Source { origin: moq_net::origin::Consumer, path: moq_net::PathOwned, - /// The publisher instance requests for this path are pinned to, once an export resolved - /// it: a request never lands on a replacement, which would splice two broadcasts. - epoch: Option, + pins: Pins, } impl Source { - /// A source rooted at `origin`, driven by the catalog of the broadcast at `path`. + /// A source driven by the catalog of `broadcast`, resolving its references through `origin`. /// - /// `path` names the broadcast whose catalog is exported; a rendition's relative - /// `broadcast` reference is resolved against it. Both the catalog broadcast and any - /// referenced broadcast are fetched via - /// [`origin.request_broadcast`](moq_net::origin::Consumer::request_broadcast), so they - /// must be reachable through `origin` (by exact path, or served by a dynamic handler). - pub fn new(origin: moq_net::origin::Consumer, path: impl AsPath) -> Self { + /// `broadcast` is the catalog broadcast as resolved through `origin`: a rendition's relative + /// `broadcast` reference resolves against its [path](moq_net::broadcast::Info::path), and the + /// referenced broadcast is fetched via + /// [`origin.request_broadcast`](moq_net::origin::Consumer::request_broadcast), so it must be + /// reachable through `origin` (by exact path, or served by a dynamic handler). A standalone + /// broadcast has an empty path, so it is its own root and any `..` reference escapes. + pub fn new(origin: moq_net::origin::Consumer, broadcast: moq_net::broadcast::Consumer) -> Self { + let path = broadcast.info().path.clone(); + let pins = HashMap::from([(path.clone(), broadcast)]); Self { origin, - path: path.as_path().to_owned(), - epoch: None, + path, + pins: Arc::new(Mutex::new(pins)), } } - /// This source with every request for its own path pinned to `epoch`, the instance an - /// export resolved. `None` (an epochless route) leaves requests unpinned. - pub(crate) fn pinned(mut self, epoch: Option) -> Self { - self.epoch = epoch; - self + /// This source holding nothing but `broadcast` at its own path: an export switching to + /// another instance starts over from the broadcast it switched to. + pub(crate) fn switched(&self, broadcast: &moq_net::broadcast::Consumer) -> Self { + let pins = HashMap::from([(self.path.clone(), broadcast.clone())]); + Self { + origin: self.origin.clone(), + path: self.path.clone(), + pins: Arc::new(Mutex::new(pins)), + } + } + + /// Serve this source's own path from `broadcast` from now on, as when the same instance + /// returns on a new route and the old broadcast's routes are gone. + pub(crate) fn pin(&self, broadcast: &moq_net::broadcast::Consumer) { + let mut pins = self.pins.lock().expect("pins poisoned"); + pins.insert(self.path.clone(), broadcast.clone()); } pub(crate) fn origin(&self) -> &moq_net::origin::Consumer { @@ -60,9 +81,10 @@ impl Source { &self.path } - /// Resolve and subscribe to the catalog broadcast (the one at this source's path). - pub async fn broadcast(&self) -> crate::Result { - Ok(self.request_catalog().await?) + /// The catalog broadcast, the one at this source's path. + pub fn broadcast(&self) -> moq_net::broadcast::Consumer { + let pins = self.pins.lock().expect("pins poisoned"); + pins.get(&self.path).expect("the own path is always held").clone() } /// Subscribe to this broadcast's catalog. @@ -74,19 +96,24 @@ impl Source { &self, format: crate::catalog::CatalogFormat, ) -> crate::Result> { - let broadcast = self.broadcast().await?; - crate::catalog::Consumer::new(&broadcast, format).await + crate::catalog::Consumer::new(&self.broadcast(), format).await } - /// Begin resolving the catalog broadcast (the one at this source's path). - pub(crate) fn request_catalog(&self) -> kio::Pending { - self.request_path(&self.path) + /// The catalog broadcast, as a binding. + pub(crate) fn request_catalog(&self) -> Binding { + Binding::new(self.broadcast()) } - /// Begin resolving `path`, pinned to this source's epoch when it names its own broadcast. - fn request_path(&self, path: &moq_net::PathOwned) -> kio::Pending { - let epoch = self.epoch.clone().filter(|_| *path == self.path); - self.origin.request_broadcast(path, epoch) + /// The broadcast `path` is pinned to, or a lookup that pins its answer. + fn request_path(&self, path: &moq_net::PathOwned) -> Binding { + if let Some(pinned) = self.pins.lock().expect("pins poisoned").get(path) { + return Binding::new(pinned.clone()); + } + Binding(Bound::Requested { + request: self.origin.request_broadcast(path, None).into_inner(), + pins: self.pins.clone(), + path: path.clone(), + }) } /// Resolve a rendition's optional broadcast reference to an origin path. @@ -113,32 +140,13 @@ impl Source { }) } - /// Begin resolving the broadcast that serves a rendition, honoring an optional - /// cross-broadcast reference. - /// - /// The broadcast is fetched from the origin, which deduplicates repeat requests for the same - /// reachable or dynamically served path so the catalog and every rendition share one upstream - /// subscription. - /// - /// Fails with [`Error::EscapingBroadcast`](crate::Error::EscapingBroadcast) if `rel` walks - /// above the origin root, naming no broadcast. - pub(crate) fn request( - &self, - rel: Option<&moq_net::path::Relative<'_>>, - ) -> crate::Result> { - Ok(self.request_path(&self.target(rel)?)) - } - - /// The skipping counterpart to [`Self::request`], returning `None` when `rel` walks above + /// The skipping counterpart to [`Self::bind`], returning `None` when `rel` walks above /// the origin root. /// /// The exporters use it to drop a single bad rendition rather than fail the whole export. /// [`Self::retain_valid`] already removes those renditions, so this is the second half of /// the same policy rather than a different one. - pub(crate) fn try_request( - &self, - rel: Option<&moq_net::path::Relative<'_>>, - ) -> Option> { + pub(crate) fn try_request(&self, rel: Option<&moq_net::path::Relative<'_>>) -> Option { Some(self.request_path(&self.resolve_reference(rel)?)) } @@ -196,7 +204,7 @@ impl Source { &self, rel: Option<&moq_net::path::Relative<'_>>, ) -> crate::Result { - Ok(self.request(rel)?.await?) + self.bind(rel)?.broadcast().await } /// Start one broadcast request now and retain its result for repeated reads. @@ -205,12 +213,14 @@ impl Source { /// A remote or dynamic handler may answer later. The binding never retries a failed request /// or resolves the path again after a publisher is replaced. /// - /// This always looks up the path, including for a self-reference. Use [`Binding::new`] - /// when the intended broadcast is already in hand. + /// A path this source already resolved binds to that broadcast; otherwise the origin is + /// asked, deduplicating repeat requests for one path so the catalog and every rendition share + /// one upstream subscription, and its answer is pinned. Use [`Binding::new`] when the + /// intended broadcast is already in hand. /// /// Rejects an escaping reference exactly as [`resolve`](Self::resolve) does. pub fn bind(&self, rel: Option<&moq_net::path::Relative<'_>>) -> crate::Result { - Ok(Binding(Bound::Requested(self.request(rel)?.into_inner()))) + Ok(self.request_path(&self.target(rel)?)) } /// Resolve an optional cross-broadcast reference and subscribe to track `name`, @@ -229,7 +239,7 @@ impl Source { rel: Option<&moq_net::path::Relative<'_>>, name: &str, ) -> crate::Result { - let broadcast = self.request(rel)?.await?; + let broadcast = self.bind(rel)?.broadcast().await?; Ok(broadcast.track(name)?.subscribe(None).await?) } } @@ -247,7 +257,12 @@ enum Bound { /// A broadcast the caller already holds. Ready(moq_net::broadcast::Consumer), /// A request issued when the binding was made. - Requested(moq_net::origin::Requesting), + Requested { + request: moq_net::origin::Requesting, + /// Where the answer is pinned, the first for its path winning. + pins: Pins, + path: moq_net::PathOwned, + }, } impl Binding { @@ -262,7 +277,12 @@ impl Binding { pub fn poll_broadcast(&self, waiter: &kio::Waiter) -> Poll> { match &self.0 { Bound::Ready(broadcast) => Poll::Ready(Ok(broadcast.clone())), - Bound::Requested(request) => Poll::Ready(Ok(ready!(request.poll_ok(waiter))?)), + Bound::Requested { request, pins, path } => { + let broadcast = ready!(request.poll_ok(waiter))?; + // A request that raced another for the path takes the one pinned first. + let mut pins = pins.lock().expect("pins poisoned"); + Poll::Ready(Ok(pins.entry(path.clone()).or_insert(broadcast).clone())) + } } } @@ -322,12 +342,23 @@ pub(crate) fn produce_origin() -> moq_net::origin::Producer { producer } -/// Test helper: serve `broadcast` on a throwaway origin's dynamic handler and return a -/// [`Source`] rooted at it, so exporter tests that build a local broadcast can still resolve -/// it by path. The origin is leaked so the broadcast stays reachable for the source's -/// lifetime (harmless in a test binary). +/// Test helper: a [`Source`] for the broadcast at `path`, resolved through `origin`. #[cfg(test)] -pub(crate) fn announced(broadcast: &moq_net::broadcast::Consumer) -> Source { +pub(crate) async fn resolved(origin: &moq_net::origin::Producer, path: &str) -> Source { + let consumer = origin.consume(); + let broadcast = consumer + .request_broadcast(path, None) + .await + .expect("the broadcast resolves"); + Source::new(consumer, broadcast) +} + +/// Test helper: serve `broadcast` at `test` (and every other path) on a throwaway origin's +/// dynamic handler and return a [`Source`] built from it, so exporter tests that build a +/// local broadcast still resolve references through an origin. The origin is leaked so the +/// broadcast stays reachable for the source's lifetime (harmless in a test binary). +#[cfg(test)] +pub(crate) async fn announced(broadcast: &moq_net::broadcast::Consumer) -> Source { let origin = produce_origin(); let dynamic = origin.dynamic("", Default::default()).unwrap(); let served = broadcast.clone(); @@ -336,7 +367,7 @@ pub(crate) fn announced(broadcast: &moq_net::broadcast::Consumer) -> Source { request.accept(served.clone()); } }); - let source = Source::new(origin.consume(), "test"); + let source = resolved(&origin, "test").await; Box::leak(Box::new(origin)); source } @@ -358,9 +389,11 @@ mod tests { #[tokio::test] async fn binding_retains_a_pending_request_across_cancelled_and_repeated_reads() { let origin = produce_origin(); - let dynamic = origin.dynamic("", Default::default()).unwrap(); - let source = Source::new(origin.consume(), "live"); - let binding = source.bind(None).unwrap(); + let _catalog = origin.publish("pub", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "pub").await; + let dynamic = origin.dynamic("media", Default::default()).unwrap(); + let binding = source.bind(Some(&Relative::new("media"))).unwrap(); let request = dynamic.requested_broadcast().await.unwrap(); assert!( @@ -386,45 +419,87 @@ mod tests { assert!(first.is_clone(&binding.broadcast().await.unwrap())); } + /// A failed lookup pins nothing: the guarantee starts at a path's first resolution. #[tokio::test] async fn failed_binding_does_not_retry_a_later_publisher() { let origin = produce_origin(); - let source = Source::new(origin.consume(), "live"); - let binding = source.bind(None).unwrap(); + let _catalog = origin.publish("pub", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "pub").await; + let media = Relative::new("media"); + let binding = source.bind(Some(&media)).unwrap(); assert!(binding.broadcast().await.is_err()); - let _publisher = origin.publish("live", Default::default()).unwrap(); + let _publisher = origin.publish("media", Default::default()).unwrap(); settle().await; - assert!(!source.broadcast().await.unwrap().is_closed()); + assert!(!source.resolve(Some(&media)).await.unwrap().is_closed()); assert!(binding.broadcast().await.is_err()); } - /// A source pinned to an epoch never resolves its path to a replacement, so an export's - /// later requests cannot splice another instance into it. Other paths stay unpinned. - #[tokio::test] - async fn a_pinned_source_refuses_a_replacement() { + /// A source serves its path from the broadcast it was built from, even when a replacement + /// already won the path: a track only the replacement carries is not read from it. + #[tokio::test(start_paused = true)] + async fn a_source_keeps_the_broadcast_it_was_built_from() { let origin = produce_origin(); - let first = moq_net::Epoch::mint(); - let _old = origin - .publish("live", moq_net::origin::Route::default().with_epoch(first.clone())) - .unwrap(); - let _sibling = origin.publish("sibling", Default::default()).unwrap(); + let _old = origin.publish("live", Default::default()).unwrap(); settle().await; - let source = Source::new(origin.consume(), "live").pinned(Some(first)); - source.broadcast().await.expect("the pinned instance resolves"); - - let _new = origin - .publish( - "live", - moq_net::origin::Route::default().with_epoch(moq_net::Epoch::mint()), - ) - .unwrap(); + let consumer = origin.consume(); + let first = consumer.request_broadcast("live", None).await.unwrap(); + + let new = origin.publish("live", Default::default()).unwrap(); + let _late = new.create_track("late", None).unwrap(); settle().await; - assert!(source.broadcast().await.is_err(), "the replacement is refused"); - let sibling = Relative::new("./sibling"); - source - .resolve(Some(&sibling)) - .await - .expect("another path is not pinned"); + let source = Source::new(consumer, first.clone()); + assert!(source.broadcast().is_clone(&first)); + assert!(source.resolve(None).await.unwrap().is_clone(&first)); + assert!( + source.subscribe_track(None, "late").await.is_err(), + "the replacement's track is never read" + ); + } + + /// A sibling reference stays on the broadcast it first resolved too. + #[tokio::test(start_paused = true)] + async fn a_source_keeps_the_sibling_it_resolved() { + let origin = produce_origin(); + let _catalog = origin.publish("a/pub", Default::default()).unwrap(); + let _old = origin.publish("a/source", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "a/pub").await; + let sibling = Relative::new("./source"); + let first = source.resolve(Some(&sibling)).await.expect("the sibling resolves"); + + let new = origin.publish("a/source", Default::default()).unwrap(); + let _late = new.create_track("late", None).unwrap(); + settle().await; + assert!(source.resolve(Some(&sibling)).await.unwrap().is_clone(&first)); + assert!( + source.subscribe_track(Some(&sibling), "late").await.is_err(), + "the replacement's track is never read" + ); + let bound = source.bind(Some(&sibling)).unwrap().broadcast().await.unwrap(); + assert!(bound.is_clone(&first), "a binding honors the pin too"); + } + + /// Every self-reference serves the broadcast the source was built from, even once it + /// ended and a replacement took the path. + #[tokio::test(start_paused = true)] + async fn self_references_keep_the_catalog_broadcast_after_replacement() { + let origin = produce_origin(); + let old = origin.publish("a/live", Default::default()).unwrap(); + settle().await; + let source = resolved(&origin, "a/live").await; + drop(old); + let _replacement = origin.publish("a/live", Default::default()).unwrap(); + settle().await; + + for reference in [None, Some(""), Some("live"), Some("./live"), Some("../a/live")] { + let rel = reference.map(Relative::new); + let bound = source.bind(rel.as_ref()).unwrap().broadcast().await.unwrap(); + assert!( + bound.is_closed(), + "{reference:?} must keep the original catalog broadcast" + ); + } } #[tokio::test] @@ -434,18 +509,20 @@ mod tests { _producer.announce(Default::default()).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/pub"); + let source = resolved(&origin, "a/pub").await; // No reference and an empty reference both resolve to the catalog broadcast. source - .request(None) + .bind(None) .expect("no reference is always resolvable") + .broadcast() .await .expect("catalog broadcast should resolve"); let empty = Relative::empty(); source - .request(Some(&empty)) + .bind(Some(&empty)) .expect("empty reference is always resolvable") + .broadcast() .await .expect("empty reference should resolve to the catalog broadcast"); } @@ -459,7 +536,7 @@ mod tests { let _video = producer.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/pub"); + let source = resolved(&origin, "a/pub").await; source .subscribe_track(None, "video") .await @@ -474,7 +551,7 @@ mod tests { let _video = producer.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/pub"); + let source = resolved(&origin, "a/pub").await; // Names the catalog within its own parent. let rel = Relative::new("./pub"); @@ -498,7 +575,7 @@ mod tests { let _clamped_video = clamped.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/pub"); + let source = resolved(&origin, "a/pub").await; // A reference resolves against the catalog's parent (`a`), so a second `..` walks // above the root. A lone `..` stops at the root, which still names a broadcast, so @@ -509,7 +586,7 @@ mod tests { source.resolve_reference(Some(&rel)).is_none(), "{reference} should escape" ); - assert!(source.request(Some(&rel)).is_err(), "{reference} should be rejected"); + assert!(source.bind(Some(&rel)).is_err(), "{reference} should be rejected"); // Neither `elsewhere` nor the catalog broadcast answers it: the rendition names // no broadcast at all. @@ -526,10 +603,16 @@ mod tests { } } + /// A source over a broadcast created at `a/pub`, built without awaiting the origin. + fn created(origin: &moq_net::origin::Producer) -> Source { + let broadcast = origin.create_broadcast("a/pub").unwrap().consume(); + Source::new(origin.consume(), broadcast) + } + #[test] fn escaping_rendition_is_removed_while_valid_sibling_remains() { let origin = produce_origin(); - let source = Source::new(origin.consume(), "a/pub"); + let source = created(&origin); let mut escaped = VideoConfig::new(H264 { profile: 0x42, constraints: 0, @@ -555,7 +638,7 @@ mod tests { #[test] fn escaping_text_rendition_is_removed() { let origin = produce_origin(); - let source = Source::new(origin.consume(), "a/pub"); + let source = created(&origin); let mut escaped = hang::catalog::TextConfig::new(hang::catalog::TextFormat::Vtt); escaped.broadcast = Some(Relative::new("../../source").to_owned()); @@ -583,7 +666,7 @@ mod tests { let _video = referenced.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/pub"); + let source = resolved(&origin, "a/pub").await; // The reference resolves to `a/source`, whose "video" track answers the subscribe. let rel = Relative::new("./source"); @@ -605,7 +688,7 @@ mod tests { let _video = referenced.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "a/source/transcode"); + let source = resolved(&origin, "a/source/transcode").await; let rel = Relative::new("."); source .subscribe_track(Some(&rel), "video") @@ -625,7 +708,7 @@ mod tests { let _video = root.create_track("video", None).unwrap(); settle().await; - let source = Source::new(origin.consume(), "top"); + let source = resolved(&origin, "top").await; let rel = Relative::new("."); source .subscribe_track(Some(&rel), "video") diff --git a/rs/moq-rtc/src/client/whip.rs b/rs/moq-rtc/src/client/whip.rs index a483ea024e..a82d34b97f 100644 --- a/rs/moq-rtc/src/client/whip.rs +++ b/rs/moq-rtc/src/client/whip.rs @@ -23,7 +23,8 @@ pub(crate) async fn dial( origin: moq_net::origin::Consumer, path: impl moq_net::AsPath, ) -> Result<()> { - let source = EgressSource::new(moq_mux::Source::new(origin, path)).await?; + let broadcast = origin.request_broadcast(path, None).await?; + let source = EgressSource::new(moq_mux::Source::new(origin, broadcast)).await?; let codecs = source.catalog_codecs(); if codecs.is_empty() { return Err(Error::NoRenditions); diff --git a/rs/moq-rtc/src/egress.rs b/rs/moq-rtc/src/egress.rs index 2987f0aa1f..a08f7360f5 100644 --- a/rs/moq-rtc/src/egress.rs +++ b/rs/moq-rtc/src/egress.rs @@ -328,7 +328,7 @@ mod tests { #[test] fn catalog_codecs_ignores_codecs_available_only_via_escaping_references() { let origin = produce_origin(); - let source = moq_mux::Source::new(origin.consume(), "a/pub"); + let source = moq_mux::Source::new(origin.consume(), origin.create_broadcast("a/pub").unwrap().consume()); let mut catalog = Catalog::default(); let mut escaped_audio = AudioConfig::new(AudioCodec::Opus, 48_000, 2); @@ -362,7 +362,7 @@ mod tests { #[test] fn picks_the_best_video_rendition_whatever_its_name() { let origin = produce_origin(); - let source = moq_mux::Source::new(origin.consume(), "a/pub"); + let source = moq_mux::Source::new(origin.consume(), origin.create_broadcast("a/pub").unwrap().consume()); let mut catalog = Catalog::default(); let rendition = |height: u32| { @@ -395,7 +395,7 @@ mod tests { #[test] fn picks_the_best_opus_rendition_whatever_its_name() { let origin = produce_origin(); - let source = moq_mux::Source::new(origin.consume(), "a/pub"); + let source = moq_mux::Source::new(origin.consume(), origin.create_broadcast("a/pub").unwrap().consume()); let mut catalog = Catalog::default(); let opus = |bitrate: Option| { diff --git a/rs/moq-rtc/src/server/whep.rs b/rs/moq-rtc/src/server/whep.rs index 73965b78ea..790b903f95 100644 --- a/rs/moq-rtc/src/server/whep.rs +++ b/rs/moq-rtc/src/server/whep.rs @@ -124,18 +124,19 @@ pub async fn accept( ) -> Result { let offer = sdp::parse_offer(offer)?; - // Resolve the broadcast on the subscribe origin by path. The `Source` fetches it (and any - // sibling broadcast a rendition's catalog `broadcast` field references, e.g. `./source`) - // via `request_broadcast`, which resolves an announced broadcast immediately and falls back - // to a dynamic handler; with neither it errors and the WHEP client retries (typical). - let broadcast = broadcast.as_path().to_string(); - let source = moq_mux::Source::new(subscriber.clone(), &broadcast); - + // Resolve the broadcast on the subscribe origin by path. `request_broadcast` resolves an + // announced broadcast immediately and falls back to a dynamic handler; with neither it + // errors and the WHEP client retries (typical). The `Source` resolves any sibling broadcast + // a rendition's catalog `broadcast` field references (e.g. `./source`) the same way. + // // Bound the wait for resolution + the first catalog: an unannounced broadcast, or an // announced-but-catalog-less one, would otherwise park this handler forever. - let source = tokio::time::timeout(CATALOG_TIMEOUT, EgressSource::new(source)) - .await - .map_err(|_| Error::CatalogTimeout)??; + let source = tokio::time::timeout(CATALOG_TIMEOUT, async { + let broadcast = subscriber.request_broadcast(broadcast.as_path(), None).await?; + EgressSource::new(moq_mux::Source::new(subscriber.clone(), broadcast)).await + }) + .await + .map_err(|_| Error::CatalogTimeout)??; let codecs = source.catalog_codecs(); if codecs.is_empty() { return Err(Error::NoRenditions); diff --git a/rs/moq-rtmp/src/dial.rs b/rs/moq-rtmp/src/dial.rs index 7ee9758c80..490c1843ce 100644 --- a/rs/moq-rtmp/src/dial.rs +++ b/rs/moq-rtmp/src/dial.rs @@ -227,7 +227,8 @@ impl Client { let queued = std::mem::take(&mut self.work); self.drain(queued).await?; - let source = moq_mux::Source::new(origin, path); + let broadcast = origin.request_broadcast(path, None).await?; + let source = moq_mux::Source::new(origin, broadcast); let catalog = source .catalog::<()>(moq_mux::catalog::CatalogFormat::default()) .await diff --git a/rs/moq-rtmp/src/server.rs b/rs/moq-rtmp/src/server.rs index 33d197afab..13b5f68318 100644 --- a/rs/moq-rtmp/src/server.rs +++ b/rs/moq-rtmp/src/server.rs @@ -828,10 +828,10 @@ impl Play { }; // The capability check consumed its snapshot, so the export follows a fresh - // subscription narrowed to what the client can play. It resolves through the - // same source as the tracks, so a publisher that reconnected since the check - // doesn't split the catalog from its media. - let source = moq_mux::Source::new(origin.consume(), path.as_str()); + // subscription narrowed to what the client can play. It reads the broadcast the + // check resolved, so a publisher that reconnected since then doesn't split the + // catalog from its media. + let source = moq_mux::Source::new(origin.clone(), broadcast); let stream = source .catalog::<()>(CatalogFormat::default()) .await diff --git a/rs/moq-srt/src/ts.rs b/rs/moq-srt/src/ts.rs index b3fdf755bb..ed69110e45 100644 --- a/rs/moq-srt/src/ts.rs +++ b/rs/moq-srt/src/ts.rs @@ -204,9 +204,10 @@ impl Subscriber { return Ok(None); } - // The export resolves the broadcast (and any referenced sibling broadcast, via the - // catalog `broadcast` field) through the origin, joining the one just announced. - let source = moq_mux::Source::new(origin.consume(), path); + // Join the broadcast just announced. The export resolves any referenced sibling + // broadcast (the catalog `broadcast` field) through the same origin. + let broadcast = origin.request_broadcast(path, None).await?; + let source = moq_mux::Source::new(origin.clone(), broadcast); let export = ts::Export::with_ts(source, moq_mux::catalog::CatalogFormat::Hang) .await? .with_delay(options.latency); diff --git a/rs/moq-video/src/decode/consumer.rs b/rs/moq-video/src/decode/consumer.rs index 7704dd365b..4add91aca9 100644 --- a/rs/moq-video/src/decode/consumer.rs +++ b/rs/moq-video/src/decode/consumer.rs @@ -248,17 +248,10 @@ mod tests { } let origin = produce_origin(); - let requests = origin.dynamic("", Default::default()).unwrap(); - let served = source_subscriber.clone(); - tokio::spawn(async move { - while let Ok(request) = requests.requested_broadcast().await { - request.accept(served.clone()); - } - }); let catalog = moq_mux::catalog::Consumer::<()>::new(&source_subscriber, moq_mux::catalog::CatalogFormat::Hang) .await .unwrap(); - let source = moq_mux::Source::new(origin.consume(), "test"); + let source = moq_mux::Source::new(origin.consume(), source_subscriber.clone()); // Both frames are encoded before the export runs, so the exporter needs a budget // wide enough to read them: its REAL_TIME default keeps only the live edge, and // the second `next()` would then block forever waiting for a group that was diff --git a/rs/moq-video/src/decode/decoder.rs b/rs/moq-video/src/decode/decoder.rs index 732930d15a..9fa8ba8ec3 100644 --- a/rs/moq-video/src/decode/decoder.rs +++ b/rs/moq-video/src/decode/decoder.rs @@ -565,11 +565,13 @@ mod tests { .unwrap(); producer.finish().unwrap(); - let catalog = moq_mux::catalog::Consumer::<()>::new(&source.consume(), moq_mux::catalog::CatalogFormat::Hang) + let broadcast = origin.consume().request_broadcast("test", None).await.unwrap(); + let catalog = moq_mux::catalog::Consumer::<()>::new(&broadcast, moq_mux::catalog::CatalogFormat::Hang) .await .unwrap(); - let mut export = moq_mux::container::fmp4::Export::new(moq_mux::Source::new(origin.consume(), "test"), catalog) - .with_max_delay(std::time::Duration::from_secs(30)); + let mut export = + moq_mux::container::fmp4::Export::new(moq_mux::Source::new(origin.consume(), broadcast), catalog) + .with_max_delay(std::time::Duration::from_secs(30)); let init = export.next().await.unwrap().expect("CMAF init"); let fragment = export.next().await.unwrap().expect("CMAF fragment");