From 24fce794c378fe3ff12bb76062db55da710ef3d1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 00:40:05 -0700 Subject: [PATCH 1/7] claim quest/m1/archive/hls-media-sequence Co-Authored-By: Grok 4.7 From 0cf54bf15c7a1c22f07d6dc4b6b3bb4a010f90cf Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 00:52:04 -0700 Subject: [PATCH 2/7] fix(moq-hls): keep the HLS reference so the media sequence cannot rewind Co-Authored-By: Grok 4.7 --- doc/bin/hls.md | 10 +- drafts/draft-lcurley-moq-hang.md | 3 +- quest/m1/archive/README.md | 1 - quest/m1/archive/hls-media-sequence.md | 37 ----- rs/moq-hls/src/export/mod.rs | 179 +++++++++++++++++++++++-- 5 files changed, 173 insertions(+), 57 deletions(-) delete mode 100644 quest/m1/archive/hls-media-sequence.md diff --git a/doc/bin/hls.md b/doc/bin/hls.md index c1b86864fa..ef8842147b 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -25,7 +25,9 @@ segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it, and idle renditions cost nothing. Segment boundaries come from one reference rendition (the first video rendition by name, or the first audio one without video) and are numbered by -its records, so every edge and every reload agree. Every other video rendition +its records. That choice sticks while the rendition stays in the catalog, so a +newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`. +Every other video rendition snaps each boundary to its nearest keyframe within about a second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame inside the segment's span. A jump in content time renders as @@ -49,9 +51,9 @@ broadcast by path: Segment boundaries come from one reference rendition's timeline, and its records number the segments. `{reference}` is a short hash of that rendition's kind and -name, so every edge derives the same URL, and a reference that changes (a new -first video rendition) starts a new numbering under new URLs rather than reusing -the old ones for other content. +name, so every edge that chose it derives the same URL. A reference that leaves +the catalog starts a new numbering under new URLs rather than reusing the old +ones for other content. A [`moq-archive`](https://docs.rs/moq-archive) recording replayed through its `Reader` is served the same way, with no second stored copy. Playlists come diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index 38dc856715..2869a82e8a 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -770,7 +770,7 @@ The final record of an ended track has no successor; its `duration` runs to the ## Derived Formats {#timeline-derived} Segmented formats such as HLS and DASH need one segment numbering across renditions, which per-track timelines do not provide on their own. -An edge deriving one takes the segment boundaries from a reference rendition's records, the first video rendition by name with a timeline, or the first such audio rendition when there is none, and numbers the segments by that rendition's record `sequence`, so every edge and every reload agree. +An edge deriving one chooses the first video rendition by name with a timeline, or the first such audio rendition when there is none, and keeps it while that rendition stays in the catalog. Segment boundaries come from its records, numbered by record `sequence`. Another video rendition snaps each boundary to its nearest record that starts a group on a keyframe; a segment with no such record nearby has no content on that rendition (a gap; HLS `EXT-X-GAP`). Other renditions take the frames whose timestamps fall inside each segment's span. A publisher wanting such an export SHOULD start video groups at the same timestamps across renditions. @@ -1068,6 +1068,7 @@ This document has no IANA actions. - One cutting rule for every track: a record ends at the first group boundary past a minimum (2 seconds RECOMMENDED, zero for sparse data such as a catalog) and splits a group between frames at a maximum (10 seconds RECOMMENDED), so a group that never closes is indexed as it grows and `durationMax` bounds every record. - Recording format version 2: each track stores record N at `segments/N`, beside its timeline's `segments/N`, with a `Frame Start` field in the segment object. Tracks commit and expire independently, and a DVR keeps each track's newest record. - Described deriving HLS and DASH at the edge from a reference rendition's records. +- An edge keeps its derived-format reference rendition while that rendition stays in the catalog. ## moq-hang-03 {:numbered="false"} diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index feb8ad71e6..e37f274422 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -162,7 +162,6 @@ owned by that prerequisite, not duplicated in archive storage. - [Replay catalog](/quest/m1/archive/replay-catalog.md) - `moq_archive::Reader` republishes the recorded catalog live with `store` set, so stock `moq export hls` serves the whole replay of `moq import archive` - [Idle flush](/quest/m1/archive/flush.md) - idle tracks are recorded within a bounded wall-clock delay, and `flush()` forces a track's pending record out - [Backward timestamps](/quest/m1/archive/backward-timestamps.md) - a resumed recording refuses a restarted source instead of dropping its groups or overlapping media time -- [HLS media sequence](/quest/m1/archive/hls-media-sequence.md) - `EXT-X-MEDIA-SEQUENCE` never decreases when the reference rendition switches - [DVR rewind](/quest/m1/archive/dvr.md) - seek through a bounded archive and return to live playback ## Related diff --git a/quest/m1/archive/hls-media-sequence.md b/quest/m1/archive/hls-media-sequence.md deleted file mode 100644 index 836e9be983..0000000000 --- a/quest/m1/archive/hls-media-sequence.md +++ /dev/null @@ -1,37 +0,0 @@ -# [S] HLS media sequence survives a reference switch - -## Goal - -A media playlist's `EXT-X-MEDIA-SEQUENCE` never decreases, as -[RFC 8216 §6.2.2](https://www.rfc-editor.org/rfc/rfc8216.html#section-6.2.2) -requires, even when the reference rendition changes. Players keep loading the -same `media.m3u8` across the switch without stalling. - -## Plan - -Today `moq-hls` numbers segments by the reference rendition's own records. A -switch (a new first video rendition with a timeline, or the reference leaving -the catalog) starts a new numbering under a new URL tag -(`seg/{reference}.{segment}.m4s`, #4034). Its URLs never collide and a -discontinuity marks the switch, but the media sequence can rewind. - -Open design, pick one: - -- **Sticky reference.** Keep the reference while it stays in the catalog, so a - new, earlier-named rendition no longer switches it. Cheap and edge-consistent, - but a reference that leaves the catalog still rewinds. -- **Offset at the switch.** Number the new reference's segments from past the - last listed one. Monotonic, but an edge that starts after the switch numbers - differently, so edges disagree on the media sequence. - -Recommendation: the sticky reference first. It removes the common trigger -without giving up the property that every edge agrees on numbering. Measure -whether a reference leaving the catalog happens in practice before taking on -the offset. - -Test a switch through repeated playlist reloads: the media sequence never -decreases, and no URL names other content. - -## Related - -- [Fixed HLS target duration](/quest/m1/archive/hls-target.md) - also changes how the reference shapes the playlist diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index aac8e24e0c..484192b2e0 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -8,10 +8,12 @@ //! way and is skipped. A `moq-archive` recording replayed onto a broadcast is served the same way: //! its playlists read only the stored timelines, and a segment GETs only its rendition's objects. //! -//! Segment boundaries come from one reference rendition's records (the first video rendition, or -//! the first audio one when there is no video), numbered by record sequence, so every edge and -//! every reload agree without the publisher cutting for HLS. Every other rendition resolves each -//! segment against its own timeline. +//! Segment boundaries come from one reference rendition's records, numbered by record sequence, +//! so edges that share the choice agree without the publisher cutting for HLS. The first choice +//! is the first video rendition, or the first audio one when there is no video, and it sticks +//! while that rendition stays in the catalog: a newer rendition that sorts earlier must not +//! rewind `EXT-X-MEDIA-SEQUENCE`. Every other rendition resolves each segment against its own +//! timeline. //! An inline-parameter-set codec with no catalog `description` GETs one keyframe //! group on the first playlist render to build its init, then caches it. //! @@ -361,8 +363,8 @@ async fn watch_catalog( } renditions.sync(&upstream, &catalog); - // Follow the reference rendition's timeline; a new reference renumbers every segment. - let next = self::reference(&catalog); + // Keep the reference while it stays. Replacing it renumbers every segment. + let next = self::reference(&catalog, reference.as_ref()); if next != reference { if let Some(watcher) = timeline_watcher.lock().unwrap().take() { watcher.abort(); @@ -392,24 +394,38 @@ async fn watch_catalog( renditions.close(); } -/// The rendition every segment boundary comes from: the first video rendition with a timeline, -/// or the first audio one when the broadcast has no video. Renditions are ordered by name, so -/// every edge picks the same one. -fn reference(catalog: &moq_mux::catalog::hang::Catalog) -> Option { +/// The rendition every segment boundary comes from. +/// +/// The first choice is the first video rendition with a timeline, or the first audio one when +/// the broadcast has no video. Renditions are ordered by name, so edges choosing together pick +/// the same one. `current` stays while it is still a rendition with a timeline: a newer +/// rendition that sorts earlier must not take over, or `EXT-X-MEDIA-SEQUENCE` rewinds. A +/// reference that leaves falls back to that first choice, which can still rewind. +fn reference(catalog: &moq_mux::catalog::hang::Catalog, current: Option<&Reference>) -> Option { let archive = catalog.archive.as_ref()?; - let indexed = |name: &&String| archive.timelines.contains_key(*name); + let indexed = |name: &str| archive.timelines.contains_key(name); + if let Some(current) = current { + let (kind, name) = current.as_ref(); + let listed = match kind { + Kind::Video => catalog.video.renditions.contains_key(name), + Kind::Audio => catalog.audio.renditions.contains_key(name), + }; + if listed && indexed(name) { + return Some(Arc::clone(current)); + } + } let video = catalog .video .renditions .keys() - .find(indexed) + .find(|name| indexed(name)) .map(|name| (Kind::Video, name)); let audio = || { catalog .audio .renditions .keys() - .find(indexed) + .find(|name| indexed(name)) .map(|name| (Kind::Audio, name)) }; video.or_else(audio).map(|(kind, name)| Arc::new((kind, name.clone()))) @@ -2879,7 +2895,7 @@ mod tests { let watcher = tokio::spawn(watch_timeline( upstream.broadcast.clone(), archive, - reference(&catalog).unwrap(), + reference(&catalog, None).unwrap(), renditions.fanout(), )); Self { @@ -3035,6 +3051,141 @@ mod tests { ); } + // RFC 8216 §6.2.2: EXT-X-MEDIA-SEQUENCE must not decrease across reloads. A rendition that + // sorts ahead of the reference must not take over, or the playlist rewinds to that + // rendition's own record numbers and its URLs name other content. + #[tokio::test] + async fn an_earlier_rendition_does_not_rewind_the_media_sequence() { + const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; + const NEW: &[u8] = b"NEWNEWNEWNEWNEWN"; + + let origin = produce_origin(); + let mut broadcast = origin.create_broadcast("live").expect("publish allowed"); + broadcast.announce(Default::default()).expect("publish allowed"); + settle().await; + let catalog = moq_mux::catalog::Producer::new(&mut broadcast, moq_mux::catalog::Config::default()).unwrap(); + + let reserved = catalog.reserve(); + let mut video0 = catalog.test_video("video0").unwrap(); + video0.set(video_config()).unwrap(); + drop(reserved); + + let track = broadcast.create_track("video0", None).unwrap(); + let mut media = catalog + .media_producer( + track, + moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Data), + ) + .unwrap(); + // 2s GOPs against a 6s window, so the oldest records leave and the sequence is no longer 0. + for micros in (0..=10_000_000u64).step_by(2_000_000) { + media.write(payload_frame(micros, true, OLD)).unwrap(); + } + + let source = moq_mux::Source::new(origin.consume(), "live"); + let config = Config { + window: Duration::from_secs(6), + }; + let broadcaster = Broadcaster::new(source, config).await.unwrap(); + let sequence_of = |playlist: &str| -> u64 { + between(playlist, "#EXT-X-MEDIA-SEQUENCE:", "\n") + .parse() + .expect("media sequence") + }; + let rendition = tokio::time::timeout(Duration::from_secs(5), async { + loop { + if let Some(rendition) = broadcaster.rendition(Kind::Video, "video0") + && let Some(playlist) = rendition.media_playlist(None) + && sequence_of(&playlist) >= 2 + { + break rendition; + } + tokio::task::yield_now().await; + } + }) + .await + .expect("video0 playlist slid past its first segments"); + + let before = rendition.media_playlist(None).expect("playable"); + let sequence = sequence_of(&before); + assert!(sequence >= 2, "the window should have slid, got {sequence}\n{before}"); + let tag = segments::tag(&(Kind::Video, "video0".to_string())); + assert!(before.contains(&format!("seg/{tag}.{sequence}.m4s\n")), "{before}"); + let bytes = rendition + .listed_segment(sequence) + .await + .unwrap() + .expect("listed segment"); + assert!(contains(&bytes, OLD), "the listed segment is not video0's media"); + + // Enroll the earlier timeline before the rendition, so the catalog snapshot that adds + // the rendition already indexes it. The reference decision runs in that same update. + let earlier = broadcast.create_track("a", None).unwrap(); + let mut earlier_media = catalog + .media_producer( + earlier, + moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Data), + ) + .unwrap(); + for micros in (0..=4_000_000u64).step_by(2_000_000) { + earlier_media.write(payload_frame(micros, true, NEW)).unwrap(); + } + let mut added = catalog.test_video("a").unwrap(); + added.set(video_config()).unwrap(); + + tokio::time::timeout(Duration::from_secs(5), async { + loop { + if broadcaster.rendition(Kind::Video, "a").is_some() { + break; + } + tokio::task::yield_now().await; + } + }) + .await + .expect("the earlier rendition is in the catalog"); + + let new_tag = segments::tag(&(Kind::Video, "a".to_string())); + let deadline = tokio::time::Instant::now() + Duration::from_secs(5); + let mut previous = sequence; + let mut reloads = 0; + while reloads < 5 && tokio::time::Instant::now() < deadline { + let Some(playlist) = rendition.media_playlist(None) else { + tokio::time::sleep(Duration::from_millis(10)).await; + continue; + }; + let next = sequence_of(&playlist); + assert!( + next >= previous, + "EXT-X-MEDIA-SEQUENCE decreased from {previous} to {next}\n{playlist}" + ); + assert!( + playlist.contains(&format!("seg/{tag}.")), + "segment URLs left the original reference\n{playlist}" + ); + assert!( + !playlist.contains(&format!("seg/{new_tag}.")), + "the earlier rendition's numbering replaced the reference\n{playlist}" + ); + previous = next; + reloads += 1; + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert!(reloads >= 5, "the playlist did not stay reloadable"); + + let again = rendition + .listed_segment(sequence) + .await + .unwrap() + .expect("the original segment URL still resolves"); + assert_eq!(again, bytes, "the original URL names other content"); + assert!( + !contains(&again, NEW), + "the earlier rendition's media replaced the segment" + ); + + drop((media, earlier_media, video0, added, broadcast)); + } + // A new reference numbers segments from its own records, so its URLs carry its own tag and // never reuse one the old reference listed for other content. #[tokio::test(start_paused = true)] From 44e8be0735640fd511ae21b74b02c5dfb5ec8509 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 13:20:22 -0700 Subject: [PATCH 3/7] test(hls): mock time in the media-sequence regression; leave room for new Config fields Co-Authored-By: Claude Opus 5.5 --- rs/moq-hls/src/export/mod.rs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 484192b2e0..3afd1cc678 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -3054,7 +3054,7 @@ mod tests { // RFC 8216 §6.2.2: EXT-X-MEDIA-SEQUENCE must not decrease across reloads. A rendition that // sorts ahead of the reference must not take over, or the playlist rewinds to that // rendition's own record numbers and its URLs name other content. - #[tokio::test] + #[tokio::test(start_paused = true)] async fn an_earlier_rendition_does_not_rewind_the_media_sequence() { const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; const NEW: &[u8] = b"NEWNEWNEWNEWNEWN"; @@ -3085,6 +3085,7 @@ mod tests { let source = moq_mux::Source::new(origin.consume(), "live"); let config = Config { window: Duration::from_secs(6), + ..Config::default() }; let broadcaster = Broadcaster::new(source, config).await.unwrap(); let sequence_of = |playlist: &str| -> u64 { From 6e0e637bb2caafb57ea217d93087489a390f2ba7 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 16:23:31 -0700 Subject: [PATCH 4/7] test(hls): pin the sticky reference and its fallback in reference() Co-Authored-By: Claude Opus 5.5 --- rs/moq-hls/src/export/mod.rs | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 3afd1cc678..2b463f66b2 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -3187,6 +3187,26 @@ mod tests { drop((media, earlier_media, video0, added, broadcast)); } + // The reference sticks while it stays listed with a timeline, and falls back to the first + // choice once it leaves. + #[test] + fn the_reference_sticks_until_it_leaves() { + let catalog = |names: &[&str]| { + let mut catalog = moq_mux::catalog::hang::Catalog::default(); + for name in names { + catalog.video.renditions.insert(name.to_string(), video_config()); + } + catalog.archive = Some(archive(names)); + catalog + }; + let video = |name: &str| Arc::new((Kind::Video, name.to_string())); + + let first = reference(&catalog(&["b"]), None); + assert_eq!(first, Some(video("b"))); + assert_eq!(reference(&catalog(&["a", "b"]), first.as_ref()), Some(video("b"))); + assert_eq!(reference(&catalog(&["a", "c"]), first.as_ref()), Some(video("a"))); + } + // A new reference numbers segments from its own records, so its URLs carry its own tag and // never reuse one the old reference listed for other content. #[tokio::test(start_paused = true)] From 777bed15800a15d84447a65dabf197465edf913f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 16:54:57 -0700 Subject: [PATCH 5/7] fix(hls): an audio reference yields once video has a timeline Keeping an audio reference after video appeared cut every video segment on audio records, so most became gaps. A video reference still sticks; an audio one switches to video once, which rewinds the media sequence as a reference that leaves does. Co-Authored-By: Claude Opus 5.5 --- doc/bin/hls.md | 8 ++--- drafts/draft-lcurley-moq-hang.md | 4 +-- rs/moq-hls/src/export/mod.rs | 58 ++++++++++++++++++++------------ 3 files changed, 43 insertions(+), 27 deletions(-) diff --git a/doc/bin/hls.md b/doc/bin/hls.md index ef8842147b..b4f79d2575 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -25,10 +25,10 @@ segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it, and idle renditions cost nothing. Segment boundaries come from one reference rendition (the first video rendition by name, or the first audio one without video) and are numbered by -its records. That choice sticks while the rendition stays in the catalog, so a -newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`. -Every other video rendition -snaps each boundary to its nearest keyframe within about a second, and a +its records. A video choice sticks while the rendition stays in the catalog, so +a newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`; an +audio choice switches once to video when a video rendition appears. Every other +video rendition snaps each boundary to its nearest keyframe within about a second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame inside the segment's span. A jump in content time renders as `EXT-X-DISCONTINUITY`. Gaps are a fallback: a publisher wanting clean HLS diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index 2869a82e8a..daab40d168 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -770,7 +770,7 @@ The final record of an ended track has no successor; its `duration` runs to the ## Derived Formats {#timeline-derived} Segmented formats such as HLS and DASH need one segment numbering across renditions, which per-track timelines do not provide on their own. -An edge deriving one chooses the first video rendition by name with a timeline, or the first such audio rendition when there is none, and keeps it while that rendition stays in the catalog. Segment boundaries come from its records, numbered by record `sequence`. +An edge deriving one chooses the first video rendition by name with a timeline, or the first such audio rendition when there is none, and keeps a video choice while that rendition stays in the catalog; an audio choice yields to the first video rendition with a timeline once there is one. Segment boundaries come from its records, numbered by record `sequence`. Another video rendition snaps each boundary to its nearest record that starts a group on a keyframe; a segment with no such record nearby has no content on that rendition (a gap; HLS `EXT-X-GAP`). Other renditions take the frames whose timestamps fall inside each segment's span. A publisher wanting such an export SHOULD start video groups at the same timestamps across renditions. @@ -1068,7 +1068,7 @@ This document has no IANA actions. - One cutting rule for every track: a record ends at the first group boundary past a minimum (2 seconds RECOMMENDED, zero for sparse data such as a catalog) and splits a group between frames at a maximum (10 seconds RECOMMENDED), so a group that never closes is indexed as it grows and `durationMax` bounds every record. - Recording format version 2: each track stores record N at `segments/N`, beside its timeline's `segments/N`, with a `Frame Start` field in the segment object. Tracks commit and expire independently, and a DVR keeps each track's newest record. - Described deriving HLS and DASH at the edge from a reference rendition's records. -- An edge keeps its derived-format reference rendition while that rendition stays in the catalog. +- An edge keeps its derived-format video reference rendition while that rendition stays in the catalog; an audio reference yields once video has a timeline. ## moq-hang-03 {:numbered="false"} diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 2b463f66b2..17c4394bbe 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -10,9 +10,9 @@ //! //! Segment boundaries come from one reference rendition's records, numbered by record sequence, //! so edges that share the choice agree without the publisher cutting for HLS. The first choice -//! is the first video rendition, or the first audio one when there is no video, and it sticks -//! while that rendition stays in the catalog: a newer rendition that sorts earlier must not -//! rewind `EXT-X-MEDIA-SEQUENCE`. Every other rendition resolves each segment against its own +//! is the first video rendition, or the first audio one when there is no video. A video choice +//! sticks while that rendition stays in the catalog: a newer rendition that sorts earlier must +//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches once to video. Every other rendition resolves each segment against its own //! timeline. //! An inline-parameter-set codec with no catalog `description` GETs one keyframe //! group on the first playlist render to build its init, then caches it. @@ -398,28 +398,29 @@ async fn watch_catalog( /// /// The first choice is the first video rendition with a timeline, or the first audio one when /// the broadcast has no video. Renditions are ordered by name, so edges choosing together pick -/// the same one. `current` stays while it is still a rendition with a timeline: a newer -/// rendition that sorts earlier must not take over, or `EXT-X-MEDIA-SEQUENCE` rewinds. A -/// reference that leaves falls back to that first choice, which can still rewind. +/// the same one. A video `current` stays while it is still a rendition with a timeline: a newer +/// rendition that sorts earlier must not take over, or `EXT-X-MEDIA-SEQUENCE` rewinds. An audio +/// `current` stays only until a video rendition has a timeline, since video segments cut on +/// audio records would mostly be gaps; that one switch, like a reference that leaves, rewinds. fn reference(catalog: &moq_mux::catalog::hang::Catalog, current: Option<&Reference>) -> Option { let archive = catalog.archive.as_ref()?; let indexed = |name: &str| archive.timelines.contains_key(name); + let video = catalog + .video + .renditions + .keys() + .find(|name| indexed(name)) + .map(|name| (Kind::Video, name)); if let Some(current) = current { let (kind, name) = current.as_ref(); let listed = match kind { Kind::Video => catalog.video.renditions.contains_key(name), - Kind::Audio => catalog.audio.renditions.contains_key(name), + Kind::Audio => video.is_none() && catalog.audio.renditions.contains_key(name), }; if listed && indexed(name) { return Some(Arc::clone(current)); } } - let video = catalog - .video - .renditions - .keys() - .find(|name| indexed(name)) - .map(|name| (Kind::Video, name)); let audio = || { catalog .audio @@ -3187,24 +3188,39 @@ mod tests { drop((media, earlier_media, video0, added, broadcast)); } - // The reference sticks while it stays listed with a timeline, and falls back to the first - // choice once it leaves. + // A video reference sticks while it stays listed with a timeline, and falls back to the + // first choice once it leaves. An audio reference yields once video has a timeline. #[test] fn the_reference_sticks_until_it_leaves() { - let catalog = |names: &[&str]| { + let catalog = |videos: &[&str], audios: &[&str]| { let mut catalog = moq_mux::catalog::hang::Catalog::default(); - for name in names { + for name in videos { catalog.video.renditions.insert(name.to_string(), video_config()); } - catalog.archive = Some(archive(names)); + for name in audios { + catalog.audio.renditions.insert( + name.to_string(), + hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Opus, 48_000, 2), + ); + } + catalog.archive = Some(archive(&[videos, audios].concat())); catalog }; let video = |name: &str| Arc::new((Kind::Video, name.to_string())); + let audio = |name: &str| Arc::new((Kind::Audio, name.to_string())); - let first = reference(&catalog(&["b"]), None); + let first = reference(&catalog(&["b"], &[]), None); assert_eq!(first, Some(video("b"))); - assert_eq!(reference(&catalog(&["a", "b"]), first.as_ref()), Some(video("b"))); - assert_eq!(reference(&catalog(&["a", "c"]), first.as_ref()), Some(video("a"))); + assert_eq!(reference(&catalog(&["a", "b"], &[]), first.as_ref()), Some(video("b"))); + assert_eq!(reference(&catalog(&["a", "c"], &[]), first.as_ref()), Some(video("a"))); + + let first = reference(&catalog(&[], &["b"]), None); + assert_eq!(first, Some(audio("b"))); + assert_eq!(reference(&catalog(&[], &["a", "b"]), first.as_ref()), Some(audio("b"))); + assert_eq!( + reference(&catalog(&["v"], &["a", "b"]), first.as_ref()), + Some(video("v")) + ); } // A new reference numbers segments from its own records, so its URLs carry its own tag and From 65c68674dc08826378e7a1f1432b31dc0be992a6 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 17:57:21 -0700 Subject: [PATCH 6/7] test(hls): pin that video without a timeline leaves an audio reference in place Co-Authored-By: Claude Opus 5.5 --- doc/bin/hls.md | 2 +- rs/moq-hls/src/export/mod.rs | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/doc/bin/hls.md b/doc/bin/hls.md index b4f79d2575..066ce367d2 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -27,7 +27,7 @@ Segment boundaries come from one reference rendition (the first video rendition by name, or the first audio one without video) and are numbered by its records. A video choice sticks while the rendition stays in the catalog, so a newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`; an -audio choice switches once to video when a video rendition appears. Every other +audio choice switches to video when a video rendition appears. Every other video rendition snaps each boundary to its nearest keyframe within about a second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame inside the segment's span. A jump in content time renders as diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 17c4394bbe..2f41947158 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -12,7 +12,7 @@ //! so edges that share the choice agree without the publisher cutting for HLS. The first choice //! is the first video rendition, or the first audio one when there is no video. A video choice //! sticks while that rendition stays in the catalog: a newer rendition that sorts earlier must -//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches once to video. Every other rendition resolves each segment against its own +//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches to video when one appears. Every other rendition resolves each segment against its own //! timeline. //! An inline-parameter-set codec with no catalog `description` GETs one keyframe //! group on the first playlist render to build its init, then caches it. @@ -400,8 +400,8 @@ async fn watch_catalog( /// the broadcast has no video. Renditions are ordered by name, so edges choosing together pick /// the same one. A video `current` stays while it is still a rendition with a timeline: a newer /// rendition that sorts earlier must not take over, or `EXT-X-MEDIA-SEQUENCE` rewinds. An audio -/// `current` stays only until a video rendition has a timeline, since video segments cut on -/// audio records would mostly be gaps; that one switch, like a reference that leaves, rewinds. +/// `current` stays only while no video rendition has a timeline, since video segments cut on +/// audio records would mostly be gaps; that switch, like a reference that leaves, rewinds. fn reference(catalog: &moq_mux::catalog::hang::Catalog, current: Option<&Reference>) -> Option { let archive = catalog.archive.as_ref()?; let indexed = |name: &str| archive.timelines.contains_key(name); @@ -3217,6 +3217,10 @@ mod tests { let first = reference(&catalog(&[], &["b"]), None); assert_eq!(first, Some(audio("b"))); assert_eq!(reference(&catalog(&[], &["a", "b"]), first.as_ref()), Some(audio("b"))); + // Video listed before its timeline exists doesn't take over yet. + let mut pending = catalog(&[], &["b"]); + pending.video.renditions.insert("v".to_string(), video_config()); + assert_eq!(reference(&pending, first.as_ref()), Some(audio("b"))); assert_eq!( reference(&catalog(&["v"], &["a", "b"]), first.as_ref()), Some(video("v")) From dac9c3f42116cc6850fb2747cc80d87d1fa91907 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Wed, 7 Oct 2026 18:07:46 -0700 Subject: [PATCH 7/7] docs(hls): audio yields once a video rendition has a timeline Co-Authored-By: Claude Opus 5.5 --- doc/bin/hls.md | 6 +++--- rs/moq-hls/src/export/mod.rs | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/bin/hls.md b/doc/bin/hls.md index 066ce367d2..40900fdde1 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -27,9 +27,9 @@ Segment boundaries come from one reference rendition (the first video rendition by name, or the first audio one without video) and are numbered by its records. A video choice sticks while the rendition stays in the catalog, so a newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`; an -audio choice switches to video when a video rendition appears. Every other -video rendition snaps each boundary to its nearest keyframe within about a second, and a -segment with none in range renders as `EXT-X-GAP`; audio takes every frame +audio choice switches to video once a video rendition has a timeline. Every +other video rendition snaps each boundary to its nearest keyframe within about a +second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame inside the segment's span. A jump in content time renders as `EXT-X-DISCONTINUITY`. Gaps are a fallback: a publisher wanting clean HLS export should align video GOPs across renditions. A timeline that fails diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index 2f41947158..adad54fec5 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -12,8 +12,8 @@ //! so edges that share the choice agree without the publisher cutting for HLS. The first choice //! is the first video rendition, or the first audio one when there is no video. A video choice //! sticks while that rendition stays in the catalog: a newer rendition that sorts earlier must -//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches to video when one appears. Every other rendition resolves each segment against its own -//! timeline. +//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches to video once a video rendition +//! has a timeline. Every other rendition resolves each segment against its own timeline. //! An inline-parameter-set codec with no catalog `description` GETs one keyframe //! group on the first playlist render to build its init, then caches it. //!