diff --git a/doc/bin/hls.md b/doc/bin/hls.md index 85b0688e1d..81739d3ad2 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -25,9 +25,11 @@ segment is servable for as long as the relay's [cache](/bin/relay/config#cache) retains it, and idle renditions cost nothing. Segment boundaries come from one reference rendition (the first video rendition by name, or the first audio one without video) and are numbered by -its records, so every edge and every reload agree. Every other video rendition -snaps each boundary to its nearest keyframe within about a second, and a -segment with none in range renders as `EXT-X-GAP`; audio takes every frame +its records. A video choice sticks while the rendition stays in the catalog, so +a newer rendition that sorts earlier does not rewind `EXT-X-MEDIA-SEQUENCE`; an +audio choice switches to video once a video rendition has a timeline. Every +other video rendition snaps each boundary to its nearest keyframe within about a +second, and a segment with none in range renders as `EXT-X-GAP`; audio takes every frame inside the segment's span. A jump in content time renders as `EXT-X-DISCONTINUITY`. Gaps are a fallback: a publisher wanting clean HLS export should align video GOPs across renditions. A timeline that fails @@ -49,9 +51,9 @@ broadcast by path: Segment boundaries come from one reference rendition's timeline, and its records number the segments. `{reference}` is a short hash of that rendition's kind and -name, so every edge derives the same URL, and a reference that changes (a new -first video rendition) starts a new numbering under new URLs rather than reusing -the old ones for other content. +name, so every edge that chose it derives the same URL. A reference that leaves +the catalog starts a new numbering under new URLs rather than reusing the old +ones for other content. A [`moq-archive`](https://docs.rs/moq-archive) recording replayed through its `Reader` is served the same way, with no second stored copy. Playlists come diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index bc6eb27c50..b3666b5520 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -773,7 +773,7 @@ The final record of an ended track has no successor; its `duration` runs to the ## Derived Formats {#timeline-derived} Segmented formats such as HLS and DASH need one segment numbering across renditions, which per-track timelines do not provide on their own. -An edge deriving one takes the segment boundaries from a reference rendition's records, the first video rendition by name with a timeline, or the first such audio rendition when there is none, and numbers the segments by that rendition's record `sequence`, so every edge and every reload agree. +An edge deriving one chooses the first video rendition by name with a timeline, or the first such audio rendition when there is none, and keeps a video choice while that rendition stays in the catalog; an audio choice yields to the first video rendition with a timeline once there is one. Segment boundaries come from its records, numbered by record `sequence`. Another video rendition snaps each boundary to its nearest record that starts a group on a keyframe; a segment with no such record nearby has no content on that rendition (a gap; HLS `EXT-X-GAP`). Other renditions take the frames whose timestamps fall inside each segment's span. A publisher wanting such an export SHOULD start video groups at the same timestamps across renditions. @@ -1074,6 +1074,7 @@ This document has no IANA actions. - One cutting rule for every track: a record ends at the first group boundary past a minimum (2 seconds RECOMMENDED, zero for sparse data such as a catalog) and splits a group between frames at a maximum (10 seconds RECOMMENDED), so a group that never closes is indexed as it grows and `durationMax` bounds every record. - Recording format version 2: each track stores record N at `segments/N`, beside its timeline's `segments/N`, with a `Frame Start` field in the segment object. Tracks commit and expire independently, and a DVR keeps each track's newest record. - Described deriving HLS and DASH at the edge from a reference rendition's records. +- An edge keeps its derived-format video reference rendition while that rendition stays in the catalog; an audio reference yields once video has a timeline. - Added an optional `epoch` to `.info`: the source route's Epoch, compared on resume so a restarted source fails enrollment. ## moq-hang-03 diff --git a/quest/m1/archive/README.md b/quest/m1/archive/README.md index 71651fe26b..df457f644b 100644 --- a/quest/m1/archive/README.md +++ b/quest/m1/archive/README.md @@ -168,7 +168,6 @@ owned by that prerequisite, not duplicated in archive storage. - [History from the start](/quest/m1/archive/replay-history.md) - history mode lists a recording from its first segment by reading stored timeline groups, not only the restated tail - [Replay catalog](/quest/m1/archive/replay-catalog.md) - `moq_archive::Reader` republishes the recorded catalog live with `store` set, so stock `moq export hls` serves the whole replay of `moq import archive` - [Idle flush](/quest/m1/archive/flush.md) - idle tracks are recorded within a bounded wall-clock delay, and `flush()` forces a track's pending record out -- [HLS media sequence](/quest/m1/archive/hls-media-sequence.md) - `EXT-X-MEDIA-SEQUENCE` never decreases when the reference rendition switches - [DVR rewind](/quest/m1/archive/dvr.md) - seek through a bounded archive and return to live playback ## Related diff --git a/quest/m1/archive/hls-media-sequence.md b/quest/m1/archive/hls-media-sequence.md deleted file mode 100644 index 836e9be983..0000000000 --- a/quest/m1/archive/hls-media-sequence.md +++ /dev/null @@ -1,37 +0,0 @@ -# [S] HLS media sequence survives a reference switch - -## Goal - -A media playlist's `EXT-X-MEDIA-SEQUENCE` never decreases, as -[RFC 8216 §6.2.2](https://www.rfc-editor.org/rfc/rfc8216.html#section-6.2.2) -requires, even when the reference rendition changes. Players keep loading the -same `media.m3u8` across the switch without stalling. - -## Plan - -Today `moq-hls` numbers segments by the reference rendition's own records. A -switch (a new first video rendition with a timeline, or the reference leaving -the catalog) starts a new numbering under a new URL tag -(`seg/{reference}.{segment}.m4s`, #4034). Its URLs never collide and a -discontinuity marks the switch, but the media sequence can rewind. - -Open design, pick one: - -- **Sticky reference.** Keep the reference while it stays in the catalog, so a - new, earlier-named rendition no longer switches it. Cheap and edge-consistent, - but a reference that leaves the catalog still rewinds. -- **Offset at the switch.** Number the new reference's segments from past the - last listed one. Monotonic, but an edge that starts after the switch numbers - differently, so edges disagree on the media sequence. - -Recommendation: the sticky reference first. It removes the common trigger -without giving up the property that every edge agrees on numbering. Measure -whether a reference leaving the catalog happens in practice before taking on -the offset. - -Test a switch through repeated playlist reloads: the media sequence never -decreases, and no URL names other content. - -## Related - -- [Fixed HLS target duration](/quest/m1/archive/hls-target.md) - also changes how the reference shapes the playlist diff --git a/rs/moq-hls/src/export/mod.rs b/rs/moq-hls/src/export/mod.rs index c0a07b11d0..52add46c37 100644 --- a/rs/moq-hls/src/export/mod.rs +++ b/rs/moq-hls/src/export/mod.rs @@ -8,10 +8,12 @@ //! way and is skipped. A `moq-archive` recording replayed onto a broadcast is served the same way: //! its playlists read only the stored timelines, and a segment GETs only its rendition's objects. //! -//! Segment boundaries come from one reference rendition's records (the first video rendition, or -//! the first audio one when there is no video), numbered by record sequence, so every edge and -//! every reload agree without the publisher cutting for HLS. Every other rendition resolves each -//! segment against its own timeline. +//! Segment boundaries come from one reference rendition's records, numbered by record sequence, +//! so edges that share the choice agree without the publisher cutting for HLS. The first choice +//! is the first video rendition, or the first audio one when there is no video. A video choice +//! sticks while that rendition stays in the catalog: a newer rendition that sorts earlier must +//! not rewind `EXT-X-MEDIA-SEQUENCE`. An audio choice switches to video once a video rendition +//! has a timeline. Every other rendition resolves each segment against its own timeline. //! An inline-parameter-set codec with no catalog `description` GETs one keyframe //! group on the first playlist render to build its init, then caches it. //! @@ -394,8 +396,8 @@ async fn watch_catalog( } renditions.sync(&upstream, &catalog); - // Follow the reference rendition's timeline; a new reference renumbers every segment. - let next = self::reference(&catalog); + // Keep the reference while it stays. Replacing it renumbers every segment. + let next = self::reference(&catalog, reference.as_ref()); if next != reference { if let Some(watcher) = timeline_watcher.lock().unwrap().take() { watcher.abort(); @@ -425,24 +427,39 @@ async fn watch_catalog( renditions.close(); } -/// The rendition every segment boundary comes from: the first video rendition with a timeline, -/// or the first audio one when the broadcast has no video. Renditions are ordered by name, so -/// every edge picks the same one. -fn reference(catalog: &moq_mux::catalog::hang::Catalog) -> Option { +/// The rendition every segment boundary comes from. +/// +/// The first choice is the first video rendition with a timeline, or the first audio one when +/// the broadcast has no video. Renditions are ordered by name, so edges choosing together pick +/// the same one. A video `current` stays while it is still a rendition with a timeline: a newer +/// rendition that sorts earlier must not take over, or `EXT-X-MEDIA-SEQUENCE` rewinds. An audio +/// `current` stays only while no video rendition has a timeline, since video segments cut on +/// audio records would mostly be gaps; that switch, like a reference that leaves, rewinds. +fn reference(catalog: &moq_mux::catalog::hang::Catalog, current: Option<&Reference>) -> Option { let archive = catalog.archive.as_ref()?; - let indexed = |name: &&String| archive.timelines.contains_key(*name); + let indexed = |name: &str| archive.timelines.contains_key(name); let video = catalog .video .renditions .keys() - .find(indexed) + .find(|name| indexed(name)) .map(|name| (Kind::Video, name)); + if let Some(current) = current { + let (kind, name) = current.as_ref(); + let listed = match kind { + Kind::Video => catalog.video.renditions.contains_key(name), + Kind::Audio => video.is_none() && catalog.audio.renditions.contains_key(name), + }; + if listed && indexed(name) { + return Some(Arc::clone(current)); + } + } let audio = || { catalog .audio .renditions .keys() - .find(indexed) + .find(|name| indexed(name)) .map(|name| (Kind::Audio, name)) }; video.or_else(audio).map(|(kind, name)| Arc::new((kind, name.clone()))) @@ -2936,7 +2953,7 @@ mod tests { let watcher = tokio::spawn(watch_timeline( upstream.broadcast.clone(), catalog.archive.clone().unwrap(), - reference(catalog).unwrap(), + reference(catalog, None).unwrap(), renditions.fanout(), )); (renditions, watcher) @@ -3102,6 +3119,181 @@ mod tests { ); } + // RFC 8216 §6.2.2: EXT-X-MEDIA-SEQUENCE must not decrease across reloads. A rendition that + // sorts ahead of the reference must not take over, or the playlist rewinds to that + // rendition's own record numbers and its URLs name other content. + #[tokio::test(start_paused = true)] + async fn an_earlier_rendition_does_not_rewind_the_media_sequence() { + const OLD: &[u8] = b"OLDOLDOLDOLDOLDO"; + const NEW: &[u8] = b"NEWNEWNEWNEWNEWN"; + + let origin = produce_origin(); + let mut broadcast = origin.create_broadcast("live").expect("publish allowed"); + broadcast.announce(Default::default()).expect("publish allowed"); + settle().await; + let catalog = moq_mux::catalog::Producer::new(&mut broadcast, moq_mux::catalog::Config::default()).unwrap(); + + let reserved = catalog.reserve(); + let mut video0 = catalog.test_video("video0").unwrap(); + video0.set(video_config()).unwrap(); + drop(reserved); + + let track = broadcast.create_track("video0", None).unwrap(); + let mut media = catalog + .media_producer( + track, + moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Data), + ) + .unwrap(); + // 2s GOPs against a 6s window, so the oldest records leave and the sequence is no longer 0. + for micros in (0..=10_000_000u64).step_by(2_000_000) { + media.write(payload_frame(micros, true, OLD)).unwrap(); + } + + let source = moq_mux::Source::new(origin.consume(), "live"); + let config = Config { + window: Duration::from_secs(6), + ..Config::default() + }; + let broadcaster = Broadcaster::new(source, config).await.unwrap(); + let sequence_of = |playlist: &str| -> u64 { + between(playlist, "#EXT-X-MEDIA-SEQUENCE:", "\n") + .parse() + .expect("media sequence") + }; + let rendition = tokio::time::timeout(Duration::from_secs(5), async { + loop { + if let Some(rendition) = broadcaster.rendition(Kind::Video, "video0") + && let Some(playlist) = rendition.media_playlist(None) + && sequence_of(&playlist) >= 2 + { + break rendition; + } + tokio::task::yield_now().await; + } + }) + .await + .expect("video0 playlist slid past its first segments"); + + let before = rendition.media_playlist(None).expect("playable"); + let sequence = sequence_of(&before); + assert!(sequence >= 2, "the window should have slid, got {sequence}\n{before}"); + let tag = segments::tag(&(Kind::Video, "video0".to_string())); + assert!(before.contains(&format!("seg/{tag}.{sequence}.m4s\n")), "{before}"); + let bytes = rendition + .listed_segment(sequence) + .await + .unwrap() + .expect("listed segment"); + assert!(contains(&bytes, OLD), "the listed segment is not video0's media"); + + // Enroll the earlier timeline before the rendition, so the catalog snapshot that adds + // the rendition already indexes it. The reference decision runs in that same update. + let earlier = broadcast.create_track("a", None).unwrap(); + let mut earlier_media = catalog + .media_producer( + earlier, + moq_mux::catalog::hang::Container::Legacy(moq_mux::container::Kind::Data), + ) + .unwrap(); + for micros in (0..=4_000_000u64).step_by(2_000_000) { + earlier_media.write(payload_frame(micros, true, NEW)).unwrap(); + } + let mut added = catalog.test_video("a").unwrap(); + added.set(video_config()).unwrap(); + + tokio::time::timeout(Duration::from_secs(5), async { + loop { + if broadcaster.rendition(Kind::Video, "a").is_some() { + break; + } + tokio::task::yield_now().await; + } + }) + .await + .expect("the earlier rendition is in the catalog"); + + let new_tag = segments::tag(&(Kind::Video, "a".to_string())); + let deadline = tokio::time::Instant::now() + Duration::from_secs(5); + let mut previous = sequence; + let mut reloads = 0; + while reloads < 5 && tokio::time::Instant::now() < deadline { + let Some(playlist) = rendition.media_playlist(None) else { + tokio::time::sleep(Duration::from_millis(10)).await; + continue; + }; + let next = sequence_of(&playlist); + assert!( + next >= previous, + "EXT-X-MEDIA-SEQUENCE decreased from {previous} to {next}\n{playlist}" + ); + assert!( + playlist.contains(&format!("seg/{tag}.")), + "segment URLs left the original reference\n{playlist}" + ); + assert!( + !playlist.contains(&format!("seg/{new_tag}.")), + "the earlier rendition's numbering replaced the reference\n{playlist}" + ); + previous = next; + reloads += 1; + tokio::time::sleep(Duration::from_millis(10)).await; + } + assert!(reloads >= 5, "the playlist did not stay reloadable"); + + let again = rendition + .listed_segment(sequence) + .await + .unwrap() + .expect("the original segment URL still resolves"); + assert_eq!(again, bytes, "the original URL names other content"); + assert!( + !contains(&again, NEW), + "the earlier rendition's media replaced the segment" + ); + + drop((media, earlier_media, video0, added, broadcast)); + } + + // A video reference sticks while it stays listed with a timeline, and falls back to the + // first choice once it leaves. An audio reference yields once video has a timeline. + #[test] + fn the_reference_sticks_until_it_leaves() { + let catalog = |videos: &[&str], audios: &[&str]| { + let mut catalog = moq_mux::catalog::hang::Catalog::default(); + for name in videos { + catalog.video.renditions.insert(name.to_string(), video_config()); + } + for name in audios { + catalog.audio.renditions.insert( + name.to_string(), + hang::catalog::AudioConfig::new(hang::catalog::AudioCodec::Opus, 48_000, 2), + ); + } + catalog.archive = Some(archive(&[videos, audios].concat())); + catalog + }; + let video = |name: &str| Arc::new((Kind::Video, name.to_string())); + let audio = |name: &str| Arc::new((Kind::Audio, name.to_string())); + + let first = reference(&catalog(&["b"], &[]), None); + assert_eq!(first, Some(video("b"))); + assert_eq!(reference(&catalog(&["a", "b"], &[]), first.as_ref()), Some(video("b"))); + assert_eq!(reference(&catalog(&["a", "c"], &[]), first.as_ref()), Some(video("a"))); + + let first = reference(&catalog(&[], &["b"]), None); + assert_eq!(first, Some(audio("b"))); + assert_eq!(reference(&catalog(&[], &["a", "b"]), first.as_ref()), Some(audio("b"))); + // Video listed before its timeline exists doesn't take over yet. + let mut pending = catalog(&[], &["b"]); + pending.video.renditions.insert("v".to_string(), video_config()); + assert_eq!(reference(&pending, first.as_ref()), Some(audio("b"))); + assert_eq!( + reference(&catalog(&["v"], &["a", "b"]), first.as_ref()), + Some(video("v")) + ); + } + // A new reference numbers segments from its own records, so its URLs carry its own tag and // never reuse one the old reference listed for other content. #[tokio::test(start_paused = true)]