diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 4c58d42b09..6bd4245b5a 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -58,18 +58,20 @@ moq --connect https://relay.example.com/anon --broadcast my-stream.hang export t moq --connect "https://relay.example.com/rooms/1?jwt=$TOKEN" --broadcast alice.hang import ts ``` -The `ts`, `fmp4`, and `flv` imports publish on the broadcast clock the catalog -advertises, not the input's own timestamps. The first frame is stamped when it -arrives, every track keeps its offset from the others, and an input that -restarts its timestamps, such as a restarted encoder, continues forward -after the real gap rather than rewinding. So a feed whose PTS starts hours in, -or whose first frame arrives late, still names the right wall time. +The `ts`, `fmp4`, and `flv` imports publish the input's own timestamps, and +the catalog clock maps the first frame to the time it arrived. So a feed whose +PTS starts hours in still names the right wall time, and every track keeps its +offset from the others. A group starting before the previous group's start, +such as a restarted encoder or a looping file wrapping to the top, ends the +import with an error; run it again to publish anew. A keyframe that merely +overlaps the previous group's last frame is not a rewind. MPEG-TS import carries H.264/H.265 and AAC/MP2/AC-3/E-AC-3, passes SCTE-35 and subtitle PIDs through as tracks, and round-trips the service tables. A `discontinuity_indicator` on the program's PCR PID is a system time-base reset, so it breaks every track's timeline and the exported clock declares the break in -turn. The same flag on an elementary PID other than the program PCR PID, a +turn. A new timeline whose first group starts before the previous group's start +ends the import like any other rewind. The same flag on an elementary PID other than the program PCR PID, a continuity-counter gap, and the 33-bit timestamp rollover move no clock and declare nothing. FLV covers H.264 + AAC. diff --git a/doc/bin/hls.md b/doc/bin/hls.md index d669f9be38..0b4b2b6658 100644 --- a/doc/bin/hls.md +++ b/doc/bin/hls.md @@ -48,7 +48,9 @@ numbers for different media. `--listen-tls-cert`/`--listen-tls-key` or `--listen-tls-generate` serve HTTPS, and `--cors-origin` opens it to browsers. H.264/H.265 and AAC/Opus renditions are served. Import handles classic HLS; -LL-HLS parts are not implemented yet. The library is +LL-HLS parts are not implemented yet. It publishes the playlist's own media +times, and the catalog clock maps the first imported segment to the time it +arrived, one mapping for every rendition. The library is [`moq-hls`](https://docs.rs/moq-hls). ## Recording segments diff --git a/doc/bin/rtmp.md b/doc/bin/rtmp.md index 6a144135d9..3dd8823fbb 100644 --- a/doc/bin/rtmp.md +++ b/doc/bin/rtmp.md @@ -29,6 +29,11 @@ the [`moq-rtmp`](https://docs.rs/moq-rtmp) library, which hands you each publish or play request to accept, map to a path, or reject. The CLI listener is unauthenticated; firewall it. +Each push is its own broadcast. Import publishes the encoder's own timestamps +and anchors the catalog clock on the first frame, so it names the wall time the +push arrived. A group starting before the previous group's start, such as an +encoder restarting its timestamps mid-push, ends that push with an error. + A player that advertises enhanced-RTMP multitrack receives every rendition. Any other player receives one video rendition: the largest picture (then highest bitrate) in a codec it advertised. A push carries the largest one. diff --git a/doc/bin/srt.md b/doc/bin/srt.md index bd8e852ab7..d923fa49b5 100644 --- a/doc/bin/srt.md +++ b/doc/bin/srt.md @@ -22,6 +22,9 @@ ffplay srt://localhost:9000 moq --connect https://relay.example.com/anon --broadcast event.hang import srt --connect 'srt://encoder.example.com:9000?streamid=live/cam' ``` +Import publishes the feed's own PTS and anchors the catalog clock on its first +frame, as [`import ts`](/bin/cli) does. + `--latency` sets the SRT receive buffer and doubles as the skip threshold on export. Export paces each SRT payload on the media clock, and re-anchors that pacing on a declared marker, so a restarted timeline plays out from the diff --git a/doc/lib/rs/moq-mux.md b/doc/lib/rs/moq-mux.md index eb1fd59e7b..a1f71ff50d 100644 --- a/doc/lib/rs/moq-mux.md +++ b/doc/lib/rs/moq-mux.md @@ -96,13 +96,20 @@ keep it in the payload. telemetry.append(moq_net::Timed::from(packet).at(received_at))?; ``` -The fMP4, MPEG-TS, and FLV importers publish the source's own timestamps unless -built with `live()`, which translates them onto the catalog's broadcast clock: -the first frame is live on arrival, every track of the input shares that one -mapping, and a source that restarts its timestamps continues forward after the -real idle gap. fMP4 passthrough rewrites each fragment's `tfdt` to match. Use -it for a live feed with its own zero; publish verbatim only when the catalog's -clock (`Config::with_clock`) already names the source's zero. +The fMP4, MPEG-TS, FLV, and MKV importers publish the source's own timestamps +(MPEG-TS after unwrapping its 33-bit PTS; fMP4 passthrough keeps each `tfdt`) +and anchor the catalog's broadcast clock instead: the first frame's timestamp +maps to the time it arrived, and every track of the input, like every importer +sharing the catalog, keeps that one mapping. A clock set with +`Config::with_clock` is never re-anchored, for a recording whose zero names its +real start. + +Group starts never go backwards. A group starting before the previous group's +start ends the import with `TimestampRewind`, as a restarted encoder or a looping +file wrapping to the top does, flagged MPEG-TS discontinuity or not; republish +it as a new broadcast. Frames may still dip below the previous group's content: +B-frames, and a keyframe overlapping the previous group's last frame. A flagged +MPEG-TS discontinuity that jumps forward continues the broadcast. ```bash cargo add moq-mux diff --git a/drafts/draft-lcurley-moq-hang.md b/drafts/draft-lcurley-moq-hang.md index 8384c75140..52d5060e68 100644 --- a/drafts/draft-lcurley-moq-hang.md +++ b/drafts/draft-lcurley-moq-hang.md @@ -576,7 +576,10 @@ A consumer MUST NOT submit the marker to a decoder. Empty groups (zero objects) are permitted and mean nothing. After a discontinuity the timeline continues forward. A publisher that stops producing and may resume on the same track (e.g. an encoder idle for lack of demand) SHOULD publish a discontinuity marker when it stops, so the group before the pause does not reach across the gap and read as live. -A group whose timestamps fall below the live edge earlier groups reached is malformed. +The first timestamp of a group MUST NOT be lower than the first timestamp of any group with a lower sequence, and no frame of a group may be lower than the first timestamp of the group before it; a group that breaks either is malformed. +A group MAY start at the same timestamp as the previous group. +Frames MAY present below the previous group's content (B-frames, or a keyframe overlapping the previous group's last frame). +A publisher whose source restarts below the previous group's start begins a new broadcast instead. A delivered sequence hole is a playhead event unless the boundary is contiguous within 1 ms. A consumer re-applies startup delay and skip at a playhead event; it does not reset codec state. @@ -1104,6 +1107,7 @@ A publisher MAY estimate an unknown final duration from the frame cadence, but M - A marker group of one empty frame declares a discontinuity. Empty groups mean nothing. Timestamps only move forward; a group below the live edge is malformed. A delivered sequence hole is a playhead event unless contiguous within 1 ms. - A publisher that stops producing and may resume on the same track SHOULD publish a discontinuity marker when it stops. - An audio endpoint bounds only the terminal packets that follow it in its own group. +- Replaced the live-edge floor with monotonic group starts: a group may overlap the previous group's content but not start before it. - Replaced the archive timeline `wall` field with a root `clock` section (`wall` plus `timescale`): one fixed broadcast mapping every track and the archive index convert into, independent of any archive. Zero timescales and walls past the JSON-safe integer range are refused. - Added optional `bitrate` and `jitter` fields to `json` and `binary` track entries. - Added the optional `delay` rendition field: how far a rendition's minimum flush lateness trails the broadcast's earliest rendition, never lowered once advertised and never subtracted across renditions. diff --git a/quest/m1/README.md b/quest/m1/README.md index 9292629513..f153fba03c 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -56,7 +56,6 @@ transport, benchmark tooling); worktrees isolate commits, not semantics. - [Full codec string](/quest/m1/publish-codec-string.md) - browser-published video carries the encoder's full RFC 6381 codec string, so native players decode it - [TS program selection](/quest/m1/ts-programs.md) - `import ts` refuses a multi-program stream unless `--program ` picks one or publishes each - [TS export jitter](/quest/m1/ts-export-jitter.md) - the video reorder bound follows later catalogs and the declared reorder depth, so a late B-frame never reorders TS output; an undeclared stream can reorder once per new maximum depth -- [TS import shared shift](/quest/m1/ts-import-shared-shift.md) - unflagged loop wraps move audio and video by one shift, so A/V sync holds across wraps - [ffmpeg muxdelay](/quest/m1/ts-publish-muxdelay.md) - the documented MPEG-TS publish line adds `-muxdelay 0`, so quiet audio is not clumped - [Same-hop importers](/quest/m1/hop-aligned-import.md) - importers sharing a `--hop` and fed one stream publish identical groups and timestamps, so failover survives - [PipeWire duplicate cameras](/quest/m1/pipewire-dup-cameras.md) - a webcam lists once with PipeWire enabled @@ -65,7 +64,6 @@ transport, benchmark tooling); worktrees isolate commits, not semantics. - [Auth client CA](/quest/m1/relay-auth-client-ca.md) - on dev, `auth::Config::validate` and `init` take the client-CA flag, so no caller can skip the check - [RTMP TLS only](/quest/m1/rtmp-tls-only.md) - an RTMP listener configured for TLS can refuse plaintext instead of sniffing and serving it - [HLS linger](/quest/m1/hls-linger.md) - `moq_hls::Server` serves an ended broadcast for its playlist window plus grace, so the moq.pro edge drops its own pool -- [Remove live()](/quest/m1/remove-live.md) - on dev, importers publish stream timestamps verbatim, the catalog clock maps them to wall time, and an encoder restart becomes a new epoch - [iroh versions](/quest/m1/iroh-lite-wip.md) - `iroh://` negotiates the configured versions, so `moq-lite-07-wip` can be opted into - [Go and Dart doc samples](/quest/m1/doc-samples-go-dart.md) - Go and Dart doc samples compile against their wrappers - [Data capture in bindings](/quest/m1/data-capture-bindings.md) - moq-ffi and every wrapper pass a data frame's capture time, and the JSON window producer takes one diff --git a/quest/m1/hop-aligned-import.md b/quest/m1/hop-aligned-import.md index 6f6f68b6c6..a322d4200e 100644 --- a/quest/m1/hop-aligned-import.md +++ b/quest/m1/hop-aligned-import.md @@ -21,8 +21,8 @@ encoded stream. Capture is out: two encoders never align. not a per-process counter. Decide how both processes agree across a timestamp wrap (TS PTS wraps every 26.5 h) when they started on opposite sides of it. -- Frame timestamps derive from the input alone too: any re-anchor shift must - come from the stream, not process start or wall clock. +- Frame timestamps derive from the input alone too. Importers publish the + stream's own timestamps and refuse a rewind, so nothing shifts them. - An importer announces only once it knows its tracks, so it never refuses a track the incumbent serves. - Docs (`doc/bin/cli.md` "Redundant publishers", the `--hop` doc comment) @@ -41,5 +41,4 @@ survives the standby joining and the incumbent stopping. ## Related - [Redundant ingest](/quest/m2/redundant-ingest.md) - splicing across first hops and two encoders, which this does not attempt -- [TS import shared shift](/quest/m1/ts-import-shared-shift.md) - the TS re-anchor shift that must stay input-derived - [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - a redundant pair shares one epoch diff --git a/quest/m1/legacy-end-overshoot.md b/quest/m1/legacy-end-overshoot.md index 77cff3dc58..5c83b136f8 100644 --- a/quest/m1/legacy-end-overshoot.md +++ b/quest/m1/legacy-end-overshoot.md @@ -4,7 +4,8 @@ A `@moq/watch` subscriber never aborts a browser-published track with "group timestamp is below the live edge" when the publisher's next group starts -inside the previous group's estimated end. +inside the previous group's estimated end: the JS consumer enforces +monotonic group starts, like Rust. ## Plan @@ -22,12 +23,20 @@ takes the live edge from the end marker, so that keyframe's group reads as below it and the track aborts. The consumer's covered-group skip tolerates the same overlap the live-edge check rejects. +Decided (maintainer, #4543): group starts are monotonic, and that is the only +hard rule. A group's keyframe may not start below the previous group's start, +and no frame may sit below the start of the group before its own. Frames, +keyframes included, may dip below the previous group's content (B-frames, or +a keyframe overlapping its last frame), so an estimated end is never a hard +edge. A group may start at the same timestamp as the previous one; strictly +increasing starts are not enforced. Group IDs never move backwards. A group +starting before the previous group's start is a restart, a new broadcast. The Rust `moq-mux` producer and +consumer enforce this since #4543. + - Reproduce with a mocked clock: cut a group, then resume with a keyframe between the last frame and the estimated end. -- Decide which side is wrong: the producer's estimate reaching past what it - will refuse, or the consumer treating an estimated end as a hard edge. - Fix it there and keep the other side's rule consistent with it. -- Check the Rust `moq-mux` producer for the same estimate. +- Align `js/hang/src/container/consumer.ts` (and the JS producer, if it + differs) to "group starts monotonic", matching `rs/moq-mux/src/container`. ## Related diff --git a/quest/m1/remove-live.md b/quest/m1/remove-live.md deleted file mode 100644 index 99c040ec2c..0000000000 --- a/quest/m1/remove-live.md +++ /dev/null @@ -1,50 +0,0 @@ -# [M] Importers publish stream timestamps; the catalog clock maps them to wall - -## Goal - -The fMP4, MPEG-TS, and FLV importers in `rs/moq-mux` have no `live()`: every -importer publishes the stream's own timestamps verbatim (after PTS unwrap), -and the catalog's root `clock` is what maps them to wall time. `moq import` -(`rs/moq-cli/src/publish.rs`) stops calling it. An encoder that restarts its -timestamps ends the broadcast with an error instead of being re-anchored -forward onto the old one; turning the republish into a new epoch is the -broadcast epoch line's outcome, not this quest's. The SRT, RTMP, and HLS gateways, which reuse these -importers, get the same behavior. - -## Plan - -Decided (2026-09-28, replacing the gateway live-clock quest, which planned -the opposite: every gateway opting into `live()`): - -- Timestamps stay verbatim from the stream. Rewriting them onto an - arrival-time anchor breaks same-hop importers, which must derive every - timestamp from the input alone (the hop-aligned import quest in - [#4388](https://github.com/moq-dev/moq/pull/4388)), and hides the source's - own timeline from anything downstream. -- The catalog clock carries the mapping. An importer establishes the root - `clock` so the stream's PTS converts to wall time (the first frame is live - on arrival), rather than translating each timestamp. Open: whether the - importer sets the catalog clock from its first frame (the mapping is fixed - at construction today, `moq_mux::Clock` and `catalog::Config::with_clock`) - or the caller builds the catalog once the first PTS is known. Every track of - one input, and every rendition of one HLS import, shares that one mapping. -- An encoder restart (a PTS rewind or a signalled time-base discontinuity) is - a new epoch, not a forward re-anchor: the importer ends the broadcast with - an error, and the caller republishes, which the broadcast epoch line turns - into a fresh `@`. Until that line lands, a restart fails loud. -- Delete `live()` from `ts::Import`, `fmp4::Import`, and `flv::Import`, the - crate-private `clock::Anchor`, and `SourceMap` if nothing else uses it. - fMP4 passthrough stops rewriting `tfdt`. A published `moq-mux` API break, - so this targets `dev`. - -Tests: per importer, a source starting at a large PTS publishes that PTS and -a catalog clock that maps it to near the arrival time; a rewind ends the -broadcast with an error. Update `doc/lib/rs/moq-mux.md` (the `live()` -paragraph), `doc/bin/cli.md`, and the gateway pages under `doc/bin/`, and -replace `ts_import_publishes_on_the_broadcast_clock` in moq-cli. - -## Related - -- [Broadcast epochs](/quest/m1/broadcast-epoch/README.md) - the new epoch an encoder restart becomes -- [Catalog wall clock](/quest/m1/catalog-wall-clock.md) - the PTS-to-wall conversion this relies on, at full precision -- [TS import shared shift](/quest/m1/ts-import-shared-shift.md) - the TS re-anchor shift that must stay input-derived diff --git a/quest/m1/ts-import-shared-shift.md b/quest/m1/ts-import-shared-shift.md deleted file mode 100644 index e5c9190bd3..0000000000 --- a/quest/m1/ts-import-shared-shift.md +++ /dev/null @@ -1,51 +0,0 @@ -# [S] moq import ts: one re-anchor shift per program - -## Goal - -An unflagged loop wrap in `moq import ts` moves audio and video forward by the -same amount, so A/V sync holds across any number of wraps. Today each -elementary stream owns a `Reanchor` and grows its shift to reach its own live -edge ([#3997](https://github.com/moq-dev/moq/pull/3997)). Audio and video -edges sit on their own last frame starts, so each wrap drifts A/V by the -difference in frame durations: about 12 ms at 30 fps with 48 kHz AAC, or -1.7 s a day on a 10-minute loop. - -## Plan - -Decided: a program-level shared shift in `rs/moq-mux/src/container/ts/import.rs`. -Every stream of a program applies the same shift, grown once per wrap by the -largest amount any stream needs to clear its edge, which preserves the source's -inter-stream offsets. A timebase break (PCR discontinuity) clears it for the -whole program, as it already does per stream. - -Guidance: - -- The shift must be known before any stream emits a frame of the new - generation (Codex on #4307): a first stream that grows only enough for - itself leaves a later-arriving stream below its edge, and growing again - then drifts the two. Decided: hold each stream's post-wrap frames until - every live stream of the program has shown its new PTS, then take the - maximum growth once. -- The hold needs its own bound: #3489 adds per-PID counters, not a timeout, - and the catalog `stalled` bit (`Stream::tick`, #3630) covers video only and - runs on the catalog's timer. Decided: bound the hold on the program clock - (PCR advance since the first stream's new generation, not wall time), and - commit the shift over the streams seen so far when it expires. A stream - that returns later applies the committed shift, clamped to its edge as - below. The `Anchor`/`Lane` split behind `live()` in `moq_mux::clock` solves - a similar problem for restarts, but the remove-live quest deletes it, so - copy what helps rather than depending on it. -- A stream whose own edge is still above the shifted timestamp after the - shared growth (its tail ran longer) is the case that forces growing by the - maximum. Landing on its edge is accepted today; keep that trade-off. -- Sections already take the video shift, so cues keep following pictures. - MPEG-2 video (`Stream::Clock`) has no track and so no edge, but it should - read the shared shift too. -- Tests beside the existing loop-wrap tests: a muxed H.264 + AAC loop whose - period is not a multiple of either frame duration keeps the first audio and - video timestamps of each pass at the source offset across three wraps. - -## Related - -- [#3489](/quest/m1/3489-ts-import-stream-liveness.md) - per-PID liveness in the same importer; touches `Stream` but not the shift -- [Remove live()](/quest/m1/remove-live.md) - deletes the restart anchor; a wrap shift stays input-derived diff --git a/quest/m1/ts-programs.md b/quest/m1/ts-programs.md index 14e17a964c..2baad24005 100644 --- a/quest/m1/ts-programs.md +++ b/quest/m1/ts-programs.md @@ -31,7 +31,3 @@ clock; `--program all` publishes both broadcasts, each timed correctly. Update ## Closes - [#4353](https://github.com/moq-dev/moq/issues/4353) - close this issue when the quest finishes - -## Related - -- [TS import shared shift](/quest/m1/ts-import-shared-shift.md) - its per-program shift assumes the single program this settles diff --git a/rs/moq-cli/src/publish.rs b/rs/moq-cli/src/publish.rs index 1dcd5607a7..b459443027 100644 --- a/rs/moq-cli/src/publish.rs +++ b/rs/moq-cli/src/publish.rs @@ -276,8 +276,8 @@ impl Publish { /// the catalog tracks, so announcing after it lands the advertisement with /// the tracks already in place. /// - /// Stdin is a live feed with its own zero, so the container importers translate its - /// timestamps onto the broadcast clock the catalog advertises (`live`). + /// Stdin is a live feed with its own zero: the container importers publish its timestamps + /// verbatim and anchor the catalog clock so the first frame is live on arrival. pub fn new( mut broadcast: moq_net::broadcast::Producer, format: &PublishFormat, @@ -290,7 +290,7 @@ impl Publish { if let PublishFormat::Ts = format { let config = config.with_catalog(moq_mux::catalog::hang::Catalog::::default()); let catalog = moq_mux::catalog::Producer::new(&mut broadcast, config)?; - let ts = ts::Import::new(broadcast.clone(), catalog.reserve()).live(); + let ts = ts::Import::new(broadcast.clone(), catalog.reserve()); return Ok(Self { source: Source::Stream { decoder: PublishDecoder::Ts(Box::new(ts)), @@ -312,12 +312,12 @@ impl Publish { } } PublishFormat::Fmp4 => { - let fmp4 = fmp4::Import::new(broadcast.clone(), catalog.reserve()).live(); + let fmp4 = fmp4::Import::new(broadcast.clone(), catalog.reserve()); PublishDecoder::Fmp4(Box::new(fmp4)) } PublishFormat::Ts => unreachable!("TS is handled above with the mpegts catalog extension"), PublishFormat::Flv => { - let flv = flv::Import::new(broadcast.clone(), catalog.reserve()).live(); + let flv = flv::Import::new(broadcast.clone(), catalog.reserve()); PublishDecoder::Flv(Box::new(flv)) } }; @@ -806,23 +806,20 @@ mod tests { ); } - /// `moq import ts` publishes on the broadcast clock it advertises: a feed arriving a minute - /// after the broadcast began is live on arrival, not stamped with its own PTS (1.4s into bbb). + /// `moq import ts` publishes the feed's own PTS (1.4s into bbb) and anchors the catalog + /// clock on it, so the first frame is live on arrival. #[tokio::test] - async fn ts_import_publishes_on_the_broadcast_clock() { - let ago = Duration::from_secs(60); - let clock = moq_mux::Clock::at(std::time::Instant::now() - ago, std::time::SystemTime::now() - ago).unwrap(); + async fn ts_import_anchors_the_catalog_clock_on_arrival() { let broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); - let config = moq_mux::catalog::Config::default().with_clock(clock); - let mut publish = Publish::new(broadcast, &PublishFormat::Ts, config).unwrap(); + let mut publish = Publish::new(broadcast, &PublishFormat::Ts, Default::default()).unwrap(); #[allow(irrefutable_let_patterns)] let Source::Stream { decoder, .. } = &mut publish.source else { panic!("expected a stream source"); }; - let before = clock.now(); + let before = std::time::SystemTime::now(); decoder.decode_chunk(BBB).unwrap(); - let after = clock.now(); + let after = std::time::SystemTime::now(); decoder.finish().unwrap(); let catalog = hang::catalog::Catalog::<()>::subscribe(&consumer) @@ -832,11 +829,7 @@ mod tests { .await .unwrap() .expect("a catalog"); - assert_eq!( - catalog.clock, - Some(clock.wall()), - "the advertised clock is the one stamped on" - ); + let clock = catalog.clock.expect("the catalog advertises a clock"); let (name, config) = catalog.video.renditions.iter().next().expect("a video rendition"); let track = consumer.track(name).unwrap().subscribe(None).await.unwrap(); let container = moq_mux::catalog::hang::Container::try_from(config).unwrap(); @@ -846,11 +839,16 @@ mod tests { .unwrap() .expect("a video frame") .timestamp; + assert!( + first.as_micros() < Duration::from_secs(5).as_micros(), + "the feed's own PTS: {first:?}" + ); // The PES that anchors the mapping need not be this frame: the mux spaces them apart. - let skew = Duration::from_secs(2).as_micros(); + let skew = Duration::from_secs(2); + let wall = clock.wall_clock(first).unwrap(); assert!( - before.as_micros() - skew <= first.as_micros() && first.as_micros() <= after.as_micros() + skew, - "the first frame is live on arrival: {first:?} not in {before:?}..={after:?}" + before - skew <= wall && wall <= after + skew, + "the first frame is live on arrival" ); } diff --git a/rs/moq-hls/src/import.rs b/rs/moq-hls/src/import.rs index 165872e63d..ea0753f998 100644 --- a/rs/moq-hls/src/import.rs +++ b/rs/moq-hls/src/import.rs @@ -556,6 +556,13 @@ impl TrackState { // `consume_segments` resolves the effective map before every segment, so a missing // importer means the playlist never carried an `EXT-X-MAP`. let importer = self.importer.as_mut().ok_or(Error::MissingMap)?; + // Only a discontinuity starts a new media timeline; a sequence skip must keep advancing. + if self + .next_discontinuity + .is_some_and(|previous| previous != discontinuity_sequence) + { + importer.discontinuity(); + } if reanchored { importer.seek(group_sequence)?; } diff --git a/rs/moq-mux/src/catalog/producer.rs b/rs/moq-mux/src/catalog/producer.rs index 4f72d6376c..a2b902f038 100644 --- a/rs/moq-mux/src/catalog/producer.rs +++ b/rs/moq-mux/src/catalog/producer.rs @@ -27,6 +27,15 @@ struct State { /// Why the catalog tracks are closed, so [`Producer::modify`] refuses further edits with the /// cause: [`Producer::finish`], or a drop-time publish that failed and aborted them. closed: Option, + + /// The broadcast's shared clock, advertised at the catalog root. Every importer on this + /// catalog reads it (the same timeline), so timestamps they synthesize when a caller has none + /// land on one timeline and audio/video stay in sync. + clock: crate::Clock, + + /// Whether the clock's mapping is settled: set by [`Config::with_clock`] or placed by an + /// importer's first timestamp. Until then [`Producer::anchor`] may still place it. + anchored: bool, } /// Take the shared state, ignoring a poisoned lock. @@ -162,14 +171,6 @@ pub struct Producer { current: Arc>>, - /// Shared wall clock for the broadcast's tracks. Every importer on this catalog - /// gets a copy (the same epoch), so timestamps they synthesize when - /// a caller has none land on one timeline and audio/video stay in sync. - /// - /// It also owns the broadcast's wall mapping, published at the catalog root as - /// `clock: { wall, timescale }` independently of any archive timeline. - clock: crate::Clock, - /// The broadcast's timeline: the shared boundary list every enrolled track's groups map /// onto, and the track those segment records are published on. See /// [`timeline`](Self::timeline). @@ -193,7 +194,6 @@ impl Clone for Producer { Self { outputs: self.outputs.clone(), current: self.current.clone(), - clock: self.clock, timeline: self.timeline.clone(), max_age: self.max_age, bandwidth: self.bandwidth.clone(), @@ -214,7 +214,7 @@ pub struct Config { catalog: Catalog, max_age: Option, bandwidth: moq_net::bandwidth::Allocator, - clock: crate::Clock, + clock: Option, timeline: crate::timeline::Config, } @@ -224,7 +224,7 @@ impl Default for Config<()> { catalog: Catalog::default(), max_age: None, bandwidth: moq_net::bandwidth::Allocator::unlimited(), - clock: crate::Clock::new(), + clock: None, timeline: crate::timeline::Config::default(), } } @@ -246,11 +246,13 @@ impl Config { /// Publish with this broadcast clock instead of a fresh one. /// - /// The clock's wall mapping is advertised at the catalog root, so pass a clock whose PTS - /// zero names the content's real start when importing a recording; live publishers use the - /// default. The mapping is fixed for the broadcast and never overwritten. + /// The clock's wall mapping is advertised at the catalog root and fixed for the broadcast. + /// Without this, the catalog starts a fresh clock, and a container importer (fMP4, MPEG-TS, + /// FLV, MKV) re-anchors it on its first timestamp, before writing any frame, so the stream's own + /// timestamps map to the arrival time. Pass one whose PTS zero names the content's real start + /// when importing a recording. pub fn with_clock(mut self, clock: crate::Clock) -> Self { - self.clock = clock; + self.clock = Some(clock); self } @@ -321,8 +323,10 @@ impl Producer { // The broadcast clock is advertised at the catalog root from the first snapshot, // independently of any archive timeline: a live-only publisher exposes its mapping // without creating a segment index. + let anchored = config.clock.is_some(); + let clock = config.clock.unwrap_or_default(); let mut catalog = config.catalog; - catalog.clock = Some(config.clock.wall()); + catalog.clock = Some(clock.wall()); // The contents are `Send + Sync` natively; on wasm moq-net's handles are // `Rc`-backed, so clippy sees a pointlessly atomic `Arc`. Keeping one type for @@ -341,8 +345,9 @@ impl Producer { owned: BTreeSet::new(), reservations: Reservations::default(), closed: None, + clock, + anchored, })), - clock: config.clock, timeline, max_age: config.max_age, bandwidth: config.bandwidth, @@ -372,7 +377,7 @@ impl Producer { pub fn timestamp(&self, hint: Option) -> crate::Result { match hint { Some(pts) => Ok(pts), - None => Ok(self.clock.now()), + None => Ok(self.clock().now()), } } @@ -421,14 +426,36 @@ impl Producer { take(&self.current).catalog.clone() } - /// The broadcast's shared clock: the epoch importers stamp against and the wall mapping + /// The broadcast's shared clock: the timeline producers stamp against and the wall mapping /// advertised at the catalog root. /// - /// Copies share the epoch, so handing them to concurrent producers keeps every track on - /// one timeline. Translate a source with its own zero through - /// [`Clock::source`](crate::Clock::source). + /// Copies share the timeline, so handing them to concurrent producers keeps every track on + /// one clock. A container importer may still re-anchor it on its first timestamp (see + /// [`Config::with_clock`]), so read it after that when mixing the two. pub fn clock(&self) -> crate::Clock { - self.clock + take(&self.current).clock + } + + /// Map a stream's first timestamp to now, unless the clock is already anchored. + /// + /// Container importers publish their stream's timestamps verbatim and call this before + /// writing each frame: the first call places the wall mapping so `pts` is live on arrival, + /// and every later call, from any track or importer sharing this catalog, is a no-op. + pub(crate) fn anchor(&mut self, pts: moq_net::Timestamp) -> crate::Result<()> { + if take(&self.current).anchored { + return Ok(()); + } + let mut guard = self.modify()?; + // Another clone may have anchored between the check and the lock. + if guard.state.anchored { + return Ok(()); + } + let since = std::time::Duration::from_micros(u64::try_from(pts.as_micros()).unwrap_or(u64::MAX)); + let clock = crate::Clock::arrival(since)?; + guard.state.clock = clock; + guard.state.anchored = true; + guard.clock = Some(clock.wall()); + guard.commit() } /// Begin reserving the initial track set, returning a clonable [`Reserved`](super::Reserved). @@ -1276,10 +1303,41 @@ mod test { let catalog = Producer::new(&mut broadcast, config).unwrap(); // A recording import advertises the content's start, not the construction instant. - assert_eq!( - catalog.snapshot().clock.map(|clock| clock.wall), - Some(moq_net::Timestamp::from_micros(60_000_000).unwrap()) + let advertised = Some(moq_net::Timestamp::from_micros(60_000_000).unwrap()); + assert_eq!(catalog.snapshot().clock.map(|clock| clock.wall), advertised); + + // An importer's first timestamp never re-anchors an explicit clock. + let mut catalog = catalog; + catalog.anchor(moq_net::Timestamp::from_secs(3600).unwrap()).unwrap(); + assert_eq!(catalog.snapshot().clock.map(|clock| clock.wall), advertised); + } + + /// The first anchor maps its timestamp to now for every clone; later ones change nothing. + #[test] + fn anchor_places_the_default_clock_once() { + use std::time::{Duration, SystemTime}; + + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let mut catalog = Producer::new(&mut broadcast, Config::default()).unwrap(); + let mut clone = catalog.clone(); + + let first = moq_net::Timestamp::from_secs(3600).unwrap(); + catalog.anchor(first).unwrap(); + let clock = clone.clock(); + assert_eq!(catalog.snapshot().clock, Some(clock.wall()), "the anchor is advertised"); + assert!( + clock.now().as_micros() >= first.as_micros(), + "the clock reads on from the anchor" ); + let drift = clock + .wall_clock(first) + .unwrap() + .duration_since(SystemTime::now()) + .unwrap_or_else(|err| err.duration()); + assert!(drift < Duration::from_secs(1), "the first timestamp is live on arrival"); + + clone.anchor(moq_net::Timestamp::ZERO).unwrap(); + assert_eq!(catalog.clock().wall(), clock.wall(), "a second anchor is a no-op"); } fn h264_config() -> VideoConfig { diff --git a/rs/moq-mux/src/clock.rs b/rs/moq-mux/src/clock.rs index ee2e0694b8..c4eead8d02 100644 --- a/rs/moq-mux/src/clock.rs +++ b/rs/moq-mux/src/clock.rs @@ -1,20 +1,18 @@ -//! The broadcast's shared clock: one monotonic epoch plus its fixed wall mapping. +//! The broadcast's shared clock: one monotonic timeline plus its wall mapping. //! -//! Create one [`Clock`] per broadcast and hand copies to every producer: because they share an -//! epoch, frames captured at the same instant get the same timestamp, keeping concurrently +//! Create one [`Clock`] per broadcast and hand copies to every producer: because they share a +//! timeline, frames captured at the same instant get the same timestamp, keeping concurrently //! produced tracks (e.g. audio and video capture on separate threads) in sync. It is `Copy`, so //! handing it out is cheap. //! //! The clock also owns the broadcast's wall mapping, advertised at the catalog root as //! `clock: { wall, timescale }`: `wall` is the wall-clock time of PTS zero in //! [`Clock::TIMESCALE`] units since the moq epoch (2020-01-01). A consumer derives any -//! timestamp's wall time as `wall + pts` after converting it into this timescale. The mapping -//! is fixed at construction and never overwritten: a discontinuity marker is a delivery event, -//! not a new epoch, and a system-clock adjustment never retimes it. +//! timestamp's wall time as `wall + pts` after converting it into this timescale. A discontinuity +//! marker is a delivery event, not a new epoch, and a system-clock adjustment never retimes it. //! -//! A source with its own zero (a file, a restarted encoder) is translated onto the mapping with -//! [`SourceMap`]: the first frame anchors onto the live edge, and a reset re-anchors forward, -//! preserving the real idle gap measured on this monotonic clock. +//! A container importer publishes its stream's own timestamps, so it places the mapping instead: +//! its first timestamp is live on arrival (see [`Config::with_clock`](crate::catalog::Config::with_clock)). use std::time::{Duration, Instant, SystemTime}; @@ -42,13 +40,13 @@ fn wall_clock(wall: SystemTime) -> crate::Result { /// concurrently, e.g. an audio and a video capture running on separate /// threads, land on a single timeline. /// -/// Copies share the epoch and the fixed wall mapping, so handing them to several producers keeps -/// the broadcast on one clock. The wall mapping is established at construction: sampling it when -/// a delayed first frame arrives would pretend that frame is timestamp zero, so the epoch is -/// pinned up front and that frame's PTS converts into the mapping instead. +/// Copies share the timeline and the wall mapping, so handing them to several producers keeps +/// the broadcast on one clock. #[derive(Clone, Copy, Debug)] pub struct Clock { - epoch: Instant, + /// A monotonic instant, and what the clock read then in micros. + instant: Instant, + reading: u64, wall: hang::catalog::Clock, } @@ -59,57 +57,69 @@ impl Clock { /// enough that the wall value stays within the JSON-safe integer range until the year 2255. pub const TIMESCALE: moq_net::Timescale = moq_net::Timescale::MICRO; - /// How far before construction a fresh clock puts PTS zero. + /// What a fresh clock reads at construction. /// - /// A translated source anchors its first frame at the current instant, but the frames - /// muxed beside it can carry earlier timestamps: a B-frame presenting before the keyframe - /// decoded ahead of it, or audio leading video in the mux. Starting the clock this far back - /// leaves them room instead of landing before the broadcast began. + /// Frames stamped beside it can carry slightly earlier timestamps (audio captured a moment + /// before the video it is muxed with). Starting the clock this far past zero leaves them room + /// instead of landing before the broadcast began. const LEAD: Duration = Duration::from_secs(10); - /// Start a clock at the current instant, with PTS zero ten seconds earlier on both the - /// monotonic and the wall clock, so earlier-stamped frames of a source anchored now still map. + /// Start a clock that reads ten seconds now, so PTS zero is ten seconds ago on the wall. pub fn new() -> Self { - let (now, wall) = (Instant::now(), SystemTime::now()); - // Shortly after boot the monotonic clock may not reach back that far; start at now then. - let (epoch, wall) = match (now.checked_sub(Self::LEAD), wall.checked_sub(Self::LEAD)) { - (Some(epoch), Some(wall)) => (epoch, wall), - _ => (now, wall), - }; - Self::at(epoch, wall).expect("the current wall time is representable as a broadcast clock") + Self::arrival(Self::LEAD).expect("the current wall time is representable as a broadcast clock") } - /// Start a clock at an explicit monotonic epoch and wall time. + /// Start a clock at an explicit monotonic epoch and wall time: PTS zero at both. /// /// The deterministic constructor: synthetic sources and fixtures pin both ends instead of /// sampling. Refuses an unrepresentable wall. pub fn at(epoch: Instant, wall: SystemTime) -> crate::Result { Ok(Self { - epoch, + instant: epoch, + reading: 0, wall: wall_clock(wall)?, }) } - /// The current timestamp since the clock's epoch. + /// A clock that reads `since` now: a source whose first timestamp is `since` is live on arrival. + /// + /// Refuses a `since` so large that PTS zero lands before the moq epoch (2020), which the wall + /// mapping cannot name. + pub(crate) fn arrival(since: Duration) -> crate::Result { + let (instant, now) = (Instant::now(), SystemTime::now()); + let unmappable = || crate::Error::UnmappableTimestamp(format!("{since:?} puts PTS zero before 2020")); + let zero = now.checked_sub(since).ok_or_else(unmappable)?; + Ok(Self { + instant, + reading: u64::try_from(since.as_micros()).map_err(|_| unmappable())?, + wall: wall_clock(zero).map_err(|_| unmappable())?, + }) + } + + /// The current timestamp on this clock. pub fn now(&self) -> moq_net::Timestamp { // u128 -> u64 truncation is unreachable: u64 microseconds is ~584,000 years. - moq_net::Timestamp::from_micros(self.epoch.elapsed().as_micros() as u64) + let elapsed = self.instant.elapsed().as_micros() as u64; + moq_net::Timestamp::from_micros(self.reading + elapsed) .expect("an instant elapsed duration fits in a timestamp") } /// Map the instant a payload was captured (a datagram's arrival, a sensor read) onto this clock. /// /// Refuses an instant ahead of now, which would claim the payload reached the transport before - /// it existed, and one before the clock's epoch, which no timestamp can name. + /// it existed, and one before PTS zero, which no timestamp can name. pub(crate) fn capture(&self, at: Instant) -> crate::Result { if at > Instant::now() { return Err(crate::Error::InvalidCapture); } - let elapsed = at - .checked_duration_since(self.epoch) - .ok_or(crate::Error::InvalidCapture)?; - Ok(moq_net::Timestamp::from_micros(elapsed.as_micros() as u64) - .expect("an instant elapsed duration fits in a timestamp")) + let micros = match at.checked_duration_since(self.instant) { + Some(after) => self.reading + after.as_micros() as u64, + None => self + .reading + .checked_sub(self.instant.duration_since(at).as_micros() as u64) + .ok_or(crate::Error::InvalidCapture)?, + }; + Ok(moq_net::Timestamp::from_micros(micros).expect("an instant elapsed duration fits in a timestamp")) } /// Map a payload's capture instant onto this clock, stamping an untimed payload now, so timed @@ -139,20 +149,13 @@ impl Clock { self.wall } - /// The wall-clock time of `pts` under this broadcast's fixed mapping. + /// The wall-clock time of `pts` under this broadcast's mapping. /// - /// Pure in the stored epoch: a system-clock adjustment after construction changes nothing. + /// Pure in the stored mapping: a system-clock adjustment after construction changes nothing. /// Refuses an unrepresentable result rather than truncating it. pub fn wall_clock(&self, pts: moq_net::Timestamp) -> crate::Result { self.wall.wall_clock(pts).map_err(crate::Error::from) } - - /// Translate a source with its own zero onto this broadcast's mapping. - /// - /// Each adapter owns one per source; see [`SourceMap`]. The mapping itself is untouched. - pub fn source(&self) -> SourceMap { - SourceMap::new(*self) - } } impl Default for Clock { @@ -161,235 +164,6 @@ impl Default for Clock { } } -/// Translates one source's timestamps onto the broadcast clock. -/// -/// A source numbers from its own zero (a file starts at its first PTS, an encoder restarts at -/// zero), while the broadcast numbers from the shared epoch. The first frame anchors onto the -/// live edge, preserving the source's spacing from there on; a reset re-anchors forward, -/// preserving the real idle gap measured on the broadcast's monotonic clock. Backwards steps -/// within [`MAX_REORDER`](Self::MAX_REORDER) keep their offset, so permitted B-frame reordering -/// inside a group survives verbatim. Translated timestamps keep the source's timescale. -/// -/// The broadcast wall mapping is never touched: translating a reset is not a new epoch, and a -/// discontinuity marker the adapter emits alongside is a delivery event the playhead reacts to, -/// not a clock the catalog republishes. Retained records keep their timestamps. -/// -/// Each publisher adapter owns one per source and wires its own restart detection to -/// [`reset`](Self::reset); the automatic path only separates reordering from resets by size. -pub struct SourceMap { - anchor: Anchor, - lane: Lane, -} - -impl SourceMap { - /// The largest backwards source step still treated as permitted in-group reordering - /// (B-frames present out of decode order) rather than a source reset. - /// - /// A larger backwards step re-anchors the source forward instead. Adapters that detect a - /// restart out of band re-anchor explicitly with [`reset`](Self::reset); this bound only - /// separates the two for the automatic path. - pub const MAX_REORDER: Duration = Duration::from_millis(500); - - /// A translator onto `clock`, unanchored until the first frame. - pub fn new(clock: Clock) -> Self { - Self { - anchor: Anchor::new(clock), - lane: Lane::default(), - } - } - - /// The broadcast clock this source translates onto. - pub fn clock(&self) -> Clock { - self.anchor.clock - } - - /// Translate `pts` onto the broadcast clock, sampling the arrival time. - pub fn translate(&mut self, pts: moq_net::Timestamp) -> crate::Result { - self.anchor.translate(&mut self.lane, pts) - } - - /// Translate `pts` onto the broadcast clock, arriving at monotonic `now` micros. - /// - /// The deterministic core behind [`translate`](Self::translate): synthetic sources pin the - /// arrival instants instead of sampling them. - pub fn translate_at(&mut self, pts: moq_net::Timestamp, now: u64) -> crate::Result { - self.anchor.translate_at(&mut self.lane, pts, now) - } - - /// Re-anchor after an explicitly detected source restart, preserving the idle gap. - /// - /// The adapter path for a restart it observes out of band (an encoder reload, a file loop): - /// the next frame continues after everything published so far plus the downtime since the - /// previous frame, instead of rewinding the broadcast. - pub fn reset(&mut self, pts: moq_net::Timestamp) -> crate::Result { - self.lane.restart(); - self.translate(pts) - } - - /// [`reset`](Self::reset) with an explicit arrival instant, for synthetic sources. - pub fn reset_at(&mut self, pts: moq_net::Timestamp, now: u64) -> crate::Result { - self.lane.restart(); - self.translate_at(pts, now) - } -} - -/// One source's mapping onto the broadcast clock, shared by every track the source muxes. -/// -/// Tracks of one source must share an offset or they drift apart by however far their first -/// frames' PTS differ. They can't share a single [`SourceMap`] either: interleaved audio and -/// video step back further than [`SourceMap::MAX_REORDER`], which would read as a reset. So -/// each track keeps its own [`Lane`] that detects its own backwards steps, and a restart any -/// lane detects moves the anchor once; the other lanes adopt that mapping when they restart too. -pub(crate) struct Anchor { - clock: Clock, - /// Broadcast micros minus source micros for the current generation; `None` until the first - /// frame anchors it. - offset: Option, - /// Bumped at each re-anchor, so a lane knows whether its restart was already applied. - generation: u64, - /// The latest broadcast micros published so far, by any lane: the idle gap's origin. - last_broadcast: Option, - /// Where the latest frames end, by any lane, so a restart never lands on one of them. - last_end: Option, - /// `clock.now()` when the last frame was translated: the idle gap's start. - last_arrival: Option, -} - -/// One track's position on its source's [`Anchor`]. -#[derive(Default)] -pub(crate) struct Lane { - /// The offset this lane translates with, in micros; `None` until it adopts one. - offset: Option, - generation: u64, - last_source: Option, - /// The shortest forward step this lane's source took: its frame duration, near enough. - step: Option, - /// The adapter observed a restart on this lane out of band. - restart: bool, -} - -impl Lane { - /// The next frame starts a new source timeline, however its PTS compares to the last. - pub(crate) fn restart(&mut self) { - self.restart = true; - } -} - -impl Anchor { - pub(crate) fn new(clock: Clock) -> Self { - Self { - clock, - offset: None, - generation: 0, - last_broadcast: None, - last_end: None, - last_arrival: None, - } - } - - /// Translate one of `lane`'s timestamps, sampling the arrival time. - pub(crate) fn translate(&mut self, lane: &mut Lane, pts: moq_net::Timestamp) -> crate::Result { - self.translate_at(lane, pts, self.clock.now().value()) - } - - /// Translate one of `lane`'s timestamps, arriving at monotonic `now` micros. - pub(crate) fn translate_at( - &mut self, - lane: &mut Lane, - pts: moq_net::Timestamp, - now: u64, - ) -> crate::Result { - let src = pts.as_micros(); - - match self.offset { - // The first frame is live now; the source keeps its spacing from there. Rebasing by - // the frame's own PTS (rather than pretending it is timestamp zero) is what keeps a - // delayed first frame honest. - None => self.offset = Some(now as i128 - src as i128), - Some(_) => { - let stepped_back = lane - .last_source - .is_some_and(|last| last > src + SourceMap::MAX_REORDER.as_micros()); - // A restart another lane already applied is adopted, not applied twice. - if (lane.restart || stepped_back) && lane.offset.is_some() && lane.generation == self.generation { - self.reanchor(src, now); - } - if lane.restart || stepped_back { - lane.offset = None; - } - } - } - lane.restart = false; - - // A lane joining late, or following a restart, takes the source's current mapping. - let offset = *lane.offset.get_or_insert_with(|| { - lane.generation = self.generation; - self.offset.expect("anchored above") - }); - - let mapped = src as i128 + offset; - if mapped < 0 { - return Err(crate::Error::UnmappableTimestamp(format!( - "{pts:?} lands before the broadcast began" - ))); - } - // The broadcast clock counts in micros, so the mapping must be nameable there too. - u64::try_from(mapped) - .ok() - .and_then(|mapped| moq_net::Timestamp::from_micros(mapped).ok()) - .ok_or_else(|| { - crate::Error::UnmappableTimestamp(format!("{pts:?} lands outside the representable range")) - })?; - - // Keep the source's timescale: the offset is constant, so the spacing stays exact. - let scale = pts.scale(); - let shift = offset * scale.as_u64() as i128 / 1_000_000; - let value = u64::try_from(pts.value() as i128 + shift) - .map_err(|_| crate::Error::UnmappableTimestamp(format!("{pts:?} lands outside the representable range")))?; - let translated = moq_net::Timestamp::new(value, scale) - .map_err(|_| crate::Error::UnmappableTimestamp(format!("{pts:?} lands outside the representable range")))?; - - if let Some(step) = lane - .last_source - .and_then(|last| src.checked_sub(last)) - .filter(|step| *step > 0) - { - lane.step = Some(lane.step.map_or(step, |min| min.min(step))); - } - lane.last_source = Some(src); - - let start = mapped as u128; - self.last_broadcast = Some(self.last_broadcast.map_or(start, |last| last.max(start))); - self.extend_micros(start + lane.step.unwrap_or(0)); - self.last_arrival = Some(now); - Ok(translated) - } - - /// Record that the broadcast has published up to `end`, e.g. a fragment's last sample end. - pub(crate) fn extend(&mut self, end: moq_net::Timestamp) { - self.extend_micros(end.as_micros()); - } - - fn extend_micros(&mut self, end: u128) { - self.last_end = Some(self.last_end.map_or(end, |last| last.max(end))); - } - - /// Move the anchor so `src` continues after everything published plus the idle gap since the - /// previous arrival: the real downtime for a paced source, and at least the last frames' end - /// for one arriving in a burst. - fn reanchor(&mut self, src: u128, now: u64) { - let base = match (self.last_broadcast, self.last_arrival) { - (Some(last), Some(arrival)) => { - let idle = last + now.saturating_sub(arrival) as u128; - idle.max(self.last_end.unwrap_or(0)) - } - _ => now as u128, - }; - self.offset = Some(base as i128 - src as i128); - self.generation += 1; - } -} - #[cfg(test)] mod tests { use super::*; @@ -411,7 +185,7 @@ mod tests { let clock = Clock::at(epoch(), moq_epoch() + Duration::from_secs(1)).unwrap(); let shared = clock; // Compare the anchors, not live readings: two `now()` calls race the clock. - assert_eq!(clock.epoch, shared.epoch); + assert_eq!(clock.instant, shared.instant); assert_eq!(clock.wall(), shared.wall()); } @@ -458,135 +232,6 @@ mod tests { assert_eq!(Clock::at(epoch(), moq_epoch()).unwrap().wall().wall.value(), 0); } - #[test] - fn delayed_first_frame_anchors_live() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - - // The source's first frame already carries 5s of PTS; it is live now, not now + 5s. - let first = source.translate_at(us(5_000_000), 100_000).unwrap(); - assert_eq!(first.as_micros(), 100_000); - - // Spacing survives: a second later on the source is a second later on the broadcast. - let second = source.translate_at(us(6_000_000), 1_100_000).unwrap(); - assert_eq!(second.as_micros(), 1_100_000); - - // And the wall mapping accounts for the PTS: broadcast 100ms is wall + 100ms. - assert_eq!( - clock.wall_clock(first).unwrap(), - moq_epoch() + Duration::from_micros(100_000) - ); - } - - #[test] - fn multiple_timescales_share_one_mapping() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - - // Two sources, 90kHz video and 48kHz audio, anchored at the same arrival instant. - let mut video = clock.source(); - let mut audio = clock.source(); - let v = video - .translate_at( - moq_net::Timestamp::new(180_000, moq_net::Timescale::new(90_000).unwrap()).unwrap(), - 1_000_000, - ) - .unwrap(); - let a = audio - .translate_at( - moq_net::Timestamp::new(96_000, moq_net::Timescale::new(48_000).unwrap()).unwrap(), - 1_000_000, - ) - .unwrap(); - - // Both said "2s of content, live now": one broadcast instant, one wall time. - assert_eq!(v.as_micros(), 1_000_000); - assert_eq!(a.as_micros(), 1_000_000); - assert_eq!(clock.wall_clock(v).unwrap(), clock.wall_clock(a).unwrap()); - - // A media second later is a broadcast second later on both. - let v2 = video - .translate_at( - moq_net::Timestamp::new(270_000, moq_net::Timescale::new(90_000).unwrap()).unwrap(), - 2_000_000, - ) - .unwrap(); - let a2 = audio - .translate_at( - moq_net::Timestamp::new(144_000, moq_net::Timescale::new(48_000).unwrap()).unwrap(), - 2_000_000, - ) - .unwrap(); - assert_eq!(v2.as_micros(), 2_000_000); - assert_eq!(a2.as_micros(), 2_000_000); - } - - #[test] - fn reset_translation_preserves_the_idle_gap() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - - assert_eq!(source.translate_at(us(0), 1_000_000).unwrap().as_micros(), 1_000_000); - assert_eq!( - source.translate_at(us(2_000_000), 3_000_000).unwrap().as_micros(), - 3_000_000 - ); - - // The encoder restarts at zero 5s later: the broadcast continues after the gap, and the - // wall epoch is untouched. - let wall_before = clock.wall(); - let resumed = source.translate_at(us(0), 8_000_000).unwrap(); - assert_eq!(resumed.as_micros(), 8_000_000); - assert_eq!(clock.wall(), wall_before); - - // Spacing resumes from the new anchor. - let next = source.translate_at(us(1_000_000), 9_000_000).unwrap(); - assert_eq!(next.as_micros(), 9_000_000); - } - - #[test] - fn explicit_reset_marks_a_detected_restart() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - - assert_eq!(source.translate_at(us(0), 1_000_000).unwrap().as_micros(), 1_000_000); - // The adapter saw the restart out of band and re-anchors, even though the PTS did not - // move backwards. - let resumed = source.reset_at(us(0), 4_000_000).unwrap(); - assert_eq!(resumed.as_micros(), 4_000_000); - } - - #[test] - fn bframe_reordering_within_a_group_survives() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - - assert_eq!(source.translate_at(us(0), 1_000_000).unwrap().as_micros(), 1_000_000); - assert_eq!( - source.translate_at(us(40_000), 1_040_000).unwrap().as_micros(), - 1_040_000 - ); - // A 20ms present-before-decode step back is reordering, not a reset: the offset stands. - let reordered = source.translate_at(us(20_000), 1_040_000).unwrap(); - assert_eq!(reordered.as_micros(), 1_020_000); - // And the broadcast continues from the reordered frontier. - let next = source.translate_at(us(80_000), 1_080_000).unwrap(); - assert_eq!(next.as_micros(), 1_080_000); - } - - #[test] - fn mapping_past_u64_micros_is_refused() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - - assert_eq!(source.translate_at(us(0), 0).unwrap().as_micros(), 0); - // A seconds-scale timestamp whose microsecond mapping exceeds u64::MAX. - let huge = moq_net::Timestamp::from_secs(u64::MAX / 1_000_000 + 2).unwrap(); - assert!(matches!( - source.translate_at(huge, 0), - Err(crate::Error::UnmappableTimestamp(_)) - )); - } - #[test] fn fresh_clock_leaves_room_before_now() { let clock = Clock::new(); @@ -600,78 +245,27 @@ mod tests { assert!(drift < Duration::from_secs(1), "wall + now is the current wall time"); } + /// A source whose first timestamp is ten hours in reads that timestamp now, mapped to now. #[test] - fn muxed_lanes_share_one_mapping() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut anchor = Anchor::new(clock); - let (mut video, mut audio) = (Lane::default(), Lane::default()); - - // Video anchors live; audio, muxed 800ms earlier, joins on the same offset. - let v = anchor.translate_at(&mut video, us(10_800_000), 2_000_000).unwrap(); - assert_eq!(v.as_micros(), 2_000_000); - let a = anchor.translate_at(&mut audio, us(10_000_000), 2_000_000).unwrap(); - assert_eq!(a.as_micros(), 1_200_000); - - // Interleaving steps back further than a reorder across lanes, which is not a reset. - let v = anchor.translate_at(&mut video, us(11_800_000), 3_000_000).unwrap(); - assert_eq!(v.as_micros(), 3_000_000); - let a = anchor.translate_at(&mut audio, us(11_000_000), 3_000_000).unwrap(); - assert_eq!(a.as_micros(), 2_200_000); - } - - #[test] - fn muxed_restart_reanchors_once() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut anchor = Anchor::new(clock); - let (mut video, mut audio) = (Lane::default(), Lane::default()); - - anchor.translate_at(&mut video, us(5_000_000), 1_000_000).unwrap(); - anchor.translate_at(&mut audio, us(5_000_000), 1_000_000).unwrap(); - - // The source restarts at zero after 4s idle: video notices first and moves the anchor to - // the last published instant plus the gap. - let v = anchor.translate_at(&mut video, us(0), 5_000_000).unwrap(); - assert_eq!(v.as_micros(), 5_000_000); - // Audio's own step back adopts that mapping rather than adding the gap again. - let a = anchor.translate_at(&mut audio, us(20_000), 5_020_000).unwrap(); - assert_eq!(a.as_micros(), 5_020_000); - - // An out-of-band restart flagged on every lane is applied once as well. - video.restart(); - audio.restart(); - let v = anchor.translate_at(&mut video, us(0), 6_000_000).unwrap(); - assert_eq!(v.as_micros(), 6_000_000); - let a = anchor.translate_at(&mut audio, us(0), 6_000_000).unwrap(); - assert_eq!(a.as_micros(), 6_000_000); - } - - #[test] - fn translation_keeps_the_source_timescale() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); - let scale = moq_net::Timescale::new(90_000).unwrap(); - - let first = source - .translate_at(moq_net::Timestamp::new(3003, scale).unwrap(), 1_000_000) - .unwrap(); - let second = source - .translate_at(moq_net::Timestamp::new(6006, scale).unwrap(), 1_033_000) - .unwrap(); - // 90 kHz in, 90 kHz out, with the frame spacing exact in ticks. - assert_eq!(first.scale(), scale); - assert_eq!(second.value() - first.value(), 3003); - } + fn arrival_maps_the_first_timestamp_to_now() { + let since = Duration::from_secs(10 * 3600); + let clock = Clock::arrival(since).unwrap(); + let now = clock.now(); + assert!(now.as_micros() >= since.as_micros()); + let drift = clock + .wall_clock(now) + .unwrap() + .duration_since(SystemTime::now()) + .unwrap_or_else(|err| err.duration()); + assert!(drift < Duration::from_secs(1), "the first timestamp is live on arrival"); - #[test] - fn mapping_before_the_broadcast_began_is_refused() { - let clock = Clock::at(epoch(), moq_epoch()).unwrap(); - let mut source = clock.source(); + // A capture from before the arrival still maps, down to PTS zero. + let earlier = clock.capture(Instant::now() - Duration::from_secs(1)).unwrap(); + assert!(earlier.as_micros() < now.as_micros()); - // The first frame carries 100ms of PTS but arrives 50ms in: the offset is negative. - assert_eq!(source.translate_at(us(100_000), 50_000).unwrap().as_micros(), 50_000); - // A small step back stays within reorder tolerance but lands before broadcast zero. + // PTS zero before 2020 cannot be named on the wire. assert!(matches!( - source.translate_at(us(0), 50_000), + Clock::arrival(Duration::from_secs(100 * 365 * 86_400)), Err(crate::Error::UnmappableTimestamp(_)) )); } diff --git a/rs/moq-mux/src/codec/aac/import.rs b/rs/moq-mux/src/codec/aac/import.rs index 4f24eae8e2..c5f6571b64 100644 --- a/rs/moq-mux/src/codec/aac/import.rs +++ b/rs/moq-mux/src/codec/aac/import.rs @@ -43,11 +43,6 @@ impl Import { self.track.name() } - /// The exclusive presentation end earlier groups have reached, if any. - pub(crate) fn live_edge(&self) -> Option { - self.track.live_edge() - } - /// A watch-only handle to this track's subscriber demand. pub fn demand(&self) -> moq_net::track::Demand { self.track.track().demand() diff --git a/rs/moq-mux/src/codec/h264/import.rs b/rs/moq-mux/src/codec/h264/import.rs index dc1ed46da5..ab2847b1e2 100644 --- a/rs/moq-mux/src/codec/h264/import.rs +++ b/rs/moq-mux/src/codec/h264/import.rs @@ -114,12 +114,6 @@ impl Import { self.track.name() } - /// The lowest timestamp the track accepts next; see - /// [`Producer::floor`](crate::container::Producer::floor). - pub(crate) fn floor(&self, keyframe: bool) -> Option { - self.track.floor(keyframe) - } - /// A watch-only handle to this track's subscriber demand. pub fn demand(&self) -> moq_net::track::Demand { self.track.track().demand() diff --git a/rs/moq-mux/src/codec/h265/import.rs b/rs/moq-mux/src/codec/h265/import.rs index a5607b3155..96374ad00f 100644 --- a/rs/moq-mux/src/codec/h265/import.rs +++ b/rs/moq-mux/src/codec/h265/import.rs @@ -118,12 +118,6 @@ impl Import { self.track.name() } - /// The lowest timestamp the track accepts next; see - /// [`Producer::floor`](crate::container::Producer::floor). - pub(crate) fn floor(&self, keyframe: bool) -> Option { - self.track.floor(keyframe) - } - /// A watch-only handle to this track's subscriber demand. pub fn demand(&self) -> moq_net::track::Demand { self.track.track().demand() diff --git a/rs/moq-mux/src/codec/legacy.rs b/rs/moq-mux/src/codec/legacy.rs index dcb9218829..d54fc8814f 100644 --- a/rs/moq-mux/src/codec/legacy.rs +++ b/rs/moq-mux/src/codec/legacy.rs @@ -157,11 +157,6 @@ impl Import { self.track.name() } - /// The exclusive presentation end earlier groups have reached, if any. - pub fn live_edge(&self) -> Option { - self.track.live_edge() - } - /// Finish the track, flushing the current group. pub fn finish(&mut self) -> crate::Result<()> { self.track.finish()?; diff --git a/rs/moq-mux/src/codec/opus/import.rs b/rs/moq-mux/src/codec/opus/import.rs index 69b4631c86..c7b69925d2 100644 --- a/rs/moq-mux/src/codec/opus/import.rs +++ b/rs/moq-mux/src/codec/opus/import.rs @@ -42,11 +42,6 @@ impl Import { self.track.track().name() } - /// The exclusive presentation end earlier groups have reached, if any. - pub(crate) fn live_edge(&self) -> Option { - self.track.live_edge() - } - /// A watch-only handle to this track's subscriber demand. pub fn demand(&self) -> moq_net::track::Demand { self.track.track().demand() diff --git a/rs/moq-mux/src/container/consumer.rs b/rs/moq-mux/src/container/consumer.rs index ae0dba0652..42c7442467 100644 --- a/rs/moq-mux/src/container/consumer.rs +++ b/rs/moq-mux/src/container/consumer.rs @@ -77,13 +77,17 @@ pub struct Consumer { max_age: std::time::Duration, // The live edge of playback: the largest timestamp delivered so far and the group that - // carried it. `None` until the first frame is delivered. A later group below this is - // malformed. + // carried it. `None` until the first frame is delivered. live_edge: Option<(u64, Timestamp)>, - // Max timestamp of groups the cursor has left. Open-GOP pictures may sit below this - // group's running max, but not below an earlier group's edge. - group_edge: Option, + // The first frame delivered from the latest group, and that group. Group starts never go + // backwards, so a later group with a frame below this is malformed. + start: Option<(u64, Timestamp)>, + + // The start of the group the cursor most recently left. B-frames, open-GOP pictures, and a + // keyframe overlapping the previous group's last frame may dip below that group's content, + // but a frame below its start is a rewind. + floor: Option, // Presentation end of the group we most recently advanced past, for the 1 ms // contiguity check on a delivered hole. @@ -133,7 +137,8 @@ impl Consumer { startup: start.is_none(), max_age, live_edge: None, - group_edge: None, + start: None, + floor: None, presented_end: None, discontinuity: 0, end: None, @@ -230,9 +235,12 @@ impl Consumer { Poll::Ready(Ok(Some(Event::Frame(frame)))) => { let seq = group.group.sequence; let ts = frame.timestamp; - if self.group_edge.is_some_and(|edge| ts.as_micros() < edge.as_micros()) { + if self.floor.is_some_and(|floor| ts.as_micros() < floor.as_micros()) { return Poll::Ready(Err(TimestampRewind.into())); } + if self.start.is_none_or(|(start, _)| start != seq) { + self.start = Some((seq, ts)); + } if self.live_edge.is_none_or(|(_, high)| ts.as_micros() > high.as_micros()) { self.live_edge = Some((seq, ts)); } @@ -431,8 +439,8 @@ impl Consumer { } fn note_group_edge(&mut self) { - if let Some((_, ts)) = self.live_edge { - self.group_edge = Some(ts); + if let Some((_, ts)) = self.start { + self.floor = Some(ts); } } @@ -471,10 +479,10 @@ impl Consumer { } } - // A group whose media timestamps sit below the live edge earlier groups reached is - // malformed. Markers have no media timestamp, so they are not this check. + // A later group with a media timestamp below the latest delivered group's start is malformed: + // group starts never go backwards. Markers have no media timestamp, so they are not this check. fn poll_malformed(&mut self, waiter: &kio::Waiter) -> Result<(), F::Error> { - let Some((prev_group, edge)) = self.live_edge else { + let Some((prev_group, edge)) = self.start else { return Ok(()); }; @@ -1110,7 +1118,7 @@ mod tests { // ---- Malformed rewind ---- #[tokio::test] - async fn a_group_below_the_live_edge_aborts() { + async fn a_group_starting_below_the_previous_start_aborts() { let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); let mut consumer = Consumer::new(track.subscribe(None), Container::Legacy(crate::container::Kind::Data)); write_group(&mut track, 0, &[ts(100_000)]); @@ -1135,6 +1143,38 @@ mod tests { assert_eq!(consumer.discontinuity(), 0); } + #[tokio::test] + async fn a_group_starting_at_the_previous_start_is_accepted() { + let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let mut consumer = Consumer::new(track.subscribe(None), Container::Legacy(crate::container::Kind::Data)); + write_group(&mut track, 0, &[ts(100_000)]); + assert_eq!(consumer.read().await.unwrap().unwrap().timestamp, ts(100_000)); + write_group(&mut track, 1, &[ts(100_000)]); + track.finish().unwrap(); + assert_eq!(consumer.read().await.unwrap().unwrap().timestamp, ts(100_000)); + } + + /// A keyframe one frame below the previous group's last frame, but above its start, is an + /// overlap rather than a rewind. + #[tokio::test] + async fn a_keyframe_overlapping_the_previous_group_is_accepted() { + let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + let mut consumer = Consumer::new(track.subscribe(None), Container::Legacy(crate::container::Kind::Data)); + write_group(&mut track, 0, &[ts(0), ts(33_000), ts(66_000)]); + // Read the first group before the second arrives: a blocked group that the next one + // already covers is skipped for latency, which is not what this checks. + for expected in [0, 33_000, 66_000] { + assert_eq!(consumer.read().await.unwrap().unwrap().timestamp, ts(expected)); + } + write_group(&mut track, 1, &[ts(50_000), ts(83_000)]); + track.finish().unwrap(); + let frames = read_all(&mut consumer).await.unwrap(); + assert_eq!( + frames.iter().map(|f| f.timestamp.as_micros()).collect::>(), + vec![50_000, 83_000] + ); + } + #[tokio::test] async fn open_gop_leading_pictures_above_the_previous_group_are_accepted() { let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); @@ -1218,7 +1258,7 @@ mod tests { } #[tokio::test] - async fn a_later_frame_below_the_previous_group_edge_aborts() { + async fn a_later_frame_below_the_previous_group_start_aborts() { let mut track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); let mut consumer = Consumer::new(track.subscribe(None), Container::Legacy(crate::container::Kind::Data)); write_group(&mut track, 0, &[ts(100_000)]); diff --git a/rs/moq-mux/src/container/flv/import.rs b/rs/moq-mux/src/container/flv/import.rs index d4d7fe267b..0f7436b91e 100644 --- a/rs/moq-mux/src/container/flv/import.rs +++ b/rs/moq-mux/src/container/flv/import.rs @@ -80,10 +80,6 @@ pub struct Import { video: BTreeMap, /// Demuxed audio tracks keyed by RTMP track id. audio: BTreeMap, - - /// The source's mapping onto the broadcast clock, set by [`live`](Self::live). `None` - /// publishes the tag timestamps verbatim. - anchor: Option, } /// The demuxed video track plus its current catalog config, so a repeated @@ -93,14 +89,12 @@ struct VideoStream { config: VideoConfig, stalled: hang::catalog::stalled::Detector, last_source: Option, - lane: crate::clock::Lane, } /// The demuxed audio track plus its current catalog config. struct AudioStream { track: crate::container::Producer, config: AudioConfig, - lane: crate::clock::Lane, } impl Import { @@ -116,22 +110,9 @@ impl Import { header_seen: false, video: BTreeMap::new(), audio: BTreeMap::new(), - anchor: None, } } - /// Publish on the broadcast clock rather than the source's own tag timestamps. - /// - /// For a live feed with its own zero: the first frame is live on arrival, every track shares - /// that one mapping, and an encoder that restarts its timestamps continues forward after the - /// real idle gap. Without this, tag timestamps are published verbatim, which suits a source - /// already on the clock the catalog advertises - /// ([`Config::with_clock`](crate::catalog::Config::with_clock)). - pub fn live(mut self) -> Self { - self.anchor = Some(crate::clock::Anchor::new(self.catalog.clock())); - self - } - /// Select the container this importer wraps decoded media renditions in. /// /// [`Legacy`](hang::catalog::Container::Legacy) unless selected. It applies to every rendition @@ -486,16 +467,14 @@ impl Import { tracing::debug!("video frame before sequence header, dropping"); return Ok(()); } + let timestamp = Timestamp::from_millis(pts_ms as u64)?; + // The first frame is live on arrival, anchored before the reservation below publishes. + self.catalog.anchor(timestamp)?; // A media frame means every sequence header has arrived (FLV sends config before data), so // the track set is declared; release the reservation to publish. self.initial_reservation = None; let written = { let stream = self.video.get_mut(&track_id).expect("checked above"); - let timestamp = Timestamp::from_millis(pts_ms as u64)?; - let timestamp = match self.anchor.as_mut() { - Some(anchor) => anchor.translate(&mut stream.lane, timestamp)?, - None => timestamp, - }; match stream.track.write(Frame { timestamp, duration: None, @@ -518,18 +497,17 @@ impl Import { /// Write one audio frame as its own group, so the relay can forward it immediately. fn write_audio(&mut self, track_id: u8, data: &[u8], timestamp: u64) -> anyhow::Result<()> { - let Some(stream) = self.audio.get_mut(&track_id) else { + if !self.audio.contains_key(&track_id) { tracing::debug!("audio frame before config, dropping"); return Ok(()); - }; + } + let timestamp = Timestamp::from_millis(timestamp)?; + // The first frame is live on arrival, anchored before the reservation below publishes. + self.catalog.anchor(timestamp)?; // A media frame means every sequence header has arrived (FLV sends config before data), so // the track set is declared; release the reservation to publish. self.initial_reservation = None; - let timestamp = Timestamp::from_millis(timestamp)?; - let timestamp = match self.anchor.as_mut() { - Some(anchor) => anchor.translate(&mut stream.lane, timestamp)?, - None => timestamp, - }; + let stream = self.audio.get_mut(&track_id).expect("checked above"); stream.track.write(Frame { timestamp, duration: None, @@ -565,7 +543,6 @@ impl Import { config, stalled: hang::catalog::stalled::Detector::new(), last_source: None, - lane: Default::default(), }, ); Ok(()) @@ -587,14 +564,7 @@ impl Import { Some(reserved) => reserved.audio(net_track, wire, config.clone())?, None => self.catalog.audio(net_track, wire, config.clone())?, }; - self.audio.insert( - track_id, - AudioStream { - track: media, - config, - lane: Default::default(), - }, - ); + self.audio.insert(track_id, AudioStream { track: media, config }); Ok(()) } @@ -664,11 +634,9 @@ impl Import { pub fn seek(&mut self, sequence: u64) -> crate::Result<()> { for stream in self.video.values_mut() { stream.track.seek(sequence)?; - stream.lane.restart(); } for stream in self.audio.values_mut() { stream.track.seek(sequence)?; - stream.lane.restart(); } Ok(()) } diff --git a/rs/moq-mux/src/container/flv/import_test.rs b/rs/moq-mux/src/container/flv/import_test.rs index ceb6a78897..e4b72b31a8 100644 --- a/rs/moq-mux/src/container/flv/import_test.rs +++ b/rs/moq-mux/src/container/flv/import_test.rs @@ -708,36 +708,35 @@ fn session(header: bool, start_ms: u32, frames: u32) -> Vec { out } -/// What an FLV import published, plus the broadcast clock's reading around the first chunk. +/// What an FLV import published, and the wall-clock window its first chunk arrived in. struct Imported { published: std::collections::BTreeMap>, video: String, - before: u128, - after: u128, + /// The root clock the catalog advertised. + clock: hang::catalog::Clock, + arrival: std::ops::RangeInclusive, + /// Why a chunk was refused, if one was. + refused: Option, } -/// Import `chunks` in order, idling `idle` between them, on a clock that began `ago` earlier. -/// `live` translates onto that clock; otherwise tag timestamps publish verbatim. -async fn import(chunks: &[Vec], live: bool, ago: Duration, idle: Duration) -> Imported { +/// Import `chunks` in order on a catalog with the default clock, stopping at the first refused +/// chunk. +async fn import(chunks: &[Vec]) -> Imported { let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); - let config = crate::catalog::Config::default().with_clock(crate::container::test_util::late_clock(ago)); - let catalog = crate::catalog::Producer::new(&mut broadcast, config).unwrap(); - let clock = catalog.clock(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); let mut importer = Import::new(broadcast, catalog.reserve()); - if live { - importer = importer.live(); - } - let before = clock.now().as_micros(); + let before = std::time::SystemTime::now(); let mut after = before; + let mut refused = None; for (i, chunk) in chunks.iter().enumerate() { - if i > 0 { - std::thread::sleep(idle); + if let Err(err) = importer.decode(chunk) { + refused = Some(err); + break; } - importer.decode(chunk).unwrap(); if i == 0 { - after = clock.now().as_micros(); + after = std::time::SystemTime::now(); } } importer.finish().unwrap(); @@ -746,63 +745,45 @@ async fn import(chunks: &[Vec], live: bool, ago: Duration, idle: Duration) - Imported { published: crate::container::test_util::published(&consumer, &snapshot).await, video: snapshot.video.renditions.keys().next().unwrap().clone(), - before, - after, + clock: snapshot.clock.expect("the catalog advertises a clock"), + arrival: before..=after, + refused, } } -/// A feed an hour into its own timeline, arriving 30s after the broadcast began, publishes on the -/// broadcast clock: the first frame is live on arrival, and audio and video keep the one offset -/// their tags gave them, B-frame reordering included. +/// A feed an hour into its own timeline publishes its tag timestamps verbatim, and the catalog +/// clock maps its first frame to the arrival time. #[tokio::test] -async fn live_import_anchors_a_late_first_frame() { - let input = [session(true, 3_600_000, 50)]; - let ago = Duration::from_secs(30); - let verbatim = import(&input, false, ago, Duration::ZERO).await; - let live = import(&input, true, ago, Duration::ZERO).await; - - crate::container::test_util::common_offset(&verbatim.published, &live.published); - let first = live.published[&live.video][0]; +async fn import_publishes_tag_timestamps_on_an_arrival_clock() { + let start_ms = 3_600_000; + let import = import(&[session(true, start_ms, 50)]).await; + assert!(import.refused.is_none()); + + // The first video frame presents 40ms after its decode time. + let first = import.published[&import.video][0]; + assert_eq!(first, (start_ms as u128 + 40) * 1000, "the source's own timestamp"); + + let tick = Duration::from_millis(1); + let wall = import + .clock + .wall_clock(moq_net::Timestamp::from_micros(first as u64).unwrap()) + .unwrap(); assert!( - (live.before..=live.after).contains(&first), - "the first frame is live on arrival: {first} not in {}..={}", - live.before, - live.after + *import.arrival.start() - tick <= wall && wall <= *import.arrival.end() + tick, + "the first frame is live on arrival" ); } -/// An encoder restarting its timestamps at zero continues the broadcast forward: after the real -/// idle gap, and with every track moving onto the one new mapping. +/// An encoder restarting its timestamps is a new epoch: the import refuses the rewind rather than +/// re-anchoring it forward, keeping everything published before it. #[tokio::test] -async fn live_import_restarts_forward_after_idle() { - for idle in [Duration::ZERO, Duration::from_millis(300)] { - let input = [session(true, 5_000, 50), session(false, 0, 50)]; - let live = import(&input, true, Duration::from_secs(30), idle).await; - - let last_before = live - .published - .values() - .map(|t| t[..50].iter().max().unwrap()) - .max() - .unwrap(); - let video = &live.published[&live.video]; - let gap = video[50] as i128 - *last_before as i128; - assert!( - gap >= idle.as_micros() as i128, - "the restart lands after the idle gap: {gap}us after {idle:?}" - ); - assert!( - gap < (idle + Duration::from_secs(5)).as_micros() as i128, - "the restart is not pushed further: {gap}us" - ); - - // The second session keeps its own A/V relationship on the new mapping. - let verbatim = import(&[session(true, 0, 50)], false, Duration::ZERO, Duration::ZERO).await; - let second = live - .published - .iter() - .map(|(name, t)| (name.clone(), t[50..].to_vec())) - .collect(); - crate::container::test_util::common_offset(&verbatim.published, &second); - } +async fn import_refuses_a_restart() { + let import = import(&[session(true, 5_000, 50), session(false, 0, 50)]).await; + let err = import.refused.expect("the restart is refused"); + assert!(matches!(err, crate::Error::TimestampRewind(_)), "{err:?}"); + assert_eq!( + import.published[&import.video].len(), + 50, + "the first session stays published" + ); } diff --git a/rs/moq-mux/src/container/fmp4/import.rs b/rs/moq-mux/src/container/fmp4/import.rs index d30facd4e4..0a04d8c06c 100644 --- a/rs/moq-mux/src/container/fmp4/import.rs +++ b/rs/moq-mux/src/container/fmp4/import.rs @@ -89,10 +89,6 @@ pub struct Import { // Only the timeline report is anchored. Each fragment still carries its own timestamp on the // wire, and `Recorder::end` still reports real content time. segment_start: Option, - - // The source's mapping onto the broadcast clock, set by `live`. `None` publishes the source's - // decode times verbatim. - anchor: Option, } /// The catalog entry for one imported track, whichever section it lives in. @@ -136,9 +132,6 @@ struct Fmp4Track { // Sequence to use for the next group, set by `Import::seek`. pending_sequence: Option, - // This track's position on the source's broadcast-clock mapping. - lane: crate::clock::Lane, - // The segment this track's open group belongs to. A mismatch with `Import::segment` rolls the // group, which is what keeps audio on the same boundaries as video. segment: Option, @@ -181,22 +174,9 @@ impl Import { segment: 0, pending_timeline_cut: false, segment_start: None, - anchor: None, } } - /// Publish on the broadcast clock rather than the source's own decode times. - /// - /// For a live feed with its own zero: the first fragment is live on arrival, every track - /// shares that one mapping, and a source that restarts its decode times continues forward - /// after the real idle gap instead of being refused. Each fragment's `tfdt` is rewritten to - /// match. Without this, decode times are published verbatim, which suits a source already on - /// the clock the catalog advertises ([`Config::with_clock`](crate::catalog::Config::with_clock)). - pub fn live(mut self) -> Self { - self.anchor = Some(crate::clock::Anchor::new(self.catalog.clock())); - self - } - /// Declare that the next fragment starts a new segment, for callers that know the source's /// segmentation out of band (e.g. an HLS import following its playlist). /// @@ -367,7 +347,6 @@ impl Import { last_decode_time: None, sample_duration: None, pending_sequence: None, - lane: Default::default(), estimator: Estimator::new(), claim: crate::catalog::Claim::new(self.catalog.bandwidth()), }, @@ -728,15 +707,7 @@ impl Import { let tfdt = traf.tfdt.as_ref().ok_or(Error::MissingTfdt)?; let timescale = moq_net::Timescale::new(trak.mdia.mdhd.timescale as u64)?; - // The decode time this fragment is published at, and so rewritten into its `tfdt`. - let base_decode_time = match self.anchor.as_mut() { - Some(anchor) => { - let source = Timestamp::new(tfdt.base_media_decode_time, timescale)?; - anchor.translate(&mut track.lane, source)?.value() - } - None => tfdt.base_media_decode_time, - }; - let mut dts = base_decode_time; + let mut dts = tfdt.base_media_decode_time; // Every fragment restates its decode time, so a stale one puts two different samples // on the same timestamp, which reads downstream as an undeclared hole. @@ -882,9 +853,6 @@ impl Import { // and ensuring trun.data_offset is Some(...) reserves 4 bytes per trun. for traf_mut in &mut adjusted_moof.traf { traf_mut.tfhd.base_data_offset = None; - traf_mut.tfdt = Some(mp4_atom::Tfdt { - base_media_decode_time: base_decode_time, - }); // A zero default/sample duration is "unknown", not "instantaneous": drop it so // the re-emitted fragment carries no bogus zero that a decoder would honor. if traf_mut.tfhd.default_sample_duration == Some(0) { @@ -937,6 +905,13 @@ impl Import { let fragment_bytes = Bytes::from(moof_buf); + // Carry the fragment's earliest presentation time as the frame timestamp, + // in the track's native timescale. The relay reads it off the wire; the + // consumer still drives playback from the fragment's internal timing. + let timestamp = min_timestamp.ok_or(Error::MissingTrun)?; + // The first fragment of the import is live on arrival. + self.catalog.anchor(timestamp)?; + // Write the per-track fragment as a single MoQ frame (passthrough). The group rolls // once per segment, so a group is a segment and the fragments inside it are frames. // The keyframe bit no longer decides that: audio flags every sample a sync sample, so @@ -956,11 +931,6 @@ impl Import { track.group.take().ok_or(Error::NoKeyframe)? }; - // Carry the fragment's earliest presentation time as the frame timestamp, - // in the track's native timescale. The relay reads it off the wire; the - // consumer still drives playback from the fragment's internal timing. - let timestamp = min_timestamp.ok_or(Error::MissingTrun)?; - if start_group { // A group just opened; report it so the broadcast's timeline can index the segment // (the timeline absorbs publish failures). Audio is always independently decodable; @@ -1002,10 +972,6 @@ impl Import { track.estimator.write(timestamp, fragment_len); let end = max_end.ok_or(Error::MissingTrun)?; - if let Some(anchor) = self.anchor.as_mut() { - // A restart continues after this fragment's last sample, not merely its start. - anchor.extend(end); - } if let Some(recorder) = track.recorder.as_mut() { recorder.end(end); } @@ -1047,7 +1013,9 @@ impl Import { /// Close the current group on every track and open the next one at `sequence`. /// /// Broadcast-wide: every track inside this fMP4 import advances together; per-track - /// control is intentionally not exposed. + /// control is intentionally not exposed. Skipping sequences is not a new timeline, so the + /// next fragment must still advance past the last decode time; see + /// [`discontinuity`](Self::discontinuity). pub fn seek(&mut self, sequence: u64) -> Result<()> { for track in self.tracks.values_mut() { track.estimator.cut(None); @@ -1056,11 +1024,17 @@ impl Import { g.finish()?; } track.pending_sequence = Some(sequence); - track.last_decode_time = None; - track.lane.restart(); } Ok(()) } + + /// The source signalled a new timeline (an HLS `EXT-X-DISCONTINUITY`), so the next + /// fragment's decode time is not compared with the last one on each track. + pub fn discontinuity(&mut self) { + for track in self.tracks.values_mut() { + track.last_decode_time = None; + } + } } /// Whether a sample's `trun` flags mark it a sync sample (an IDR): `sample_depends_on == 2` and diff --git a/rs/moq-mux/src/container/fmp4/import_test.rs b/rs/moq-mux/src/container/fmp4/import_test.rs index 40c10a626d..e4737c85a0 100644 --- a/rs/moq-mux/src/container/fmp4/import_test.rs +++ b/rs/moq-mux/src/container/fmp4/import_test.rs @@ -1097,22 +1097,6 @@ fn non_advancing_fragment_decode_time_is_rejected() { ); } -#[test] -fn seek_resets_fragment_decode_time() { - let (ftyp, moov) = decode_init(include_bytes!("test_data/bbb.mp4")); - let mut init = Vec::new(); - ftyp.encode(&mut init).unwrap(); - moov.encode(&mut init).unwrap(); - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); - let mut fmp4 = crate::container::fmp4::Import::new(broadcast, catalog.reserve()); - fmp4.decode(&init).unwrap(); - fmp4.decode(&audio_fragment(4096, 1024, 327)).unwrap(); - fmp4.seek(1).unwrap(); - fmp4.decode(&audio_fragment(0, 1024, 327)).unwrap(); - fmp4.decode(&audio_fragment(1024, 1024, 327)).unwrap(); -} - #[test] fn rejected_fragment_preserves_decode_time() { let (ftyp, moov) = decode_init(include_bytes!("test_data/bbb.mp4")); @@ -1290,52 +1274,44 @@ fn live_session(start_us: u64, frames: u64) -> Vec { out } -/// What an fMP4 import published, plus the broadcast clock's reading around the first chunk. -struct LiveImport { +/// What an fMP4 import published, and the wall-clock window its first chunk arrived in. +struct Imported { published: std::collections::BTreeMap>, video: String, - clock: crate::Clock, - before: u128, - after: u128, + /// The root clock the catalog advertised. + clock: hang::catalog::Clock, + arrival: std::ops::RangeInclusive, /// The first segment the broadcast timeline recorded, the index an archive replays from. record: moq_net::Timestamp, + /// Why a chunk was refused, if one was. + refused: Option, } -/// Import bbb's init then `chunks`, idling `idle` between them, on a clock that began `ago` -/// earlier. `live` translates onto that clock; otherwise decode times publish verbatim. -async fn live_import( - chunks: &[Vec], - live: bool, - ago: std::time::Duration, - idle: std::time::Duration, -) -> LiveImport { +/// Import bbb's init then `chunks` on a catalog with the default clock, stopping at the first +/// refused chunk. +async fn import_session(chunks: &[Vec]) -> Imported { let (init, _, _) = bbb_init(); let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); - let config = crate::catalog::Config::default().with_clock(crate::container::test_util::late_clock(ago)); - let mut catalog = crate::catalog::Producer::new(&mut broadcast, config).unwrap(); - let clock = catalog.clock(); + let mut catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); let mut fmp4 = crate::container::fmp4::Import::new(broadcast, catalog.reserve()); - if live { - fmp4 = fmp4.live(); - } fmp4.decode(&init).unwrap(); - let snapshot = catalog.snapshot(); - let section = snapshot.archive.clone().expect("the import advertises a timeline"); + let section = catalog.snapshot().archive.expect("the import advertises a timeline"); let mut timeline = crate::timeline::Consumer::<()>::subscribe(&consumer, §ion) .await .unwrap(); - let before = clock.now().as_micros(); + let before = std::time::SystemTime::now(); let mut after = before; + let mut refused = None; for (i, chunk) in chunks.iter().enumerate() { - if i > 0 { - std::thread::sleep(idle); + if let Err(err) = fmp4.decode(chunk) { + refused = Some(err); + break; } - fmp4.decode(chunk).unwrap(); if i == 0 { - after = clock.now().as_micros(); + after = std::time::SystemTime::now(); } } fmp4.finish().unwrap(); @@ -1346,84 +1322,86 @@ async fn live_import( panic!("the first timeline event was not a segment"); }; - LiveImport { + let snapshot = catalog.snapshot(); + Imported { published: crate::container::test_util::published(&consumer, &snapshot).await, video: snapshot.video.renditions.keys().next().unwrap().clone(), - clock, - before, - after, + clock: snapshot.clock.expect("the catalog advertises a clock"), + arrival: before..=after, record: entry.pts, + refused, } } -/// A feed an hour into its own decode timeline, arriving 30s after the broadcast began, publishes -/// on the broadcast clock: the `tfdt` a decoder reads is rewritten so the first fragment is live -/// on arrival, and every track moves by the one offset. The archive index records the same times, -/// so a replay names each segment's real wall time. +/// A feed an hour into its own decode timeline publishes that hour verbatim (the `tfdt` a +/// decoder reads is untouched), and the catalog clock maps its first fragment to the arrival +/// time. The archive index records the same times, so a replay names each segment's wall time. #[tokio::test] -async fn live_import_rewrites_tfdt_onto_the_broadcast_clock() { - let input = [live_session(3_600_000_000, 20)]; - let ago = std::time::Duration::from_secs(30); - let verbatim = live_import(&input, false, ago, std::time::Duration::ZERO).await; - let wall_before = std::time::SystemTime::now(); - let live = live_import(&input, true, ago, std::time::Duration::ZERO).await; - let wall_after = std::time::SystemTime::now(); - - crate::container::test_util::common_offset(&verbatim.published, &live.published); - let first = live.published[&live.video][0]; - assert!( - (live.before..=live.after).contains(&first), - "the first fragment is live on arrival: {first} not in {}..={}", - live.before, - live.after +async fn import_publishes_decode_times_on_an_arrival_clock() { + let start = 3_600_000_000; + let import = import_session(&[live_session(start, 20)]).await; + assert!(import.refused.is_none()); + + let first = import.published[&import.video][0]; + assert_eq!(first, start as u128, "the source's own decode time"); + assert_eq!( + import.record.as_micros(), + first, + "the archive indexes the source's time" ); - // The timeline records at a coarser scale, so allow its rounding. + // The advertised clock is micros, so allow its rounding. let tick = std::time::Duration::from_millis(1); + let wall = import.clock.wall_clock(import.record).unwrap(); assert!( - first.abs_diff(live.record.as_micros()) < tick.as_micros(), - "the archive indexes the translated time" + *import.arrival.start() - tick <= wall && wall <= *import.arrival.end() + tick, + "the first fragment is live on arrival" ); - let wall = live.clock.wall_clock(live.record).unwrap(); +} + +/// A source whose decode times restart is a new epoch: the import refuses the rewind rather than +/// re-anchoring it forward, keeping everything published before it. +#[tokio::test] +async fn import_refuses_a_restart() { + let import = import_session(&[live_session(5_000_000, 20), live_session(0, 20)]).await; + let err = import.refused.expect("the restart is refused"); assert!( - wall_before - tick <= wall && wall <= wall_after, - "a replayed segment names the wall time it went live" + matches!( + err, + crate::Error::Cmaf(crate::container::fmp4::Error::NonMonotonicDecodeTime { .. }) + ), + "{err:?}" + ); + assert_eq!( + import.published[&import.video].len(), + 20, + "the first session stays published" ); } -/// A source whose decode times restart at zero continues forward after the real idle gap rather -/// than being refused as non-monotonic, with every track moving onto one new mapping. +/// Skipping sequences (an HLS media-sequence gap) is not a new timeline: a rewound decode time +/// after a seek is still refused. Only a signalled discontinuity starts the comparison afresh. #[tokio::test] -async fn live_import_restarts_forward_after_idle() { - for idle in [std::time::Duration::ZERO, std::time::Duration::from_millis(300)] { - let input = [live_session(5_000_000, 20), live_session(0, 20)]; - let live = live_import(&input, true, std::time::Duration::from_secs(30), idle).await; - - // The restart lands the arrival gap after the first session's last fragment, and never on - // top of it: that fragment lasts 100ms. - let last = live - .published - .values() - .map(|t| t[..20].iter().max().unwrap()) - .max() - .unwrap(); - let gap = live.published[&live.video][20] as i128 - *last as i128; - let floor = idle.max(std::time::Duration::from_millis(100)); - assert!( - gap >= floor.as_micros() as i128, - "the restart lands after the idle gap: {gap}us after {idle:?}" - ); - assert!( - gap < (idle + std::time::Duration::from_secs(5)).as_micros() as i128, - "the restart is not pushed further: {gap}us" - ); +async fn seek_keeps_the_decode_time_until_a_discontinuity() { + let (init, _, _) = bbb_init(); + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut fmp4 = crate::container::fmp4::Import::new(broadcast, catalog.reserve()); + fmp4.decode(&init).unwrap(); + fmp4.decode(&live_session(5_000_000, 5)).unwrap(); - let verbatim = live_import(&input[1..], false, std::time::Duration::ZERO, std::time::Duration::ZERO).await; - let second = live - .published - .iter() - .map(|(name, t)| (name.clone(), t[20..].to_vec())) - .collect(); - crate::container::test_util::common_offset(&verbatim.published, &second); - } + fmp4.seek(100).unwrap(); + let err = fmp4.decode(&live_session(0, 5)).unwrap_err(); + assert!( + matches!( + err, + crate::Error::Cmaf(crate::container::fmp4::Error::NonMonotonicDecodeTime { .. }) + ), + "{err:?}" + ); + + fmp4.discontinuity(); + fmp4.seek(200).unwrap(); + fmp4.decode(&live_session(0, 5)) + .expect("a signalled discontinuity starts a new timeline"); } diff --git a/rs/moq-mux/src/container/mkv/import.rs b/rs/moq-mux/src/container/mkv/import.rs index 0fbf2bb4c3..72b90d4de7 100644 --- a/rs/moq-mux/src/container/mkv/import.rs +++ b/rs/moq-mux/src/container/mkv/import.rs @@ -49,6 +49,11 @@ pub struct Import { /// Accumulated unparsed input. buffer: BytesMut, + /// Bytes already dropped from the front of `buffer`, so a tag's offset in it is absolute. + consumed: u64, + /// Where the latest handled tag starts, as an absolute offset. A drain pass restarts from a + /// replay point, so any tag starting at or before it was already handled. + handled: Option, /// Whether the Tracks element has been processed. tracks_seen: bool, @@ -82,9 +87,6 @@ struct MkvTrack { kind: TrackKind, track: Media, group: Option, - /// Highest block timestamp (Matroska ticks: cluster_ts + block_relative) already emitted. - /// Used to dedup re-parsed blocks across decode() calls. - last_emitted_ticks: Option, } enum Media { @@ -138,6 +140,8 @@ impl Import { container, initial_reservation: Some(reserved), buffer: BytesMut::new(), + consumed: 0, + handled: None, tracks_seen: false, timestamp_scale_ns: DEFAULT_TIMESTAMP_SCALE_NS, cluster_timestamp: 0, @@ -168,9 +172,8 @@ impl Import { /// Run the iterator over the buffered bytes, processing every fully-parsed top-level tag. /// - /// On each call, the iterator restarts from the beginning of the retained buffer. Tag - /// handling is idempotent (state flags for header/tracks, per-track timestamp dedup for - /// blocks). After parsing stops (UnexpectedEOF or end of buffer), bytes up to the start + /// On each call, the iterator restarts from the beginning of the retained buffer, so tags + /// already handled are skipped by their absolute offset. After parsing stops (UnexpectedEOF or end of buffer), bytes up to the start /// of the most-recently emitted top-level tag are discarded so memory does not grow /// unboundedly. fn drain(&mut self) -> Result<()> { @@ -195,8 +198,8 @@ impl Import { // We restart the iterator from the beginning of the retained buffer on every // drain pass. Once data is replayed mid-Segment, ebml-iterable would otherwise // reject Segment children (Cluster, Tracks, etc.) as appearing without their - // parent. Allowing hierarchy problems plus our own dedup logic on emitted - // blocks gives us idempotent streaming behavior. + // parent. Allowing hierarchy problems plus skipping tags by offset gives us + // idempotent streaming behavior. iter.allow_errors(&[AllowableErrors::HierarchyProblems]); // Don't synthesize Master::End tags when the buffer ends mid-element. iter.emit_master_end_when_eof(false); @@ -207,6 +210,12 @@ impl Import { match iter.next() { Some(Ok(tag)) => { last_offset = iter.last_emitted_tag_offset(); + // A Master::End reports its master's start, so it is skipped here too. + let start = self.consumed + last_offset as u64; + if self.handled.is_some_and(|handled| start <= handled) { + continue; + } + self.handled = Some(start); self.handle_tag(tag)?; } Some(Err(TagIteratorError::UnexpectedEOF { .. })) => break, @@ -227,6 +236,7 @@ impl Import { // If we never emitted anything (very first call with too few bytes), keep everything. if last_offset > 0 { self.buffer.advance(last_offset); + self.consumed += last_offset as u64; } Ok(()) @@ -248,9 +258,7 @@ impl Import { } } } - // Idempotency: if the parser restarts mid-stream and `last_offset` - // happens to point at Tracks (i.e. Tracks was the last fully-emitted - // tag), we'll see it again. Process once. + // A second Tracks element would redeclare a published track set. MatroskaSpec::Tracks(Master::Full(children)) if !self.tracks_seen => { self.handle_tracks(children)?; self.tracks_seen = true; @@ -374,7 +382,6 @@ impl Import { kind, track: media, group: None, - last_emitted_ticks: None, }, ); @@ -419,18 +426,12 @@ impl Import { return Err(Error::NegativeBlockTimestamp.into()); } - // Skip blocks we've already emitted on a previous decode() pass (buffer replay). - if let Some(last) = track.last_emitted_ticks - && block_ticks <= last - { - return Ok(()); - } - track.last_emitted_ticks = Some(block_ticks); - let pts_ns = (block_ticks as u64) .checked_mul(self.timestamp_scale_ns) .ok_or(Error::TimestampOverflow)?; let timestamp = Timestamp::from_nanos(pts_ns)?; + // The first block is live on arrival. + self.catalog.anchor(timestamp)?; // Audio tracks: always treat as keyframes (matches fmp4 behavior). let keyframe = matches!(track.kind, TrackKind::Audio) || keyframe; diff --git a/rs/moq-mux/src/container/mkv/import_test.rs b/rs/moq-mux/src/container/mkv/import_test.rs index e244513c68..354e0c7d71 100644 --- a/rs/moq-mux/src/container/mkv/import_test.rs +++ b/rs/moq-mux/src/container/mkv/import_test.rs @@ -325,46 +325,6 @@ fn test_mp3_catalog() { assert!(a.description.is_none(), "MP3 config is in band"); } -#[test] -fn test_chunked_decode_dedup() { - // Build the same WebM and feed it in tiny chunks. The dedup logic should ensure - // that frames aren't emitted twice across the parse restarts. - let data = MkvBuilder::new() - .header("webm") - .segment_start() - .info(1_000_000) - .track_video(1, "V_VP9", 320, 240) - .cluster(0, || { - vec![ - simple_block(1, 0, true, b"k0"), - simple_block(1, 33, false, b"p1"), - simple_block(1, 66, false, b"p2"), - ] - }) - .cluster(100, || { - vec![simple_block(1, 0, true, b"k1"), simple_block(1, 33, false, b"p3")] - }) - .segment_end() - .build(); - - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); - let mut mkv = crate::container::mkv::Import::new(broadcast, catalog.reserve()); - - // Feed in 16-byte chunks to stress the chunked-restart code path. - for chunk in data.chunks(16) { - let b = bytes::BytesMut::from(chunk); - mkv.decode(&b).expect("decode chunk"); - } - mkv.finish().expect("finish"); - - let catalog = catalog.snapshot(); - assert_eq!(catalog.video.renditions.len(), 1); - let v = catalog.video.renditions.values().next().unwrap(); - assert_eq!(v.coded_width, Some(320)); - assert_eq!(v.coded_height, Some(240)); -} - #[test] fn test_unsupported_codec_skipped() { // Mix of supported (Opus) and unsupported (Vorbis) audio tracks. The Vorbis track @@ -450,3 +410,122 @@ fn rendition_is_not_published_when_the_media_track_fails() { "a rendition whose media producer failed must not be advertised" ); } + +/// What an MKV import published, and the wall-clock window its input arrived in. +struct Imported { + published: std::collections::BTreeMap>, + /// The root clock the catalog advertised. + clock: hang::catalog::Clock, + arrival: std::ops::RangeInclusive, + /// Why a chunk was refused, if one was. + refused: Option, +} + +/// Import `data` in `chunk`-byte pieces on a catalog with the default clock, stopping at the +/// first refused piece. +async fn import_chunked(data: &[u8], chunk: usize) -> Imported { + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let consumer = broadcast.consume(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); + let mut mkv = crate::container::mkv::Import::new(broadcast, catalog.reserve()); + + let before = std::time::SystemTime::now(); + let mut refused = None; + for piece in data.chunks(chunk) { + if let Err(err) = mkv.decode(piece) { + refused = Some(err); + break; + } + } + let after = std::time::SystemTime::now(); + mkv.finish().unwrap(); + + let snapshot = catalog.snapshot(); + Imported { + published: crate::container::test_util::published(&consumer, &snapshot).await, + clock: snapshot.clock.expect("the catalog advertises a clock"), + arrival: before..=after, + refused, + } +} + +/// Every drain pass restarts from a replay point, so a small chunk replays tags; each block +/// still publishes exactly once. +#[tokio::test] +async fn chunked_decode_publishes_each_block_once() { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 320, 240) + .cluster(0, || { + vec![ + simple_block(1, 0, true, b"k0"), + simple_block(1, 33, false, b"p1"), + simple_block(1, 66, false, b"p2"), + ] + }) + .cluster(100, || { + vec![simple_block(1, 0, true, b"k1"), simple_block(1, 33, false, b"p3")] + }) + .segment_end() + .build(); + + let import = import_chunked(&data, 16).await; + assert!(import.refused.is_none()); + let video = import.published.values().next().unwrap(); + assert_eq!(video, &[0, 33_000, 66_000, 100_000, 133_000]); +} + +/// A feed an hour into its own timeline publishes its block timestamps verbatim, and the +/// catalog clock maps its first block to the arrival time. +#[tokio::test] +async fn import_publishes_block_timestamps_on_an_arrival_clock() { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 16, 16) + .cluster(3_600_000, || { + vec![simple_block(1, 0, true, b"k0"), simple_block(1, 33, false, b"p1")] + }) + .segment_end() + .build(); + + let import = import_chunked(&data, data.len()).await; + assert!(import.refused.is_none()); + let first = import.published.values().next().unwrap()[0]; + assert_eq!(first, 3_600_000_000, "the source's own timestamp"); + + let tick = std::time::Duration::from_millis(1); + let wall = import + .clock + .wall_clock(moq_net::Timestamp::from_micros(first as u64).unwrap()) + .unwrap(); + assert!( + *import.arrival.start() - tick <= wall && wall <= *import.arrival.end() + tick, + "the first block is live on arrival" + ); +} + +/// A cluster that restarts the timeline is a new epoch: the import refuses the rewind rather +/// than dropping it, keeping everything published before it. +#[tokio::test] +async fn import_refuses_a_restart() { + let data = MkvBuilder::new() + .header("webm") + .segment_start() + .info(1_000_000) + .track_video(1, "V_VP9", 16, 16) + .cluster(5_000, || { + vec![simple_block(1, 0, true, b"k0"), simple_block(1, 33, false, b"p1")] + }) + .cluster(0, || vec![simple_block(1, 0, true, b"k1")]) + .segment_end() + .build(); + + let import = import_chunked(&data, data.len()).await; + let err = import.refused.expect("the restart is refused"); + assert!(matches!(err, crate::Error::TimestampRewind(_)), "{err:?}"); + assert_eq!(import.published.values().next().unwrap().len(), 2); +} diff --git a/rs/moq-mux/src/container/mod.rs b/rs/moq-mux/src/container/mod.rs index 006fafbafc..af7fed0436 100644 --- a/rs/moq-mux/src/container/mod.rs +++ b/rs/moq-mux/src/container/mod.rs @@ -121,9 +121,9 @@ pub struct MissingKeyframe; #[error("video group endpoint precedes its last frame")] pub struct InvalidEnd; -/// A frame's timestamp sits below the live edge earlier groups reached. +/// A group starts before the previous group did: the source restarted, which is a new broadcast. #[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] -#[error("frame timestamp is below the live edge")] +#[error("frame timestamp is below the previous group's start")] pub struct TimestampRewind; /// Encode and decode media frames over a moq-lite group. diff --git a/rs/moq-mux/src/container/producer.rs b/rs/moq-mux/src/container/producer.rs index abdd85fd72..0fa14fc670 100644 --- a/rs/moq-mux/src/container/producer.rs +++ b/rs/moq-mux/src/container/producer.rs @@ -68,9 +68,15 @@ pub struct Producer { /// for a duration marker when the caller does not pass a bound. end: Option, - /// Exclusive presentation end of finished groups. A frame below this is refused. + /// Exclusive presentation end of finished groups, where a payload with no timestamp of its + /// own lands. Not a floor: a group may start below it. live_edge: Option, + /// The keyframe timestamp that opened the latest group. The next group may not start below it. + start: Option, + /// The start of the group before the latest one. No frame of the latest group sits below it. + floor: Option, + /// Duration of the frame that last raised [`end`](Self::end), if it had one. Distinguishes /// an exclusive presentation point from a max timestamp that still needs an estimate. last_duration: Option, @@ -149,6 +155,8 @@ impl Producer { recorder: None, end: None, live_edge: None, + start: None, + floor: None, last_duration: None, previous_timestamp: None, cadence: None, @@ -183,6 +191,8 @@ where recorder: None, end: None, live_edge: None, + start: None, + floor: None, last_duration: None, previous_timestamp: None, cadence: None, @@ -377,21 +387,12 @@ where /// The exclusive presentation end earlier groups have reached, if any. /// - /// A later [`write`](Self::write) below this is refused. Trusted sources that must - /// re-anchor (a PES resync, a capture restart) clamp to it rather than rewind. + /// A source with no timestamp of its own (a section, a PES without PTS) lands on it. It is + /// not a floor: a keyframe may start below it (see [`write`](Self::write)). pub fn live_edge(&self) -> Option { self.live_edge } - /// The lowest timestamp the next [`write`](Self::write) accepts: the live edge, or for a - /// keyframe, the edge once the group it closes is counted too. - pub(crate) fn floor(&self, keyframe: bool) -> Option { - match (self.live_edge, self.end.filter(|_| keyframe)) { - (Some(edge), Some(end)) if timestamp_lt(edge, end) => Some(end), - (edge, end) => edge.or(end), - } - } - /// Write a frame to the track. /// /// A keyframe closes any open group and starts a new one. A non-keyframe extends the current @@ -400,28 +401,33 @@ where /// independently decodable (audio) marks only the first frame of each group a keyframe (see /// [`needs_keyframe`](Self::needs_keyframe)) so the group spans more than one frame. /// - /// A timestamp below the live edge earlier groups reached returns - /// [`TimestampRewind`](super::TimestampRewind) without writing, the way an oversized - /// frame is refused. B-frames and open-GOP leading pictures still qualify when they sit - /// above that edge. + /// Group starts never go backwards: a keyframe below the start of the previous group, or any + /// frame below the start of the group before its own, returns + /// [`TimestampRewind`](super::TimestampRewind) without writing, the way an oversized frame is + /// refused. That is a restart, which is a new broadcast. Frames may still dip below the + /// previous group's content: B-frames, open-GOP leading pictures, and a keyframe that + /// overlaps the previous group's last frame. pub fn write(&mut self, frame: Frame) -> crate::Result<()> { - // A keyframe cuts the previous group, using its timestamp as the boundary - // where the previous group's content ends. Cut first so this group's live - // edge includes what we just closed, then refuse a rewind against that. + let floor = match frame.keyframe { + true => self.start, + false => self.floor, + }; + if floor.is_some_and(|floor| timestamp_lt(frame.timestamp, floor)) { + return Err(super::TimestampRewind.into()); + } + + // A keyframe cuts the previous group, using its timestamp as the boundary where the + // previous group's content ends, unless it overlaps that content. if frame.keyframe { - let rewound = self + let overlaps = self .previous_timestamp .is_some_and(|previous| timestamp_lt(frame.timestamp, previous)); - self.cut((!rewound).then_some(frame.timestamp))?; - if rewound { + self.cut((!overlaps).then_some(frame.timestamp))?; + if overlaps { self.cadence = None; } } - if self.live_edge.is_some_and(|edge| timestamp_lt(frame.timestamp, edge)) { - return Err(super::TimestampRewind.into()); - } - // Start a new group if needed; the first frame of a group must be a keyframe. if self.group.is_none() { if !frame.keyframe { @@ -441,6 +447,8 @@ where recorder.record(group.sequence, frame.timestamp, frame.keyframe); } + self.floor = self.start; + self.start = Some(frame.timestamp); self.group = Some(group); } @@ -605,7 +613,7 @@ where /// That object exists on moq-transport, so the live edge moves immediately; a consumer /// MUST NOT submit it to a decoder. Data tracks skip a sequence with no object, because /// an empty payload is data. The next [`write`](Self::write) opens the group after the - /// marker and must continue forward from the live edge; it cannot rewind. + /// marker and may not start below the last media group's start. /// /// To bound the closing group's final frame, [`cut(end)`](Self::cut) before calling this. /// Otherwise the open group closes without a duration marker: what resumes may land sooner @@ -1442,14 +1450,41 @@ mod tests { assert!(matches!(err, crate::Error::TimestampRewind(_))); } + /// #3798's join: an IDR one frame below the previous group's last frame, but above its + /// start, overlaps that group rather than rewinding it. #[tokio::test] - async fn a_group_below_the_live_edge_is_refused() { + async fn a_keyframe_overlapping_the_previous_group_publishes() { let track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + let consumer = track.subscribe(replay()); let mut producer = Producer::new(track, Container::Legacy(crate::container::Kind::Video)); - producer.write(frame(0, true)).unwrap(); - producer.write(frame(33_000, false)).unwrap(); - producer.cut(None).unwrap(); - let err = producer.write(frame(16_000, true)).unwrap_err(); + producer.write(frame(100_000, true)).unwrap(); + producer.write(frame(133_000, false)).unwrap(); + producer.write(frame(166_000, false)).unwrap(); + producer.write(frame(150_000, true)).unwrap(); + producer.write(frame(183_000, false)).unwrap(); + producer.finish().unwrap(); + let groups = collect_payloads(consumer).await; + assert_eq!(groups[1][..2], [(150_000, 2), (183_000, 2)]); + } + + /// A group may start exactly where the previous one did: only a start below it is a rewind. + #[tokio::test] + async fn a_keyframe_at_the_previous_group_start_is_accepted() { + let track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.audio)); + let mut producer = Producer::new(track, Container::Legacy(crate::container::Kind::Audio)); + producer.write(frame(100_000, true)).unwrap(); + producer.write(frame(100_000, true)).unwrap(); + } + + /// A frame below the start of the group before its own reaches back past a whole group. + #[tokio::test] + async fn a_frame_below_the_previous_group_start_is_refused() { + let track = track_producer("test", hang::container::track_info(hang::catalog::PRIORITY.video)); + let mut producer = Producer::new(track, Container::Legacy(crate::container::Kind::Video)); + producer.write(frame(100_000, true)).unwrap(); + producer.write(frame(133_000, false)).unwrap(); + producer.write(frame(200_000, true)).unwrap(); + let err = producer.write(frame(90_000, false)).unwrap_err(); assert!(matches!(err, crate::Error::TimestampRewind(_))); } diff --git a/rs/moq-mux/src/container/test_util.rs b/rs/moq-mux/src/container/test_util.rs index 50d5f4de97..3940408fe0 100644 --- a/rs/moq-mux/src/container/test_util.rs +++ b/rs/moq-mux/src/container/test_util.rs @@ -129,11 +129,6 @@ pub(crate) fn raw_frame(timestamp_us: u64, payload: &'static [u8], keyframe: boo } } -/// A broadcast clock that began `ago` before now, so a first frame arriving now reads as late. -pub(crate) fn late_clock(ago: std::time::Duration) -> crate::Clock { - crate::Clock::at(std::time::Instant::now() - ago, std::time::SystemTime::now() - ago).unwrap() -} - /// Every frame timestamp, in micros, that each media rendition in `catalog` published, by track. /// /// Reads through each rendition's own container, so an fMP4 timestamp comes from the fragment's @@ -168,9 +163,9 @@ pub(crate) async fn published( out } -/// The one offset every published timestamp moved by between a verbatim and a live import of the -/// same input, within a tick of rounding: one mapping for every track, so A/V sync and B-frame -/// order survive exactly. +/// The one offset every published timestamp moved by between two imports of the same media, +/// within a tick of rounding: one mapping for every track, so A/V sync and B-frame order survive +/// exactly. pub(crate) fn common_offset( verbatim: &std::collections::BTreeMap>, live: &std::collections::BTreeMap>, diff --git a/rs/moq-mux/src/container/ts/import.rs b/rs/moq-mux/src/container/ts/import.rs index 0cfa35a533..3965a5cd50 100644 --- a/rs/moq-mux/src/container/ts/import.rs +++ b/rs/moq-mux/src/container/ts/import.rs @@ -128,15 +128,11 @@ pub struct Import { /// the PAT into the catalog service record yet (set once; the PAT is stable). identity_recorded: bool, /// Latest video PTS: the media clock used to timestamp private sections, which - /// carry no PES PTS of their own. Unwrapped independently of the video stream, and - /// shifted with it, so a wrap the video re-anchors moves the cues along with it. + /// carry no PES PTS of their own. Unwrapped independently of the video stream. /// SPTS scope: one clock for the whole input. Under MPTS every program's video /// advances it, so a cue could be stamped with another program's PTS. last_pts: Option, media_unwrap: PtsUnwrap, - /// The source's mapping onto the broadcast clock, set by [`live`](Self::live). `None` - /// publishes the source's unwrapped PTS verbatim. - anchor: Option, } impl Import { @@ -179,22 +175,9 @@ impl Import { identity_recorded: false, last_pts: None, media_unwrap: PtsUnwrap::default(), - anchor: None, } } - /// Publish on the broadcast clock rather than the source's own PTS. - /// - /// For a live feed with its own zero: the first frame is live on arrival, every elementary - /// stream shares that one mapping, and a restart (a PTS rewind or a signalled time-base - /// discontinuity) continues forward after the real idle gap. Without this, the unwrapped PTS - /// is published verbatim, which suits a source already on the clock the catalog advertises - /// ([`Config::with_clock`](crate::catalog::Config::with_clock)). - pub fn live(mut self) -> Self { - self.anchor = Some(crate::clock::Anchor::new(self.catalog.clock())); - self - } - /// Select the container this importer wraps decoded media renditions in. /// /// [`Legacy`](hang::catalog::Container::Legacy) unless selected. It applies to every rendition @@ -473,7 +456,6 @@ impl Import { split: h264::Split::new(), import: Box::new(h264::Import::new(track, self.reserve(), self.video_hint())?), unwrap: PtsUnwrap::default(), - reanchor: Reanchor::default(), } } StreamType::H265 => { @@ -484,7 +466,6 @@ impl Import { split: h265::Split::new(), import: Box::new(h265::Import::new(track, self.reserve(), self.video_hint())?), unwrap: PtsUnwrap::default(), - reanchor: Reanchor::default(), } } // Only ADTS-framed AAC (0x0F). 0x11 is LATM/LOAS, which uses a different @@ -495,7 +476,6 @@ impl Import { reserved: Some(self.reserve()), container: self.container.clone(), unwrap: PtsUnwrap::default(), - reanchor: Reanchor::default(), tail: Vec::new(), tail_pts: None, resync: Resync::new(pid.as_u16(), ".aac"), @@ -523,7 +503,6 @@ impl Import { Stream::Opus(Box::new(OpusStream { import: opus::Import::new(track, self.reserve(), config)?, unwrap: PtsUnwrap::default(), - reanchor: Reanchor::default(), })) } StreamType::Mpeg1Video | StreamType::Mpeg2Video => Stream::Clock, @@ -574,7 +553,6 @@ impl Import { tail: Vec::new(), tail_pts: None, resync: Resync::new(pid.as_u16(), descriptor.track_suffix), - reanchor: Reanchor::default(), })) } @@ -662,37 +640,16 @@ impl Import { // frame must be timestamped with this frame's PTS ("now"), not the // previous one's. if pes.header.pts.is_some() { - let pts = unwrap_pts( - &mut self.media_unwrap, - pes.header.pts.map(|t| t.as_u64()), - self.anchor.as_mut(), - )?; - let video = match self.streams.get(&pid) { - Some(Stream::H264 { reanchor, import, .. }) => { - Some((reanchor, import.floor(false), import.floor(true))) - } - Some(Stream::H265 { reanchor, import, .. }) => { - Some((reanchor, import.floor(false), import.floor(true))) - } - _ => None, - }; - self.last_pts = match (pts, video) { - (Some(pts), Some((reanchor, edge, floor))) => { - let pts = reanchor.shifted(pts)?; - // Below the live edge is a wrap or restart, which only a keyframe re-anchored to - // the floor continues. That write waits for this PES to end, so stamp sections - // arriving meanwhile there too rather than on the old timeline. - match floor { - Some(floor) if edge.is_some_and(|edge| pts.as_micros() < edge.as_micros()) => Some(floor), - _ => Some(pts), - } - } - (pts, _) => pts, - }; + self.last_pts = unwrap_pts(&mut self.media_unwrap, pes.header.pts.map(|t| t.as_u64()))?; } } if is_clock { + // A clock-only stream never flushes, but sections are stamped with its PTS, so it + // anchors the catalog here like a flushed PES would. + if let Some(pts) = pes.header.pts { + self.catalog.anchor(Timestamp::from_scale(pts.as_u64(), 90_000)?)?; + } return Ok(()); } @@ -737,7 +694,11 @@ impl Import { let Some(stream) = self.streams.get_mut(&pid) else { return Ok(()); }; - self.published |= stream.write(pending, batched, self.anchor.as_mut())?; + // The first PES is live on arrival: its PTS needs no unwrap yet. + if let Some(pts) = pending.pts { + self.catalog.anchor(Timestamp::from_scale(pts, 90_000)?)?; + } + self.published |= stream.write(pending, batched)?; // Record the decoded media track's PID + PMT descriptors (language, ...) once // its lazily created track exists, so export can preserve them. @@ -1152,8 +1113,7 @@ impl SectionStream { /// Publish one complete section as a frame in its own group. /// - /// The clock is already on the video's re-anchored timeline, so it only steps back by a - /// reorder: a B-frame presents before the P-frame decoded ahead of it. Such a cue lands on + /// The clock follows the video's PTS, so it only steps back by a reorder: a B-frame presents before the P-frame decoded ahead of it. Such a cue lands on /// the edge rather than shifting every cue after it, which would drift the cue track off /// the video by the reorder depth at each one. fn emit(&mut self, section: Vec, pts: Option) -> anyhow::Result<()> { @@ -1203,7 +1163,6 @@ struct VerbatimStream { track: crate::container::Producer, entry: VerbatimEntry, unwrap: PtsUnwrap, - reanchor: Reanchor, /// Whether the PES stream_id has been recorded into the catalog yet (once). stream_id_recorded: bool, } @@ -1232,14 +1191,13 @@ impl VerbatimStream { track, entry, unwrap: PtsUnwrap::default(), - reanchor: Reanchor::default(), stream_id_recorded: false, }) } /// Publish one reassembled PES payload verbatim, in its own group, stamped with /// its PTS (or the live edge when the PES carried none). - fn write(&mut self, pending: Pending, anchor: Option<&mut crate::clock::Anchor>) -> anyhow::Result { + fn write(&mut self, pending: Pending) -> anyhow::Result { // Record the original PES stream_id once, from the first PES, so export // re-emits the stream under its real id (e.g. 0xBD for teletext/DVB AC-3). if !self.stream_id_recorded { @@ -1252,11 +1210,10 @@ impl VerbatimStream { self.stream_id_recorded = true; } - let edge = self.track.live_edge(); - let pts = match unwrap_pts(&mut self.unwrap, pending.pts, anchor)? { - Some(pts) => self.reanchor.apply(pts, edge)?, - // No clock to shift, so land on the edge and leave the shift alone. - None => edge.unwrap_or(Timestamp::ZERO), + let pts = match unwrap_pts(&mut self.unwrap, pending.pts)? { + Some(pts) => pts, + // No clock of its own, so land on the edge. + None => self.track.live_edge().unwrap_or(Timestamp::ZERO), }; let frame = crate::container::Frame { timestamp: pts, @@ -1271,7 +1228,6 @@ impl VerbatimStream { fn discontinuity(&mut self, published: bool) -> anyhow::Result<()> { self.unwrap.discontinuity(); - self.reanchor.discontinuity(); if published { self.track.discontinuity()?; } @@ -1547,13 +1503,11 @@ enum Stream { split: h264::Split, import: Box, unwrap: PtsUnwrap, - reanchor: Reanchor, }, H265 { split: h265::Split, import: Box, unwrap: PtsUnwrap, - reanchor: Reanchor, }, Aac(Box>), Opus(Box), @@ -1567,27 +1521,16 @@ enum Stream { } impl Stream { - fn write( - &mut self, - pending: Pending, - batched: bool, - anchor: Option<&mut crate::clock::Anchor>, - ) -> anyhow::Result { + fn write(&mut self, pending: Pending, batched: bool) -> anyhow::Result { match self { - Stream::H264 { - split, - import, - unwrap, - reanchor, - } => { + Stream::H264 { split, import, unwrap } => { let reorder = reorder_delay(pending.pts, pending.dts); - let pts = unwrap_pts(unwrap, pending.pts, anchor)?; + let pts = unwrap_pts(unwrap, pending.pts)?; // Each PES is one access unit, so flush to emit it immediately. let mut frames = split.decode(&pending.data, pts)?; frames.extend(split.flush(pts)?); let mut published = false; - for mut frame in frames { - frame.timestamp = reanchor.apply(frame.timestamp, import.floor(frame.keyframe))?; + for frame in frames { published |= skip_missing_keyframe(import.decode([frame]))?; } // After decode, so the track (and its catalog rendition) exists. @@ -1596,20 +1539,14 @@ impl Stream { } Ok(published) } - Stream::H265 { - split, - import, - unwrap, - reanchor, - } => { + Stream::H265 { split, import, unwrap } => { let reorder = reorder_delay(pending.pts, pending.dts); - let pts = unwrap_pts(unwrap, pending.pts, anchor)?; + let pts = unwrap_pts(unwrap, pending.pts)?; // Each PES is one access unit, so flush to emit it immediately. let mut frames = split.decode(&pending.data, pts)?; frames.extend(split.flush(pts)?); let mut published = false; - for mut frame in frames { - frame.timestamp = reanchor.apply(frame.timestamp, import.floor(frame.keyframe))?; + for frame in frames { published |= skip_missing_keyframe(import.decode([frame]))?; } if let Some(reorder) = reorder { @@ -1617,10 +1554,10 @@ impl Stream { } Ok(published) } - Stream::Aac(stream) => stream.write(pending, batched, anchor), - Stream::Opus(stream) => stream.write(pending, anchor), - Stream::Legacy(stream) => stream.write(pending, anchor), - Stream::Verbatim(stream) => stream.write(pending, anchor), + Stream::Aac(stream) => stream.write(pending, batched), + Stream::Opus(stream) => stream.write(pending), + Stream::Legacy(stream) => stream.write(pending), + Stream::Verbatim(stream) => stream.write(pending), Stream::Clock | Stream::Ignored => Ok(false), } } @@ -1675,27 +1612,15 @@ impl Stream { /// next PTS against a sample from the timebase that just ended. fn discontinuity(&mut self, published: bool) -> anyhow::Result<()> { match self { - Stream::H264 { - import, - unwrap, - reanchor, - .. - } => { + Stream::H264 { import, unwrap, .. } => { unwrap.discontinuity(); - reanchor.discontinuity(); if published { import.discontinuity()?; } Ok(()) } - Stream::H265 { - import, - unwrap, - reanchor, - .. - } => { + Stream::H265 { import, unwrap, .. } => { unwrap.discontinuity(); - reanchor.discontinuity(); if published { import.discontinuity()?; } @@ -2017,7 +1942,6 @@ struct AacStream { /// The container this importer publishes decoded renditions with. container: hang::catalog::Container, unwrap: PtsUnwrap, - reanchor: Reanchor, /// Partial frame left at the end of the previous PES. ISO 13818-1 doesn't require /// audio frames to align with PES boundaries, so a legitimate mux can split one. tail: Vec, @@ -2030,13 +1954,8 @@ struct AacStream { } impl AacStream { - fn write( - &mut self, - pending: Pending, - batched: bool, - anchor: Option<&mut crate::clock::Anchor>, - ) -> anyhow::Result { - let pes_base = unwrap_pts(&mut self.unwrap, pending.pts, anchor)?; + fn write(&mut self, pending: Pending, batched: bool) -> anyhow::Result { + let pes_base = unwrap_pts(&mut self.unwrap, pending.pts)?; // Prepend the partial frame left by the previous PES, if any. let carried = self.tail.len(); @@ -2052,11 +1971,7 @@ impl AacStream { // PTS for the next frame to emit. The tail frame keeps the PTS computed at its cut; // the first frame that BEGINS in this PES takes the PES PTS (per ISO 13818-1). - let mut pts = if carried > 0 { - self.tail_pts.take() - } else { - self.reanchor(pes_base)? - }; + let mut pts = if carried > 0 { self.tail_pts.take() } else { pes_base }; let mut in_tail = carried > 0; // A single PES can carry several ADTS frames; split and feed each raw frame. @@ -2067,7 +1982,7 @@ impl AacStream { let mut fallback = None; while offset + adts::MIN_HEADER_LEN <= data.len() { if in_tail && offset >= carried { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } @@ -2127,12 +2042,12 @@ impl AacStream { None => anyhow::Error::msg(context), }); } - // Re-anchor only where the tail is actually abandoned: restoring a fallback + // Take the PES PTS only where the tail is actually abandoned: restoring a fallback // keeps the candidate, so it keeps the timestamp it was found with. match self.resync.recover(data, offset, &SyncWord::ADTS) { Recover::At(next) => { if in_tail { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } offset = next; @@ -2150,7 +2065,7 @@ impl AacStream { } None => { if in_tail { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } offset = next; @@ -2209,7 +2124,7 @@ impl AacStream { // with the PTS it should carry. if offset < data.len() { if in_tail && offset >= carried { - pts = self.reanchor(pes_base)?; + pts = pes_base; } self.tail = data[offset..].to_vec(); self.tail_pts = pts; @@ -2218,17 +2133,9 @@ impl AacStream { Ok(!burst.is_zero()) } - /// The PES PTS on the published timeline. Resolved where each frame starts rather than - /// once per PES, so the frames a carried tail publishes first count toward the edge. - fn reanchor(&mut self, pts: Option) -> anyhow::Result> { - let edge = self.import.as_ref().and_then(aac::Import::live_edge); - pts.map(|pts| self.reanchor.apply(pts, edge)).transpose() - } - fn discontinuity(&mut self, published: bool) -> anyhow::Result<()> { self.desync(); self.unwrap.discontinuity(); - self.reanchor.discontinuity(); if published && let Some(import) = &mut self.import { import.discontinuity()?; } @@ -2263,7 +2170,7 @@ impl AacStream { if !self.tail.is_empty() && self.import.is_some() { self.resync.drain(); // No PTS to translate, so no mapping needed. - self.write(Pending::empty(), true, None)?; + self.write(Pending::empty(), true)?; } // A partial frame at end of stream isn't emissible; drop it, but leave a trace for // diagnosing truncated captures. @@ -2295,14 +2202,11 @@ impl AacStream { struct OpusStream { import: opus::Import, unwrap: PtsUnwrap, - reanchor: Reanchor, } impl OpusStream { - fn write(&mut self, pending: Pending, anchor: Option<&mut crate::clock::Anchor>) -> anyhow::Result { - let base = unwrap_pts(&mut self.unwrap, pending.pts, anchor)?; - let edge = self.import.live_edge(); - let base = base.map(|base| self.reanchor.apply(base, edge)).transpose()?; + fn write(&mut self, pending: Pending) -> anyhow::Result { + let base = unwrap_pts(&mut self.unwrap, pending.pts)?; let data = &pending.data; let mut offset = 0; @@ -2339,7 +2243,6 @@ impl OpusStream { fn discontinuity(&mut self, published: bool) -> anyhow::Result<()> { self.unwrap.discontinuity(); - self.reanchor.discontinuity(); if published { self.import.discontinuity()?; } @@ -2451,13 +2354,12 @@ struct LegacyStream { /// only covers frames that begin in that PES. tail_pts: Option, resync: Resync, - reanchor: Reanchor, } impl LegacyStream { - fn write(&mut self, pending: Pending, anchor: Option<&mut crate::clock::Anchor>) -> anyhow::Result { + fn write(&mut self, pending: Pending) -> anyhow::Result { let mut published = false; - let pes_base = unwrap_pts(&mut self.unwrap, pending.pts, anchor)?; + let pes_base = unwrap_pts(&mut self.unwrap, pending.pts)?; // Prepend the partial frame left by the previous PES, if any. let carried = self.tail.len(); @@ -2476,11 +2378,7 @@ impl LegacyStream { // ISO 13818-1, a PES PTS refers to the first access unit starting in it). // After each frame it advances by that frame's duration (per frame, not // `index * constant`: E-AC-3 varies the samples per frame). - let mut pts = if carried > 0 { - self.tail_pts.take() - } else { - self.reanchor(pes_base)? - }; + let mut pts = if carried > 0 { self.tail_pts.take() } else { pes_base }; let mut in_tail = carried > 0; let mut offset = 0; @@ -2489,7 +2387,7 @@ impl LegacyStream { let mut fallback = None; while offset + self.descriptor.min_header_len <= data.len() { if in_tail && offset >= carried { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } @@ -2566,7 +2464,7 @@ impl LegacyStream { match self.resync.recover(data, offset, &self.descriptor.into()) { Recover::At(next) => { if in_tail { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } offset = next; @@ -2584,7 +2482,7 @@ impl LegacyStream { } None => { if in_tail { - pts = self.reanchor(pes_base)?; + pts = pes_base; in_tail = false; } offset = next; @@ -2633,7 +2531,7 @@ impl LegacyStream { // PES, with the PTS it should carry. if offset < data.len() { if in_tail && offset >= carried { - pts = self.reanchor(pes_base)?; + pts = pes_base; } self.tail = data[offset..].to_vec(); self.tail_pts = pts; @@ -2642,17 +2540,9 @@ impl LegacyStream { Ok(published) } - /// The PES PTS on the published timeline. Resolved where each frame starts rather than - /// once per PES, so the frames a carried tail publishes first count toward the edge. - fn reanchor(&mut self, pts: Option) -> anyhow::Result> { - let edge = self.import.as_ref().and_then(legacy::Import::live_edge); - pts.map(|pts| self.reanchor.apply(pts, edge)).transpose() - } - fn discontinuity(&mut self, published: bool) -> anyhow::Result<()> { self.desync(); self.unwrap.discontinuity(); - self.reanchor.discontinuity(); if published && let Some(import) = &mut self.import { import.discontinuity()?; } @@ -2686,7 +2576,7 @@ impl LegacyStream { if !self.tail.is_empty() && self.import.is_some() { self.resync.drain(); // No PTS to translate, so no mapping needed. - self.write(Pending::empty(), None)?; + self.write(Pending::empty())?; } // A partial frame at end of stream isn't emissible verbatim; drop it, but // leave a trace for diagnosing truncated captures. @@ -2754,19 +2644,11 @@ fn advance_pts(pts: Option, samples: u64, sample_rate: u32) -> anyhow /// Convert a raw 90 kHz PTS to a microsecond [`Timestamp`], unwrapping the /// 33-bit field. Returns `None` when the PES carried no PTS. -fn unwrap_pts( - unwrap: &mut PtsUnwrap, - pts: Option, - anchor: Option<&mut crate::clock::Anchor>, -) -> anyhow::Result> { +fn unwrap_pts(unwrap: &mut PtsUnwrap, pts: Option) -> anyhow::Result> { let Some(raw) = pts else { return Ok(None); }; - let extended = Timestamp::from_scale(unwrap.unwrap(raw), 90_000)?; - Ok(Some(match anchor { - Some(anchor) => anchor.translate(&mut unwrap.lane, extended)?, - None => extended, - })) + Ok(Some(Timestamp::from_scale(unwrap.unwrap(raw), 90_000)?)) } /// The reorder delay `PTS - DTS` for one PES, as a microsecond [`Timestamp`]. `None` unless @@ -2791,8 +2673,6 @@ fn reorder_delay(pts: Option, dts: Option) -> Option { struct PtsUnwrap { last: Option, offset: u64, - /// This stream's position on the source's broadcast-clock mapping, when publishing live. - lane: crate::clock::Lane, } impl PtsUnwrap { @@ -2816,54 +2696,6 @@ impl PtsUnwrap { /// whatever the source does with its clock next. fn discontinuity(&mut self) { self.last = None; - self.lane.restart(); - } -} - -/// Keeps one track's timeline at or past its live edge when the source clock steps back. -/// -/// A content join, a playout loop wrap, or a restarted encoder lands a timestamp below what -/// the track already published, and the producer refuses a rewind. TS ingest trusts its -/// source, so it shifts forward instead, by just enough to reach the edge. The shift only -/// grows: a looping source wraps again every period, and each wrap lands below the edge the -/// last one reached. A timebase break clears it, since the new clock is measured against the -/// edge afresh. -#[derive(Default)] -struct Reanchor { - shift: Option, -} - -impl Reanchor { - /// `pts` on the published timeline, as shifted so far. - fn shifted(&self, pts: Timestamp) -> anyhow::Result { - Ok(match self.shift { - Some(shift) => pts.checked_add(shift.convert(pts.scale())?)?, - None => pts, - }) - } - - /// `pts` on the published timeline, at or past `edge`. - fn apply(&mut self, pts: Timestamp, edge: Option) -> anyhow::Result { - let scale = pts.scale(); - let pts = self.shifted(pts)?; - let Some(edge) = edge.filter(|edge| pts.as_micros() < edge.as_micros()) else { - return Ok(pts); - }; - // Round up to the source's scale: landing a tick short of the edge is still a rewind. - let mut target = Timestamp::new(u64::try_from(edge.as_scale(scale))?, scale)?; - if target.as_micros() < edge.as_micros() { - target = Timestamp::new(target.value() + 1, scale)?; - } - let grow = target.checked_sub(pts)?; - self.shift = Some(match self.shift { - Some(shift) => shift.convert(scale)?.checked_add(grow)?, - None => grow, - }); - Ok(target) - } - - fn discontinuity(&mut self) { - self.shift = None; } } @@ -3413,6 +3245,31 @@ mod test { ); } + // MPEG-2 video only drives the section clock, so with nothing else to flush it must still + // anchor the catalog, or cues stamped with its PTS map to the construction-time clock. + #[test] + fn a_clock_only_stream_anchors_the_catalog() { + const VIDEO_PID: u16 = 0x0050; + const PTS_SECS: u64 = 3600; + + let mut broadcast = moq_net::broadcast::Info::new().produce(); + let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); + let mut import = super::Import::new(broadcast, catalog.reserve()); + + import + .decode(&synth_pmt(&[(StreamType::Mpeg2Video, VIDEO_PID)], true)) + .unwrap(); + import + .decode(pes_packet(VIDEO_PID, PTS_SECS * 90_000).as_slice()) + .unwrap(); + + let now = catalog.clock().now().as_micros() / 1_000_000; + assert!( + (PTS_SECS as u128..PTS_SECS as u128 + 5).contains(&now), + "catalog clock reads {now}s, not the first video PTS" + ); + } + /// A PUSI TS packet on `pid`: a bounded audio PES (stream_id 0xC0) carrying /// `payload` (whole codec frames or a fragment of one), sized exactly via /// adaptation-field stuffing so the PES completes (and flushes) on this packet. @@ -3748,18 +3605,14 @@ mod test { frames } - // The production shape from #2729, on a real capture: a looping publisher plays part of - // a file and wraps to the top, so the audio PES open at the cut leaves a tail that - // splices onto the file's first frames. Every wrap used to end the session with - // "missing AC-3 sync word", which is what accumulated 216 restarts on the reporter's - // feed. AC-3 also exercises a different sync word and parser than the synthetic MP2 - // and ADTS tests above. + // A looping publisher plays part of a file and wraps to the top (#2729). The wrap rewinds + // the audio PTS, which is an encoder restart, so the import ends and the caller publishes + // a new broadcast. Every frame published before it is still a whole AC-3 frame. #[tokio::test(start_paused = true)] - async fn legacy_survives_a_looping_file_wrap() { + async fn legacy_refuses_a_looping_file_wrap() { let data = include_bytes!("test_data/ac3.ts"); - // Wrap mid-file on a packet boundary, so the TS layer stays aligned and the - // discontinuity lands squarely on the audio frame the last PES left open. + // Wrap mid-file on a packet boundary, so the TS layer stays aligned. let cut = (data.len() / 2) / 188 * 188; let mut looped = data[..cut].to_vec(); looped.extend_from_slice(data); @@ -3768,43 +3621,16 @@ mod test { let consumer = broadcast.consume(); let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); let mut import = super::Import::new(broadcast, catalog.reserve()); - import + let err = import .decode(&bytes::BytesMut::from(&looped[..])) - .expect("a loop wrap must not end the session"); - import.finish().unwrap(); - let looped_stats = import.stats(); + .expect_err("a loop wrap is a restart"); + assert!(is_rewind(&err), "{err:?}"); let frames = read_audio_frames(&consumer, &catalog).await; - // Every frame published is still a whole AC-3 frame: resync lands on sync words, - // never mid-frame. + assert!(!frames.is_empty(), "the pass before the wrap published"); assert!( frames.iter().all(|f| f.payload[0] == 0x0B && f.payload[1] == 0x77), - "resync published a frame that doesn't start at an AC-3 sync word" - ); - // The wrap costs at most the frame it interrupted, not the rest of the stream. - let pristine = { - let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); - let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); - let mut import = super::Import::new(broadcast, catalog.reserve()); - import.decode(&bytes::BytesMut::from(&data[..])).unwrap(); - import.finish().unwrap(); - read_audio_frames(&consumer, &catalog).await.len() - }; - // The wrap costs nothing after it: the whole second copy is published, on top of - // the frames the truncated first pass already delivered. - assert!( - frames.len() > pristine, - "the wrap swallowed frames: {} looped vs {pristine} pristine", - frames.len() - ); - // Nothing to report on this shape: the continuity counter catches the wrap a packet - // before the codec would see spliced bytes, so the tail is dropped rather than - // scanned past, and the next PES starts a frame the one after it confirms. The - // counters exist for the shapes that don't get caught that early. - assert!( - looped_stats.is_empty(), - "a wrap the continuity check absorbs should cost no frame sync: {looped_stats:?}" + "published a frame that doesn't start at an AC-3 sync word" ); } @@ -4330,15 +4156,14 @@ mod test { ); } - // #3533: a content join on a continuous transport timeline can land a PES PTS a few - // milliseconds below the extrapolated high-water mark. Re-anchor forward from the live - // edge instead of refusing the frame and aborting the import. + // #3533: a content join landed a PES PTS a millisecond below the last frame published. Each + // audio frame is its own group, so that group starts before the previous one: a restart, + // which ends the import rather than shifting it forward. #[tokio::test(start_paused = true)] - async fn legacy_resync_below_the_live_edge_publishes_forward() { + async fn legacy_pes_below_the_live_edge_is_refused() { const MP2_PID: u16 = 0x0061; let mut broadcast = moq_net::broadcast::Info::new().produce(); - let consumer = broadcast.consume(); let catalog = crate::catalog::Producer::new(&mut broadcast, crate::catalog::Config::default()).unwrap(); let mut import = super::Import::new(broadcast, catalog.reserve()); @@ -4355,26 +4180,11 @@ mod test { // A join a millisecond behind the live edge. let mut join = mp2_frame(0xDD); join.extend_from_slice(&mp2_frame(0xEE)); - import + let err = import .decode(audio_pes_packet(MP2_PID, 1, 89_910, &join).as_slice()) - .expect("a PES PTS below the live edge must not abort the import"); - import.finish().unwrap(); - - let frames = read_audio_frames(&consumer, &catalog).await; - assert!( - frames.len() >= 3, - "expected published frames after the join, got {}", - frames.len() - ); - let mut last = 0; - for frame in &frames { - assert!( - frame.timestamp.as_micros() >= last, - "published a rewind: {last} then {}", - frame.timestamp.as_micros() - ); - last = frame.timestamp.as_micros(); - } + .and_then(|()| import.finish()) + .expect_err("a PES PTS below the live edge is a restart"); + assert!(is_rewind(&err), "{err:?}"); } // Confirming a joined frame means holding it when the buffer ends too soon after it to @@ -5332,12 +5142,12 @@ mod test { } } - /// A backwards restart is the same signal; only the timestamps differ. The unwrapper must - /// not read the step back as the 33-bit field wrapping, which would put the new timeline - /// 26 hours out. + /// A backwards restart is the same signal, but a rewind: a new broadcast, so the import + /// ends. The unwrapper must not read the step back as the 33-bit field wrapping, which + /// would put the new timeline 26 hours out and publish it. #[tokio::test(start_paused = true)] - async fn a_backwards_restart_keeps_its_own_timestamps() { - let (consumer, catalog, mut import) = two_stream_import(); + async fn a_flagged_backwards_restart_is_refused() { + let (_consumer, _catalog, mut import) = two_stream_import(); for pid in [PCR_PID, PEER_PID] { import @@ -5345,23 +5155,12 @@ mod test { .unwrap(); } import.decode(clock_break_packet(PCR_PID).as_slice()).unwrap(); - for pid in [PCR_PID, PEER_PID] { - import.decode(mp2_pes(pid, 1, 90_000, [0xCC, 0xDD]).as_slice()).unwrap(); - } - import.finish().unwrap(); - - for (frames, breaks) in read_all_breaks(&consumer, &catalog).await { - // Backwards timestamps are a break a consumer can also derive for itself, so the - // marker need not be the only thing counted here. The exporter compares the - // counter across frames, so however many land between two frames are one reset. - assert!(breaks >= 1, "the timebase break did not reach this track"); - let stamps: Vec = frames.iter().map(|f| f.timestamp.as_micros()).collect(); - assert_eq!(stamps[0], 45_000_000, "the old timeline"); - assert!( - stamps.last().is_some_and(|&last| last >= 45_000_000), - "the restarted clock rewound instead of continuing forward: {stamps:?}" - ); - } + let err = [PCR_PID, PEER_PID] + .into_iter() + .try_for_each(|pid| import.decode(mp2_pes(pid, 1, 90_000, [0xCC, 0xDD]).as_slice())) + .and_then(|()| import.finish()) + .expect_err("a restart below the live edge is a new broadcast"); + assert!(is_rewind(&err), "{err:?}"); } /// A 5-byte PES timestamp field with the 4-bit `prefix` (0b0010 PTS only, 0b0011 PTS of a @@ -5501,18 +5300,41 @@ mod test { use crate::container::ts::catalog::Ext; - /// Every frame lands at or past the end of the groups before it, which is what the - /// producer demands. A group's first frame closes the one before, so it clears that - /// group too; later frames may present below it (B-frames). + /// Whether `err` is a refused timestamp rewind: the source restarted, which is a new + /// broadcast rather than a continuation. + fn is_rewind(err: &anyhow::Error) -> bool { + err.chain().any(|cause| { + matches!( + cause.downcast_ref::(), + Some(crate::Error::TimestampRewind(_)) + ) + }) + } + + /// Import `data`, which must end in a refused rewind. + fn import_refused(data: &[u8]) { + let Err(err) = import_all(data) else { + panic!("the import accepted a rewind"); + }; + assert!(is_rewind(&err), "{err:?}"); + } + + /// Group starts never go backwards, and no frame sits below the start of the group before + /// its own, which is what the producer demands. Frames may still present below the previous + /// group's content (B-frames, an overlapping keyframe). fn assert_forward(frames: &[crate::container::Frame]) { - let (mut edge, mut open) = (0, 0); + let (mut start, mut floor) = (None, None); for frame in frames { let ts = frame.timestamp.as_micros(); if frame.keyframe { - edge = edge.max(open); + assert!( + start.is_none_or(|start| ts >= start), + "a group started at {ts} before {start:?}" + ); + floor = start; + start = Some(ts); } - assert!(ts >= edge, "rewound to {ts} below the edge {edge}"); - open = open.max(ts); + assert!(floor.is_none_or(|floor| ts >= floor), "rewound to {ts} below {floor:?}"); } } @@ -5525,9 +5347,10 @@ mod test { const VIDEO: u16 = 0x0050; /// #3798's join: a new IDR presents below the last P-frame on a continuous clock, since - /// the B-frames before it presented earlier than the P-frame decoded ahead of them. + /// the B-frames before it presented earlier than the P-frame decoded ahead of them. It + /// still starts after the previous group did, so it overlaps rather than rewinds. #[tokio::test(start_paused = true)] - async fn h264_join_below_a_p_frame_publishes_forward() { + async fn h264_join_below_a_p_frame_publishes() { let mut mux = Mux { out: synth_pmt(&[(StreamType::H264, VIDEO)], false), ..Default::default() @@ -5541,9 +5364,9 @@ mod test { } /// An encoder restart that declares the new clock on the PCR PID still restarts lower - /// than the live edge, which the break marker does not lower. + /// than the live edge: a new broadcast, not a continuation. #[tokio::test(start_paused = true)] - async fn h264_flagged_backward_restart_publishes_forward() { + async fn h264_flagged_backward_restart_is_refused() { let mut mux = Mux { out: synth_pmt(&[(StreamType::H264, VIDEO)], false), ..Default::default() @@ -5551,61 +5374,40 @@ mod test { mux.gops(VIDEO, 45 * 90_000, 2); mux.out.extend_from_slice(&clock_break_packet(VIDEO)); mux.gops(VIDEO, 90_000, 2); - let frames = video_frames(&mux.out).await; - assert_eq!(frames.len(), 16); - assert_forward(&frames); + import_refused(&mux.out); } /// The same restart unflagged, as a looping playout wraps to the top of its file. #[tokio::test(start_paused = true)] - async fn h264_unflagged_backward_restart_publishes_forward() { + async fn h264_unflagged_backward_restart_is_refused() { let mut mux = Mux { out: synth_pmt(&[(StreamType::H264, VIDEO)], false), ..Default::default() }; mux.gops(VIDEO, 45 * 90_000, 2); mux.gops(VIDEO, 90_000, 2); - let frames = video_frames(&mux.out).await; - assert_eq!(frames.len(), 16); - assert_forward(&frames); + import_refused(&mux.out); } - /// A loop wraps every period, so the shift that absorbed the first wrap leaves the second - /// one below the edge again unless it grows. + /// A flagged break that leaps forward is not a rewind: the new timeline publishes. #[tokio::test(start_paused = true)] - async fn h264_survives_two_loop_wraps() { + async fn h264_flagged_forward_restart_publishes() { let mut mux = Mux { out: synth_pmt(&[(StreamType::H264, VIDEO)], false), ..Default::default() }; - for _ in 0..3 { - mux.gops(VIDEO, 90_000, 2); - } + mux.gops(VIDEO, 90_000, 2); + mux.out.extend_from_slice(&clock_break_packet(VIDEO)); + // Inside the track's 30 s retention, so the reader still reaches the first pass. + mux.gops(VIDEO, 20 * 90_000, 2); let frames = video_frames(&mux.out).await; - assert_eq!(frames.len(), 24, "every pass publishes"); + assert_eq!(frames.len(), 16); assert_forward(&frames); - // A pass keeps its own spacing, and each wrap lands the same distance past the last: - // the second wrap grew the shift by one more pass rather than reusing the first. - let stamps: Vec = frames - .iter() - .filter(|f| f.keyframe) - .map(|f| f.timestamp.as_micros()) - .collect(); - let gop = 4 * 1_000_000 * FRAME as u128 / 90_000; - let steps: Vec = stamps.windows(2).map(|pair| pair[1] - pair[0]).collect(); - // Within a microsecond: the wire is in microseconds and the source in 90 kHz ticks. - let near = |a: u128, b: u128| a.abs_diff(b) <= 1; - assert!( - [steps[0], steps[2], steps[4]].iter().all(|&step| near(step, gop)), - "a pass was respaced: {stamps:?}" - ); - assert!(near(steps[1], steps[3]), "the wraps landed differently: {stamps:?}"); } - /// The legacy audio path re-anchored before, but only once: a second wrap landed below - /// the edge again. + /// Legacy audio wrapping to the top of a looping file is a restart. #[tokio::test(start_paused = true)] - async fn legacy_survives_two_loop_wraps() { + async fn legacy_loop_wrap_is_refused() { const MP2_PID: u16 = 0x0061; let mut mux = Mux { out: synth_pmt(&[(StreamType::Mpeg1Audio, MP2_PID)], false), @@ -5613,25 +5415,20 @@ mod test { }; // One PES is two 72 ms MP2 frames. const PES_TICKS: u64 = 2 * 72 * 90; - for _ in 0..3 { + for _ in 0..2 { for i in 0..4 { let cc = mux.cc(MP2_PID); mux.out .extend_from_slice(&mp2_pes(MP2_PID, cc, 90_000 + i * PES_TICKS, [0xAA, 0xBB])); } } - let (consumer, catalog, _import) = import_all(&mux.out).expect("a second wrap must not end the import"); - let name = catalog.snapshot().audio.renditions.keys().next().unwrap().clone(); - let frames = read_track(&consumer, &name, crate::container::Kind::Audio).await; - assert!(frames.len() >= 20, "every pass publishes: {}", frames.len()); - assert_forward(&frames); + import_refused(&mux.out); } - /// A looping file whose audio runs past the loop period, so every wrap restarts a frame - /// below the last frame published. (The edge is that frame's start: audio frames carry no - /// duration, so a wrap exactly one frame short lands on it and passes.) + /// A looping file whose audio runs past the loop period restarts a frame below the last + /// frame published: a restart like any other. #[tokio::test(start_paused = true)] - async fn aac_loop_overlapping_the_edge_publishes_forward() { + async fn aac_loop_overlapping_the_edge_is_refused() { const AAC_PID: u16 = 0x0060; // 1024 samples at 48 kHz, in 90 kHz ticks. const AAC_FRAME: u64 = 1024 * 90_000 / 48_000; @@ -5639,7 +5436,7 @@ mod test { out: synth_pmt(&[(StreamType::AdtsAac, AAC_PID)], false), ..Default::default() }; - for pass in 0..3u64 { + for pass in 0..2u64 { for i in 0..4u64 { let mut payload = adts_frame(17, 0xA0 | i as u8); payload.extend_from_slice(&adts_frame(17, 0xB0 | i as u8)); @@ -5648,32 +5445,23 @@ mod test { mux.out.extend_from_slice(&audio_pes_packet(AAC_PID, cc, pts, &payload)); } } - let (consumer, catalog, _import) = import_all(&mux.out).expect("an overlapping wrap must not end the import"); - let name = catalog.snapshot().audio.renditions.keys().next().unwrap().clone(); - let frames = read_track(&consumer, &name, crate::container::Kind::Audio).await; - assert!(frames.len() >= 20, "every pass publishes: {}", frames.len()); - assert_forward(&frames); + import_refused(&mux.out); } - /// A PES-framed stream we carry verbatim steps back like any other. + /// A PES-framed stream we carry verbatim refuses a step back like any other. #[tokio::test(start_paused = true)] - async fn verbatim_pes_below_the_edge_publishes_forward() { + async fn verbatim_pes_below_the_edge_is_refused() { const DATA_PID: u16 = 0x0052; let mut mux = Mux { out: synth_pmt(&[(StreamType::Mpeg2PacketizedData, DATA_PID)], false), ..Default::default() }; - for pts in [45 * 90_000, 46 * 90_000, 90_000, 2 * 90_000, 90_000] { + for pts in [45 * 90_000, 46 * 90_000, 90_000] { let cc = mux.cc(DATA_PID); mux.out .extend_from_slice(&audio_pes_packet(DATA_PID, cc, pts, &[0xDE, 0xAD])); } - let (consumer, catalog, _import) = - import_all(&mux.out).expect("a verbatim PES below the edge must not end the import"); - let name = catalog.snapshot().ext.mpegts.tracks.keys().next().unwrap().clone(); - let frames = read_track(&consumer, &name, crate::container::Kind::Data).await; - assert_eq!(frames.len(), 5); - assert_forward(&frames); + import_refused(&mux.out); } /// A cue takes its time from the video it arrives with, and a B-frame decoded after a @@ -5691,12 +5479,13 @@ mod test { ), ..Default::default() }; - // Twice, the second an unflagged wrap to the top. - for _ in 0..2 { + // Two GOPs back to back. + for gop in 0..2 { + let base = 90_000 + gop * 4 * FRAME; for (keyframe, pts, dts) in [ - (true, 90_000 + FRAME, Some(90_000)), - (false, 90_000 + 4 * FRAME, Some(90_000 + FRAME)), - (false, 90_000 + 2 * FRAME, None), + (true, base + FRAME, Some(base)), + (false, base + 4 * FRAME, Some(base + FRAME)), + (false, base + 2 * FRAME, None), ] { let cc = mux.cc(VIDEO); mux.out @@ -5711,59 +5500,14 @@ mod test { let frames = read_track(&consumer, &name, crate::container::Kind::Data).await; assert_eq!(frames.len(), 6, "every cue publishes"); assert_forward(&frames); - // The cues follow the video across the wrap rather than piling up on the edge. + // The next GOP's cues follow the video rather than piling up on the edge. assert!( - frames[4].timestamp.as_micros() > frames[2].timestamp.as_micros(), - "the cues after the wrap did not move with the video: {:?}", + frames[3].timestamp.as_micros() > frames[2].timestamp.as_micros(), + "the cues did not move with the video: {:?}", frames.iter().map(|f| f.timestamp.as_micros()).collect::>() ); } - /// A cue arriving with the keyframe of an unflagged wrap lands with that keyframe, before - /// the write that grows the video shift has run. - #[tokio::test(start_paused = true)] - async fn section_with_a_wrap_keyframe_follows_it() { - const CUE_PID: u16 = 0x0021; - let mut mux = Mux { - out: synth_pmt( - &[ - (StreamType::H264, VIDEO), - (StreamType::Dts8ChannelLosslessAudio, CUE_PID), - ], - true, - ), - ..Default::default() - }; - for _ in 0..2 { - let cc = mux.cc(VIDEO); - let idr = video_pes(VIDEO, cc, 90_000 + FRAME, Some(90_000), &annexb_au(true)); - mux.out.extend_from_slice(&idr); - let cc = mux.cc(CUE_PID); - mux.out.extend_from_slice(&packet(true, cc, 0, &CUE)); - for (pts, dts) in [ - (90_000 + 4 * FRAME, Some(90_000 + FRAME)), - (90_000 + 2 * FRAME, None), - (90_000 + 3 * FRAME, None), - ] { - let cc = mux.cc(VIDEO); - mux.out - .extend_from_slice(&video_pes(VIDEO, cc, pts, dts, &annexb_au(false))); - } - mux.gops(VIDEO, 90_000 + 4 * FRAME, 1); - } - let (consumer, catalog, _import) = import_all(&mux.out).expect("the import must survive the wrap"); - let snapshot = catalog.snapshot(); - let video = snapshot.video.renditions.keys().next().unwrap().clone(); - let cues = snapshot.ext.mpegts.tracks.keys().find(|name| **name != video).unwrap(); - let cues = read_track(&consumer, cues, crate::container::Kind::Data).await; - let video = read_track(&consumer, &video, crate::container::Kind::Video).await; - assert_eq!(cues.len(), 2); - assert_eq!( - cues[1].timestamp, video[8].timestamp, - "the cue left its keyframe behind" - ); - } - /// An adaptation-only clock packet on `pid` carrying `ticks` of the 27 MHz PCR. fn pcr_packet(pid: u16, ticks: u64) -> Vec { let (base, ext) = (ticks / 300, ticks % 300); diff --git a/rs/moq-mux/src/container/ts/import_test.rs b/rs/moq-mux/src/container/ts/import_test.rs index 05ce77c687..0814f2be15 100644 --- a/rs/moq-mux/src/container/ts/import_test.rs +++ b/rs/moq-mux/src/container/ts/import_test.rs @@ -550,109 +550,116 @@ fn import_handles_unaligned_chunks() { assert_eq!(snapshot.audio.renditions.len(), 1); } -/// What a TS import published, plus the broadcast clock's reading around the first chunk. -struct LiveImport { +/// `data` with every PES PTS/DTS and PCR base moved `ticks` later on the 90 kHz clock, as an +/// encoder that had been running that much longer would stamp it. +fn shift_clock(data: &[u8], ticks: u64) -> Vec { + const FIELD: u64 = (1 << 33) - 1; + let mut out = data.to_vec(); + for pkt in out.as_chunks_mut::<188>().0 { + assert_eq!(pkt[0], 0x47, "an aligned TS packet"); + let mut payload = 4; + if pkt[3] & 0x20 != 0 { + let len = pkt[4] as usize; + if len >= 7 && pkt[5] & 0x10 != 0 { + let pcr = &mut pkt[6..11]; + let base = (pcr[0] as u64) << 25 + | (pcr[1] as u64) << 17 + | (pcr[2] as u64) << 9 + | (pcr[3] as u64) << 1 + | (pcr[4] as u64) >> 7; + let base = (base + ticks) & FIELD; + pcr[0] = (base >> 25) as u8; + pcr[1] = (base >> 17) as u8; + pcr[2] = (base >> 9) as u8; + pcr[3] = (base >> 1) as u8; + pcr[4] = (pcr[4] & 0x7F) | ((base as u8 & 1) << 7); + } + payload += 1 + len; + } + // Only a payload-unit start with a payload can open a PES header. + if pkt[1] & 0x40 == 0 || pkt[3] & 0x10 == 0 || payload + 9 > 188 { + continue; + } + let pes = &mut pkt[payload..]; + if pes[..3] != [0, 0, 1] { + continue; + } + let flags = pes[7] >> 6; + let mut at = 9; + for present in [flags & 0b10 != 0, flags == 0b11] { + if !present { + continue; + } + let t = &mut pes[at..at + 5]; + let v = ((t[0] as u64 >> 1) & 0x07) << 30 + | (t[1] as u64) << 22 + | (t[2] as u64 >> 1) << 15 + | (t[3] as u64) << 7 + | t[4] as u64 >> 1; + let v = (v + ticks) & FIELD; + t[0] = (t[0] & 0xF1) | (((v >> 30) as u8 & 0x07) << 1); + t[1] = (v >> 22) as u8; + t[2] = ((((v >> 15) & 0x7F) as u8) << 1) | 1; + t[3] = (v >> 7) as u8; + t[4] = (((v & 0x7F) as u8) << 1) | 1; + at += 5; + } + } + out +} + +/// What a TS import published, and the wall-clock window it arrived in. +struct Imported { published: std::collections::BTreeMap>, - before: u128, - after: u128, + /// The root clock the catalog advertised. + clock: hang::catalog::Clock, + arrival: std::ops::RangeInclusive, } -/// Import `chunks` in order, idling `idle` between them, on a clock that began `ago` earlier. -/// `live` translates onto that clock; otherwise the unwrapped PTS publishes verbatim. -async fn live_import(chunks: &[&[u8]], live: bool, ago: std::time::Duration, idle: std::time::Duration) -> LiveImport { +/// Import `data` on a catalog with the default clock. +async fn import_stream(data: &[u8]) -> Imported { let mut broadcast = moq_net::broadcast::Info::new().produce(); let consumer = broadcast.consume(); - let config = crate::catalog::Config::default().with_clock(crate::container::test_util::late_clock(ago)); - let catalog = crate::catalog::Producer::new(&mut broadcast, config).unwrap(); - let clock = catalog.clock(); + let catalog = crate::catalog::Producer::new(&mut broadcast, Default::default()).unwrap(); let mut import = crate::container::ts::Import::new(broadcast, catalog.reserve()); - if live { - import = import.live(); - } - let before = clock.now().as_micros(); - let mut after = before; - for (i, chunk) in chunks.iter().enumerate() { - if i > 0 { - std::thread::sleep(idle); - } - import.decode(chunk).unwrap(); - if i == 0 { - after = clock.now().as_micros(); - } - } + let before = std::time::SystemTime::now(); + import.decode(data).unwrap(); + let after = std::time::SystemTime::now(); import.finish().unwrap(); - LiveImport { - published: crate::container::test_util::published(&consumer, &catalog.snapshot()).await, - before, - after, + let snapshot = catalog.snapshot(); + Imported { + published: crate::container::test_util::published(&consumer, &snapshot).await, + clock: snapshot.clock.expect("the catalog advertises a clock"), + arrival: before..=after, } } -/// A TS feed arriving 30s after the broadcast began publishes on the broadcast clock rather than -/// its own PTS: the stream is live on arrival, and H.264 and AAC keep the one offset their PES -/// headers gave them. +/// A feed ten hours into its own PTS publishes those timestamps verbatim, H.264 and AAC keeping +/// the offset their PES headers gave them, and the catalog clock maps it to the arrival time. #[tokio::test] -async fn live_import_anchors_a_late_first_frame() { +async fn import_publishes_stream_pts_on_an_arrival_clock() { let data: &[u8] = include_bytes!("test_data/bbb_cbr.ts"); - let ago = std::time::Duration::from_secs(30); - let verbatim = live_import(&[data], false, ago, std::time::Duration::ZERO).await; - let live = live_import(&[data], true, ago, std::time::Duration::ZERO).await; - - let offset = crate::container::test_util::common_offset(&verbatim.published, &live.published); - // The earliest PES anchors at its arrival, which frames muxed ahead of it may precede. - let earliest = verbatim.published.values().map(|t| t[0]).min().unwrap() as i128 + offset; - let skew = std::time::Duration::from_secs(2).as_micros() as i128; + let hours = std::time::Duration::from_secs(10 * 3600); + let original = import_stream(data).await; + let shifted = import_stream(&shift_clock(data, hours.as_secs() * 90_000)).await; + + let offset = crate::container::test_util::common_offset(&original.published, &shifted.published); assert!( - live.before as i128 - skew <= earliest && earliest <= live.after as i128, - "the stream is live on arrival: {earliest} not near {}..={}", - live.before, - live.after + (offset - hours.as_micros() as i128).abs() <= 1_000, + "the source's own PTS: moved {offset}us" ); -} -/// The same feed played twice, as when an encoder restarts its PTS, continues forward after the -/// real idle gap instead of rewinding. -#[tokio::test] -async fn live_import_restarts_forward_after_idle() { - let data: &[u8] = include_bytes!("test_data/bbb_cbr.ts"); - let once = live_import(&[data], false, std::time::Duration::ZERO, std::time::Duration::ZERO).await; - - // How much source time one pass covers, across every stream. - let starts = once.published.values().map(|t| *t.iter().min().unwrap()); - let ends = once.published.values().map(|t| *t.iter().max().unwrap()); - let span = (ends.max().unwrap() - starts.min().unwrap()) as i128; - - for idle in [std::time::Duration::ZERO, std::time::Duration::from_millis(300)] { - let live = live_import(&[data, data], true, std::time::Duration::from_secs(30), idle).await; - - // Each pass lands on one mapping for every stream: the first frames on the first, the last - // frames on the second. - let offset = |pick: fn(&Vec) -> u128| { - let deltas: Vec = live - .published - .iter() - .map(|(name, t)| pick(t) as i128 - pick(&once.published[name]) as i128) - .collect(); - assert!( - deltas.iter().all(|d| (d - deltas[0]).abs() <= 1_000), - "every stream shares one mapping: {deltas:?}" - ); - deltas[0] - }; - let first = offset(|t| t[0]); - let second = offset(|t| *t.last().unwrap()); - - // The second pass continues after the first plus the real idle gap, not on top of it. - let shift = second - first; - assert!( - shift >= span + idle.as_micros() as i128, - "the restart resumes after the first pass and the idle gap: {shift} < {span} + {idle:?}" - ); - assert!( - shift < span + (idle + std::time::Duration::from_secs(5)).as_micros() as i128, - "the restart is not pushed further: {shift}" - ); - } + // The first PES anchors at its arrival, which frames muxed ahead of it may precede. + let earliest = shifted.published.values().map(|t| t[0]).min().unwrap(); + let wall = shifted + .clock + .wall_clock(moq_net::Timestamp::from_micros(earliest as u64).unwrap()) + .unwrap(); + let skew = std::time::Duration::from_secs(2); + assert!( + *shifted.arrival.start() - skew <= wall && wall <= *shifted.arrival.end(), + "the stream is live on arrival" + ); } diff --git a/rs/moq-mux/src/error.rs b/rs/moq-mux/src/error.rs index 046307d8a7..091a24e1ef 100644 --- a/rs/moq-mux/src/error.rs +++ b/rs/moq-mux/src/error.rs @@ -163,7 +163,7 @@ pub enum Error { #[error("{0}")] InvalidEnd(#[from] crate::container::InvalidEnd), - /// A frame's timestamp sits below the live edge earlier groups reached. + /// A group starts before the previous group did: the source restarted. #[error("{0}")] TimestampRewind(#[from] crate::container::TimestampRewind), @@ -214,8 +214,8 @@ pub enum Error { #[error("invalid timeline timescale: {0}")] InvalidTimescale(u32), - /// A source timestamp cannot be mapped onto the broadcast clock: it would land before the - /// broadcast began or outside the representable range. + /// A source's first timestamp cannot anchor the broadcast clock: it is so large that PTS zero + /// would land before the moq epoch (2020). #[error("timestamp cannot be mapped onto the broadcast clock: {0}")] UnmappableTimestamp(String), /// Tried to set an application catalog section whose name collides with a diff --git a/rs/moq-mux/src/lib.rs b/rs/moq-mux/src/lib.rs index b128bf22de..aa44b6fecb 100644 --- a/rs/moq-mux/src/lib.rs +++ b/rs/moq-mux/src/lib.rs @@ -40,7 +40,7 @@ pub mod select; mod source; pub mod timeline; -pub use clock::{Clock, SourceMap}; +pub use clock::Clock; pub use error::*; pub use pace::Pacer; pub use source::{Binding, Source};