diff --git a/quest/m1/README.md b/quest/m1/README.md index fef5e2a0e8..254eb4639f 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -72,6 +72,9 @@ QUIC studies there on that rule. - [Audio capture without ALSA link](/quest/m1/capture-alsa-link.md) - moq-audio capture and playback build on Linux without linking libasound - [Capture by default](/quest/m1/capture-default.md) - moq-video and moq-audio build `capture` by default, so pre-merge checks test it and the capture gate goes away - [Ship capture and playback](/quest/m1/cli-packaging.md) - a released moq binary can capture and play, which no distribution currently enables +- [Import at the first frame](/quest/m1/import-first-frame.md) - fMP4, MKV, and MPEG-TS imports publish the catalog at their first frame, so a lone importer's root clock never moves +- [CMAF frame timestamp](/quest/m1/cmaf-frame-timestamp.md) - CMAF decoders time samples from the moq-lite frame timestamp, using `tfdt` only within the fragment +- [Shared import clock](/quest/m1/shared-clock.md) - an importer joining a clock already in use offsets its PTS instead of moving it, so captures and imports share one timeline - [Publishing never invents a timestamp](/quest/m1/publish-timestamp.md) - no Rust or binding publish API fills in a timestamp; an untimed payload goes out untimed - [JS publishing never invents a timestamp](/quest/m1/js-publish-timestamp.md) - the same in @moq/json, @moq/flate, and @moq/net - [Untimed model](/quest/m1/untimed-model.md) - moq-net frames and datagrams carry an absent timestamp to every subscriber; no receiver fills in arrival time diff --git a/quest/m2/cmaf-frame-timestamp.md b/quest/m1/cmaf-frame-timestamp.md similarity index 69% rename from quest/m2/cmaf-frame-timestamp.md rename to quest/m1/cmaf-frame-timestamp.md index fb5f3a8d78..d4a7957ab1 100644 --- a/quest/m2/cmaf-frame-timestamp.md +++ b/quest/m1/cmaf-frame-timestamp.md @@ -25,17 +25,27 @@ shifts `trun` data offsets. confirm with a test. - Check the TS exporter's verbatim carriage, which keeps PES PTS inside the payload. Rewrite those from the frame timestamp, or refuse a mismatch. -- An untimed CMAF frame is refused once the frame timestamp is the - fragment's timeline (decided 2026-10-02, with - [moq-net carries untimed frames faithfully](/quest/m1/untimed-model.md)). +- An untimed CMAF frame decodes at its `tfdt` (decided 2026-10-05, replacing + the 2026-10-02 decision to refuse it). Rejected: refusing it, and parking + the quest. - `main`: a behavior fix with no API change. +Open PR [#4826](https://github.com/moq-dev/moq/pull/4826) implements this. It +lands after [#4822](https://github.com/moq-dev/moq/pull/4822) +([untimed model](/quest/m1/untimed-model.md)), which brings untimed frames to +`main` and itself waits on a typed-timedness mock-up quest. + Test: in Rust and JS, a fragment whose `tfdt` disagrees with its frame -timestamp decodes at the frame timestamp, with B-frame offsets preserved. +timestamp decodes at the frame timestamp, with B-frame offsets preserved, and +an untimed fragment decodes at its `tfdt`. Public API: none. Wire: none; this states what the frame timestamp already means. +## Required + +- [Untimed model](/quest/m1/untimed-model.md) - untimed frames reach `main`, so the `tfdt` fallback has something to decode + ## Related -- [Shared clock](/quest/m2/shared-clock.md) - the first publisher to rely on it +- [Shared clock](/quest/m1/shared-clock.md) - the first publisher to rely on it diff --git a/quest/m1/import-first-frame.md b/quest/m1/import-first-frame.md new file mode 100644 index 0000000000..4356b04a6a --- /dev/null +++ b/quest/m1/import-first-frame.md @@ -0,0 +1,69 @@ +# [S] Importers publish the catalog at their first frame + +## Goal + +A catalog fed by one container importer is first published with its final +root `clock`. Today fMP4 and MKV imports publish the catalog when their init +segment (`moov`, `Tracks`) resolves the reservation, and an Opus-only MPEG-TS +import when its PMT does, on a provisional clock, then re-anchor it on the +first frame (`catalog::Producer::anchor`). Readers that copy the clock once +(moq-hls export's `EXT-X-PROGRAM-DATE-TIME` and `availabilityStartTime`, +derived broadcasts) keep the provisional one, and the hang draft says the +mapping is "fixed for the broadcast". + +Non-goal: a clock that never moves after any publish. A container set up +after a data track or catalog section has published still re-anchors once; +[shared-clock](/quest/m1/shared-clock.md) makes the first publish fix the +clock and offsets that container instead. + +## Plan + +The anchor is unreleased, and this breaks an in-tree path (an fMP4 or MKV +import read by moq-hls export), so it lands before the next release cut. +Requested by an external consumer (OneTooMany). Ranked first in m1, ahead of +shared-clock and cmaf-frame-timestamp (maintainer, 2026-10-05). + +Decided (2026-10-05): + +- fMP4, MKV, and MPEG-TS hold their initial reservation until their first + frame anchors, as FLV already does (`flv/import.rs`, "anchored before the + reservation below publishes"). The first snapshot then carries the anchored + clock. PTS stays verbatim for a single importer, so a passthrough fragment's + `tfdt` still agrees with its frame timestamp. +- Split from the strict "first publish fixes the clock" rule, which needs + shared-clock's offset for a container joining a fixed clock. The hold alone + needs no offset and regresses nothing: an earlier publish still lets the + anchor move the clock, as on `main`. +- Rejected: omitting `clock` until anchored. Adding it later is still a change + copy-once readers miss. + +Guidance: + +- An fMP4 `moov` declares every track at once, and MKV's `Tracks` likewise, so + the first frame of any track releases the hold. +- MPEG-TS drops its hold after the PMT (`ts/import.rs`, "Every stream in the + initial program is registered now"). Video configs resolve only on the first + frame, after the anchor, but Opus builds its config from PMT descriptors, so + an Opus-only stream publishes before its first PES anchors. Keep the hold + until the first anchor. +- fMP4 detects a duplicate `moov` by its reservation being gone + (`fmp4/import.rs`, `DuplicateMoov`). Holding it to the first fragment lets a + second `moov` slip through, so key the check on the parsed `moov` instead. + Likewise check that a TS PMT version change before the first PES does not + re-reserve against the held handle. +- Tests (fail on `main` for fMP4, MKV, and Opus-only TS): for each of fMP4, + MKV, FLV, and TS, a catalog consumer's first snapshot carries the anchored + clock and later snapshots never change it. The TS case is Opus-only with a + nonzero first PTS, since a video stream's config resolves only on its first + frame, after the anchor. Also a TS program of only verbatim PIDs, which + publish through `modify` with no reservation. Plus an end-to-end fMP4 import + into moq-hls export whose `EXT-X-PROGRAM-DATE-TIME` matches the anchored + clock. +- Docs: fix the `Config::with_clock` and `Producer::anchor` comments and + `doc/lib/rs/moq-mux.md`, and add an `Unreleased` line in + `doc/setup/upgrade.md`: the catalog now appears at an importer's first + frame, not its init segment. + +Public API: no new items. A catalog fed by an fMP4, MKV, or Opus-only TS +importer is published at the first frame instead of the init segment. Wire: +none; the hang draft already says the mapping is fixed. diff --git a/quest/m1/shared-clock.md b/quest/m1/shared-clock.md new file mode 100644 index 0000000000..f742f84e72 --- /dev/null +++ b/quest/m1/shared-clock.md @@ -0,0 +1,144 @@ +# [L] One timeline for every source on a catalog + +## Goal + +Every source sharing a catalog (captures, and any number of container +importers) lands on one forward timeline, whichever starts first. Today the +first importer's first frame re-anchors the catalog clock +(`catalog::Producer::anchor`). A capture that started earlier has copied the +old clock, and a second importer's anchor is a no-op, so it publishes its own +PTS base on a clock it didn't place. Either way the sources drift apart. Following the new clock doesn't help: sources near PTS 0 step it +backwards, and `container::Producer::write` refuses a group below the last one +(`TimestampRewind`), even across a discontinuity. + +The catalog's root `clock` is also final from the first snapshot a consumer +sees. [import-first-frame](/quest/m1/import-first-frame.md) covers a lone +importer; a container set up after a data track or catalog section has +published (any order moq-c and moq-ffi allow) still re-anchors it after +copy-once readers (moq-hls export, derived broadcasts) took the old one. + +## Plan + +Follow-up of [#4668](https://github.com/moq-dev/moq/pull/4668), which moved +data tracks onto the anchored clock. The anchor is unreleased. + +Decided (2026-10-01): + +- The clock never moves once something stamps with it. The public + `catalog::Producer::clock()` fixes the mapping (sets `anchored`). Crate-internal + readers, like #4668's `Listing`, read the state without fixing it. +- An importer anchors once, on its first frame, and gets back an offset: zero + if it placed the mapping (PTS stays verbatim), `clock.now() - first_pts` + otherwise. Every frame from that importer, on all of its tracks, is shifted + by that offset, so its audio and video stay in sync. Separate importers get + separate offsets. +- Captures read the clock once and never re-check it. Audio already stamps by + sample count from one `clock.now()` per epoch; there is no per-frame clock + read to add. +- Rejected: captures re-read the clock per frame and mark a break when it + changes. A backward anchor would hit `TimestampRewind`, and every audio + buffer would take the catalog lock to watch for an event that should never + happen while it is live. +- The offset rewrites only the moq-lite frame timestamp, never the payload. An + fMP4 passthrough fragment keeps its source `tfdt`, so the two disagree. The + frame timestamp is the broadcast timeline; a payload's timestamps are only + relative within the frame. Ad insertion (splicing sources with unrelated PTS + bases) needs the same rule. + +Decided (2026-10-05): + +- The first catalog publish also fixes the clock, whatever triggered it: a + snapshot on the wire is a mapping a reader may have copied. A data track or + section publishes the moment it is registered (`Producer::data_entry` holds + its reservation only for `init`), so a container set up after one gets an + offset. This lands with the offset, not before it, so the rule never exists + without it. +- m1: the anchor is unreleased, and landing this before the cut means no + release ships a clock that moves after a publish. +- Rejected for a container joining a published clock: a no-op with verbatim + PTS (an fMP4 an hour into its `tfdt`, or a TS PTS up to about 26.5 h, lands + that far off), a warning or refusal (a regression for an order the bindings + allow), and holding every publish until a container anchors (a data-only or + capture-only catalog would never publish). Fixing the clock at catalog + creation is rejected too: even a lone importer would carry frame timestamps + that disagree with its `tfdt`. +- One source's offset lives on a public `catalog::Input` handle, separate + from the publication gate: `catalog.input()` returns a clonable handle for + one PTS base, and `input.reserve()` mints a `Reserved` gated as today that + shares the input's offset. The first anchor through any of them sets it. + `catalog.reserve()` keeps a fresh offset per call. Holding an `Input` never + withholds the catalog. moq-hls import holds one for the whole import, so its + renditions (separate fMP4 importers on one PTS base) and every replacement + importer on an `EXT-X-MAP` change share one offset; separate offsets would + shift each by its first frame's arrival gap. Named `Input`, not `Source`, + which would clash with the public `moq_mux::Source`; `Timebase` was the + other candidate. +- Rejected: sharing the offset through `Reserved` clones. A live `Reserved` + withholds the initial catalog, so a handle kept for later importers would + hold it for the whole import. Also rejected: importers handing the offset + to each other (`with_offset` on all four). +- A `with_clock` catalog is fixed from the start, so every importer gets an + offset, the first included. Otherwise a later importer could end up with a + negative timestamp. This drops the promise that a `with_clock` PTS zero + names a recording's real start. Rejected: the first importer keeps its PTS + and only later ones shift, and no `with_clock` importer shifts. +- One payload exception: an SCTE-35 section on a section-framed verbatim track + absorbs the offset in its `pts_adjustment` (modulo 2^33, at 90 kHz), with + `CRC_32` recomputed. Its `pts_time` sits on the source PTS base, and nothing + downstream can recover the offset: the section's frame timestamp is the video + clock at arrival plus that same offset. The field keeps its size and is clear + even in an encrypted section, so TS export and the typed cues of + [#2279](/quest/m2/2279-hang-typed-scte-35-ad-cue-signaling-carried-opaquely.md) + read splice times on the broadcast timeline with no change of their own. +- Rejected for SCTE-35: recording the offset in the `mpegts` catalog section for + TS export to add (a new catalog field every typed consumer would also have to + apply), refusing to offset a TS import that carries sections (SCTE-35 could + never share a clock), and deferring to #2279 (this quest is what makes the + splice times stale). + +Guidance: + +- The offset is signed: a stream starting at 3600 s on a clock reading 10 s + shifts down. Refuse a frame that would land below zero rather than clamp it. +- Docs: document on `catalog::Producer::clock` that taking the clock fixes + it. Fix every doc that says the clock re-anchors or that `with_clock` keeps + PTS verbatim: `Config::with_clock` in `catalog/producer.rs`, + `doc/lib/rs/moq-mux.md` (the importer paragraph: a data track created before + the first frame following the anchor, and a `with_clock` zero naming the + real start), `doc/setup/upgrade.md` (pinning with `with_clock` when the + source's zero is known, and reading `catalog.clock()` at write time because + an importer re-anchors it), and the `binary.rs` module doc's "read at write + time" note. Replace `with_clock_names_the_contents_real_start` with a test + that a `with_clock` catalog's first importer is shifted onto it. +- Test: a data track registered before a container's first frame, then a + container starting at a nonzero PTS: the clock never moves, and the + container's frames land at now. This rewrites + `a_write_follows_the_anchored_clock` (`binary.rs`, `json.rs`) and + `an_anchor_is_not_jitter` (`binary.rs`), which create a data track and then + anchor. +- Test: moq-hls import receives its initial catalog while its `Input` is + alive, then replaces a rendition's importer, and the replacement keeps the + same offset. +- Test: a TS import carrying a `splice_insert` joins a clock already in use. In + its TS export, `pts_time + pts_adjustment` lands on the exported video PTS of + the splice point, and the section's CRC verifies. +- Test: start a synthetic capture, then import an fMP4 starting at PTS 0. Both + tracks advance from the capture's timeline with no rewind. Also cover the + reverse order: an importer first on a default clock keeps its PTS verbatim. + A passthrough fragment whose `tfdt` disagrees with its frame timestamp + decodes at the frame timestamp, in Rust and JS. + +Public API: `catalog::Input`, `catalog::Producer::input`, and +`Input::reserve` are new. `catalog::Producer::clock()` and the first publish +now fix the mapping, and importers no longer publish verbatim PTS when the +clock was already taken or set with `Config::with_clock`. +Wire: none. + +## Required + +- [Import at the first frame](/quest/m1/import-first-frame.md) - a lone importer anchors before the first publish, so the publish rule doesn't offset it +- [CMAF frame timestamp](/quest/m1/cmaf-frame-timestamp.md) - decoders honour an offset frame timestamp on passthrough tracks + +## Related + +- [Audio capture time](/quest/m2/audio-capture-time.md) - maps audio's capture timeline onto the broadcast clock once per open diff --git a/quest/m1/subscribe-live-time.md b/quest/m1/subscribe-live-time.md index 3068133a14..a81f28b84a 100644 --- a/quest/m1/subscribe-live-time.md +++ b/quest/m1/subscribe-live-time.md @@ -50,4 +50,4 @@ Decided in planning (2026-10-03): ## Related - [Untimed model](/quest/m1/untimed-model.md) - absent timestamps end to end; only real pairs feed the estimate -- [Shared clock](/quest/m2/shared-clock.md) - the hang catalog's `{wall, timescale}` anchor sits a layer up; this stays media-agnostic +- [Shared clock](/quest/m1/shared-clock.md) - the hang catalog's `{wall, timescale}` anchor sits a layer up; this stays media-agnostic diff --git a/quest/m2/README.md b/quest/m2/README.md index 736fd67ebe..e50b496522 100644 --- a/quest/m2/README.md +++ b/quest/m2/README.md @@ -111,8 +111,6 @@ waiting on an upstream release waits in [m4](/quest/m4/README.md). - [Windows capture parity](/quest/m2/capture-windows.md) - system audio and a settled app-capture policy - [Linux capture parity](/quest/m2/capture-linux.md) - Wayland window/system-audio capture with explicit display-selection and app-capture limits - [Audio capture time](/quest/m2/audio-capture-time.md) - native audio stamps a buffer's capture instant, not when the driver reads it -- [CMAF frame timestamp](/quest/m2/cmaf-frame-timestamp.md) - CMAF decoders time samples from the moq-lite frame timestamp, using `tfdt` only within the fragment -- [Shared import clock](/quest/m2/shared-clock.md) - an importer joining a clock already in use offsets its PTS instead of moving it, so captures and imports share one timeline - [Interop graceful close](/quest/m2/interop-graceful-close.md) - successful runner publications withdraw before disconnecting; external repository approval required - [Egress profile](/quest/m2/quic-egress-profile.md) - measure relay send-path syscalls, pacing bursts, and allocations before optimizing any of them - [SEI separation study](/quest/m2/sei.md) - measure whether separating SEI saves enough, or has a metadata-only consumer, to justify a split diff --git a/quest/m2/shared-clock.md b/quest/m2/shared-clock.md deleted file mode 100644 index 80800597cb..0000000000 --- a/quest/m2/shared-clock.md +++ /dev/null @@ -1,67 +0,0 @@ -# [M] One timeline for every source on a catalog - -## Goal - -Every source sharing a catalog (captures, and any number of container -importers) lands on one forward timeline, whichever starts first. Today the -first importer's first frame re-anchors the catalog clock -(`catalog::Producer::anchor`). A capture that started earlier has copied the -old clock, and a second importer's anchor is a no-op, so it publishes its own -PTS base on a clock it didn't place. Either way the sources drift apart. Following the new clock doesn't help: sources near PTS 0 step it -backwards, and `container::Producer::write` refuses a group below the last one -(`TimestampRewind`), even across a discontinuity. - -## Plan - -Follow-up of [#4668](https://github.com/moq-dev/moq/pull/4668), which moved -data tracks onto the anchored clock. The anchor is unreleased. - -Decided (2026-10-01): - -- The clock never moves once something stamps with it. The public - `catalog::Producer::clock()` fixes the mapping (sets `anchored`). Crate-internal - readers, like #4668's `Listing`, read the state without fixing it. -- An importer anchors once, on its first frame, and gets back an offset: zero - if it placed the mapping (PTS stays verbatim), `clock.now() - first_pts` - otherwise. Every frame from that importer, on all of its tracks, is shifted - by that offset, so its audio and video stay in sync. Separate importers get - separate offsets. -- Captures read the clock once and never re-check it. Audio already stamps by - sample count from one `clock.now()` per epoch; there is no per-frame clock - read to add. -- Rejected: captures re-read the clock per frame and mark a break when it - changes. A backward anchor would hit `TimestampRewind`, and every audio - buffer would take the catalog lock to watch for an event that should never - happen while it is live. -- The offset rewrites only the moq-lite frame timestamp, never the payload. An - fMP4 passthrough fragment keeps its source `tfdt`, so the two disagree. The - frame timestamp is the broadcast timeline; a payload's timestamps are only - relative within the frame. Ad insertion (splicing sources with unrelated PTS - bases) needs the same rule. -- m2: nothing in-tree reaches it. `moq-cli publish` runs a capture or a stdin - import, never both, and moq-ffi does not expose capture. - -Guidance: - -- The offset is signed: a stream starting at 3600 s on a clock reading 10 s - shifts down. Refuse a frame that would land below zero rather than clamp it. -- A `with_clock` catalog is fixed from the start, so its importers offset too. - Today they publish verbatim PTS on a clock they didn't place. -- Document on `catalog::Producer::clock` that taking the clock fixes it. -- Test: start a synthetic capture, then import an fMP4 starting at PTS 0. Both - tracks advance from the capture's timeline with no rewind. Also cover the - reverse order: an importer first keeps its PTS verbatim. A passthrough - fragment whose `tfdt` disagrees with its frame timestamp decodes at the frame - timestamp, in Rust and JS. - -Public API: no new items. `catalog::Producer::clock()` now fixes the mapping, -and importers no longer publish verbatim PTS when the clock was already taken. -Wire: none. - -## Required - -- [CMAF frame timestamp](/quest/m2/cmaf-frame-timestamp.md) - decoders honour an offset frame timestamp on passthrough tracks - -## Related - -- [Audio capture time](/quest/m2/audio-capture-time.md) - maps audio's capture timeline onto the broadcast clock once per open