feat(mux): import timed SRT and WebVTT captions

doc/lib/rs/moq-mux.md

@@ -13,10 +13,10 @@ back. This is what `moq import` / `export` and the gateways are built on.
 
 | Format | Import | Export | Notes |
 | --- | --- | --- | --- |
-| fMP4 / CMAF | yes | yes | Passthrough as `cmaf` or repackaged as `legacy`. |
+| fMP4 / CMAF | yes | yes | Audio/video passthrough as `cmaf` or repackaged as `legacy`; `wvtt` subtitles import as timed WebVTT text, preserving source language and label when present. |
 | MPEG-TS | yes | yes | H.264/H.265; AAC, MP2, AC-3, E-AC-3, Opus up to 7.1; SCTE-35 and subtitle PIDs as tracks; service tables round-trip. |
 | FLV / RTMP | yes | yes | Legacy H.264 + AAC + MP3, plus enhanced-RTMP HEVC, AV1, VP9, Opus, AC-3, E-AC-3, and multitrack. |
-| Matroska / WebM | yes | yes | |
+| Matroska / WebM | yes | yes | Timed SRT and WebVTT subtitles import as WebVTT text renditions, preserving source language and label when present; other subtitle codecs are skipped. |
 | Annex-B (H.264, H.265) | yes | yes | Parameter sets extracted to the catalog or re-injected per keyframe. |
 
 Importers fill the catalog from the bitstream, split groups at keyframes, and

quest/m1/README.md

@@ -154,7 +154,8 @@ session outcomes, and embedder refusals inside session outcomes.
 - [Android capture](/quest/m1/mobile-capture-android.md) - Rust captures through NDK/JNI on Android, reusing the existing codecs and setting the catalog rotation
 - [Time stretch](/quest/m1/watch-audio-time-stretch.md) - js/watch: the audio ring converges by time-stretching instead of skipping or going silent
 - [Native audio quality](/quest/m1/audio-quality-native.md) - the browser lane's profiles, budgets, and metric schema run against `moq play` on a dummy device
-- [Caption import](/quest/m1/captions-import.md) - fMP4 and MKV subtitle tracks import as text renditions instead of erroring or being dropped
+- [TTML caption import](/quest/m1/captions-ttml.md) - fMP4 `stpp` tracks within a supported TTML/IMSC profile import as `ttml` renditions on the shared clock instead of being skipped
+- [fMP4 WebVTT ctim and vtta](/quest/m1/wvtt-ctim-vtta.md) - `wvtt` samples with `ctim` cue times or `vtta` comments import instead of failing
 - [Encoder colour](/quest/m1/color-model.md) - every moq-video encode path signals the colour its output actually has, or refuses instead of mislabelling
 - [T-STD TS export](/quest/m1/tstd/README.md) - `moq export ts` is a proper remux that passes the T-STD buffer model, starting with a fixed `--delay`
 - [ATSC AC-3](/quest/m1/ts-atsc-ac3.md) - `export ts` carries 48 kHz ATSC AC-3 at every A/52 rate up to 640 kb/s instead of aborting from 384 kb/s up

quest/m1/captions-ttml.md

@@ -0,0 +1,24 @@
+# [M] TTML caption import
+
+## Goal
+
+Importing an fMP4 `stpp` track within a supported TTML/IMSC profile yields a `ttml` text
+rendition whose cues stay on the shared media clock. Today those XML tracks are
+skipped with a warning while SRT and WebVTT import as `vtt`.
+
+## Plan
+
+Copying XML bytes is not enough. MP4 `stpp` names an XML namespace, which is not
+necessarily TTML, and the importer shifts the source clock, so every TTML time
+expression must be rewritten onto the catalog clock.
+
+- Support a bounded TTML/IMSC timing profile: validate the namespace and refuse
+  anything outside the profile instead of relabelling arbitrary XML as TTML.
+- Convert every time expression, including nested `begin`/`end`/`dur` and
+  `ttp:timeBase` variants the profile admits, after the clock shift.
+- Keep skipping XML tracks the profile does not cover, one track at a time.
+
+Matroska has no registered TTML codec ID (the old `S_TEXT/TTML` claim was
+removed), so confirm a real mapping before adding one.
+
+Cover a namespace mismatch, nested timing, and a shifted source clock.

quest/m1/wvtt-ctim-vtta.md

@@ -0,0 +1,23 @@
+# [S] fMP4 WebVTT ctim and vtta
+
+## Goal
+
+An fMP4 `wvtt` sample whose cue carries a `ctim` CueTimeBox, or which
+interleaves `vtta` additional-text boxes, imports as WebVTT text instead of
+failing the import. Today both are refused with `InvalidSubtitle`, as is any
+inline timestamp in a cue payload.
+
+## Plan
+
+Both boxes are defined by ISO/IEC 14496-30 §6.6.
+
+- `ctim` records the cue's start on the original WebVTT timeline. Rebase each
+  inline `<hh:mm:ss.ttt>` timestamp in the payload by `inline - ctim` onto the
+  sample start, reusing the Matroska rebase in `rs/moq-mux/src/container/text.rs`.
+  Refuse inline timestamps that arrive without `ctim`, since their origin is
+  unknown.
+- `vtta` holds text between cues (such as `NOTE` blocks). Serialize it in sample
+  order, validated like the Matroska cue-comment path.
+
+Cover an inline timestamp with `ctim` on a shifted clock, one without `ctim`,
+and a sample mixing `vtta` and `vttc`.

rs/moq-mux/src/container/fmp4/import.rs

@@ -26,6 +26,8 @@ use crate::catalog::Estimator;
 /// - AAC (MP4A)
 /// - Opus
 /// - FLAC
+///
+/// **Text:** WebVTT (`wvtt`), reconstructed as timed text; unsupported subtitle codecs are skipped.
 pub struct Import<E: crate::catalog::hang::CatalogExt = ()> {
 	/// The broadcast being produced
 	broadcast: moq_net::broadcast::Producer,
@@ -47,6 +49,7 @@ pub struct Import<E: crate::catalog::hang::CatalogExt = ()> {
 
 	// A lookup to tracks in the broadcast
 	tracks: HashMap<u32, Fmp4Track<E>>,
+	text: HashMap<u32, super::text::Track>,
 
 	// Track ids skipped by `select`, so their moof fragments are ignored rather
 	// than treated as referencing an unknown track.
@@ -152,6 +155,7 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 			initial_reservation: Some(reserved),
 			select: None,
 			tracks: HashMap::default(),
+			text: HashMap::default(),
 			skipped: HashSet::default(),
 			moov: None,
 			moof: None,
@@ -286,6 +290,15 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 			let handler = &trak.mdia.hdlr.handler;
 			let suffix = ".m4s";
 
+			if matches!(handler.as_ref(), b"sbtl" | b"subt" | b"text") {
+				if self.init_text(trak, &moov, &reserved)? {
+					continue;
+				}
+				tracing::warn!(track_id, handler = %handler, "unsupported fMP4 subtitle encoding, skipping");
+				self.skipped.insert(track_id);
+				continue;
+			}
+
 			let kind = Kind::from_handler(*handler)?;
 
 			// Drop tracks whose role isn't selected before minting or publishing them; their
@@ -347,7 +360,7 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 
 		// The reservation stays held until the first fragment anchors the clock, unless no track
 		// was declared: then no fragment ever will.
-		if self.tracks.is_empty() {
+		if self.tracks.is_empty() && self.text.is_empty() {
 			self.initial_reservation = None;
 		}
 
@@ -356,6 +369,39 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 		Ok(())
 	}
 
+	fn init_text(&mut self, trak: &Trak, moov: &Moov, reserved: &crate::catalog::Reserved<E>) -> Result<bool> {
+		let [mp4_atom::Codec::Wvtt(codec)] = trak.mdia.minf.stbl.stsd.codecs.as_slice() else {
+			return Ok(false);
+		};
+		if codec.plaintext.data_reference_index != 1 {
+			return Err(Error::InvalidSubtitle("external subtitle data reference").into());
+		}
+		let header = crate::container::text::header(&codec.config.config).map_err(Error::InvalidSubtitle)?;
+		let mut config = hang::catalog::TextConfig::new(hang::catalog::TextFormat::Vtt);
+		config.role = crate::container::text::role(&header);
+		config.lang = Some(trak.mdia.mdhd.language.clone());
+		// The picker label is the HandlerBox name; `vlab` is an opaque source identifier.
+		config.label = (!trak.mdia.hdlr.name.is_empty()).then(|| trak.mdia.hdlr.name.clone());
+		let wire = crate::catalog::hang::Container::new(&config.container, crate::container::Kind::Data)?;
+		let track = self.broadcast.create_track(
+			self.broadcast.unique_name(".mp4-t"),
+			self.catalog.track_info(hang::catalog::PRIORITY.text),
+		)?;
+		let producer = reserved.text(track, wire, config)?;
+		self.text.insert(
+			trak.tkhd.track_id,
+			super::text::Track {
+				producer,
+				header,
+				defaults: sample::Defaults::from_moov(moov, trak.tkhd.track_id),
+				timescale: moq_net::Timescale::new(trak.mdia.mdhd.timescale as u64)?,
+				end_decode_time: None,
+				discontinuity: false,
+			},
+		);
+		Ok(true)
+	}
+
 	fn container(&self, trak: &Trak, moov: &Moov) -> Result<Container> {
 		// Build a single-track init segment (ftyp+moov) for this track.
 		{
@@ -674,6 +720,19 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 		// Loop over all of the traf boxes in the moof.
 		for traf in &moof.traf {
 			let track_id = traf.tfhd.track_id;
+			if let Some(track) = self.text.get_mut(&track_id) {
+				track.extract(
+					super::text::Fragment {
+						traf,
+						data: &mdat.data,
+						moof_size,
+						header_size,
+					},
+					&mut self.timebase,
+				)?;
+				self.initial_reservation = None;
+				continue;
+			}
 			let track = match self.tracks.get_mut(&track_id) {
 				Some(track) => track,
 				// A fragment for a track `select` dropped: ignore it.
@@ -960,6 +1019,9 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 	/// Finish all tracks, flushing current groups.
 	pub fn finish(&mut self) -> Result<()> {
+		for track in self.text.values_mut() {
+			track.producer.finish()?;
+		}
 		// No frame follows to anchor the clock, so publish the declared track set now.
 		self.initial_reservation = None;
 		for track in self.tracks.values_mut() {
@@ -979,6 +1041,9 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 	/// Abort all tracks with `err` instead of finishing, so subscribers see the real
 	/// cause rather than [`moq_net::Error::Dropped`]. Consumes the importer.
 	pub fn abort(mut self, err: moq_net::Error) {
+		for track in std::mem::take(&mut self.text).into_values() {
+			track.producer.abort(err.clone());
+		}
 		// Dropping each track's rendition retires its catalog entry.
 		for mut track in std::mem::take(&mut self.tracks).into_values() {
 			if let Some(g) = track.group.take() {
@@ -993,8 +1058,13 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 	/// Broadcast-wide: every track inside this fMP4 import advances together; per-track
 	/// control is intentionally not exposed. Skipping sequences is not a new timeline, so the
 	/// next fragment must still advance past the last decode time; see
-	/// [`discontinuity`](Self::discontinuity).
+	/// [`discontinuity`](Self::discontinuity). Text spends a group per cue, so a text track that
+	/// is already past `sequence` continues from its own next group instead.
 	pub fn seek(&mut self, sequence: u64) -> Result<()> {
+		for track in self.text.values_mut() {
+			let next = track.producer.track().latest().map_or(0, |latest| latest + 1);
+			track.producer.seek(sequence.max(next))?;
+		}
 		for track in self.tracks.values_mut() {
 			track.estimator.cut(None);
 			track.publish_estimate()?;
@@ -1012,6 +1082,10 @@ impl<E: crate::catalog::hang::CatalogExt> Import<E> {
 	/// The source signalled a new timeline (an HLS `EXT-X-DISCONTINUITY`), so the next
 	/// fragment's decode time is not compared with the last one on each track.
 	pub fn discontinuity(&mut self) {
+		for track in self.text.values_mut() {
+			track.end_decode_time = None;
+			track.discontinuity = true;
+		}
 		for track in self.tracks.values_mut() {
 			track.last_decode_time = None;
 		}