diff --git a/quest/m1/README.md b/quest/m1/README.md index fee0f21d4e..805509a56d 100644 --- a/quest/m1/README.md +++ b/quest/m1/README.md @@ -108,6 +108,7 @@ QUIC studies there on that rule. - [CMAF surround Opus](/quest/m1/cmaf-opus-surround.md) - fMP4 import and export carry an Opus channel mapping table - [A self-hosted NVIDIA runner is registered](/quest/m1/gpu-runner.md) - the maintainer registers the host that runs the NVIDIA tests - [GPU CI](/quest/m1/gpu-ci.md) - NVIDIA tests run nightly on a self-hosted GPU runner, and `just rs nvidia` runs them locally instead of skipping +- [Rendition preference](/quest/m1/rendition-preference.md) - automatic selection by ``, `Video::ranked`, and WHEP keeps the highest `preference` that decodes, so a compatibility transcode is only picked when nothing preferred decodes - [JS rendition ranking](/quest/m1/js-ranked.md) - `@moq/hang` ranks video renditions like Rust, and `@moq/watch`'s fallback uses it - [Audio rendition pick](/quest/m1/audio-ranked.md) - single-track FLV/RTMP and WHEP serve the best audio rendition, not the first by name - [FLV catalog stream](/quest/m1/flv-catalog-stream.md) - on dev, `flv::Export` takes a catalog stream like fmp4, replacing `with_select` diff --git a/quest/m1/js-ranked.md b/quest/m1/js-ranked.md index c7f2131e89..2611d75706 100644 --- a/quest/m1/js-ranked.md +++ b/quest/m1/js-ranked.md @@ -4,11 +4,14 @@ `@moq/hang` ranks video renditions the same way as Rust's `hang::catalog::Video::ranked` (largest picture, then highest bitrate, ties in -name order), and `@moq/watch`'s fallback pick uses it, so the browser and the +name order), and `@moq/watch`'s no-target pick uses it, so the browser and the native egresses can't drift apart. ## Plan -Mirror the Rust name and semantics. `bestRendition` in `js/watch` already -matches today; replace it rather than keep two copies. Check whether the +Mirror the Rust name and semantics, including sorting by `preference` first +once [Rendition preference](/quest/m1/rendition-preference.md) lands. +`bestRendition` in `js/watch` already matches except exact ties, which keep +catalog order instead of name order; replace it rather than keep two copies. +Check whether the `byDimensions`/`byBitrate` filters can reuse the same ordering. diff --git a/quest/m1/rendition-preference.md b/quest/m1/rendition-preference.md new file mode 100644 index 0000000000..d591afe19f --- /dev/null +++ b/quest/m1/rendition-preference.md @@ -0,0 +1,78 @@ +# [M] Rendition preference + +## Goal + +A hang video rendition carries an optional `preference` (a signed integer, +absent means 0, higher wins). Automatic selection keeps only the highest +preference among the renditions it can decode, then picks by target and +bitrate inside that tier as today. ``, +`hang::catalog::Video::ranked`, and WHEP's codec choice honor it, so a viewer +and every single-rendition egress (single-track RTMP play, WHEP, FLV export, +moq-transcode's source pick) only subscribe to a compatibility transcode +(such as H.265 republished as H.264, marked `-1`) when nothing preferred +decodes. A multitrack RTMP client still receives every rendition. + +## Plan + +Requested by an external consumer (OneTooMany), who publishes an H.265 source +with its own H.264 transcode beside it, so a viewer that can decode H.265 +should never pull the transcode. + +Decided in planning interviews on 2026-10-01: + +- A strict numeric tier, not `fallback: bool`. Two tiers cover the request, + but a ladder per codec (AV1 > H.265 > H.264) needs three, and today's + area-then-bitrate sort would pick H.264 over a same-size AV1 rung because + its bitrate is higher. The tier costs the same to implement as the bool. +- Not a total order (DASH `@qualityRanking`, HLS `SCORE`): the publisher + would have to rank every rung, and ABR still needs size and bitrate to + step down a ladder. +- Not a routing-style `cost`: an honest "produced on demand" cost would mark + moq-transcode's rungs, so the strict rule would stop capable viewers from + stepping down. The publisher sets `preference` explicitly. +- moq-transcode's rungs inherit their source's `preference`, so a ladder + adapts within one tier even when the source is not at 0. `rung_entry` + copies it like `optimize_for_latency`, and the per-snapshot refresh beside + `inherit_stalled` keeps it current when the source catalog changes. +- Named `preference`, higher wins, after DASH's `@selectionPriority` (strict + across per-codec Adaptation Sets, higher preferred). Not `priority`, which + already means track send priority. Signed, so a new preferred ladder is `1` + without renumbering existing renditions, and a fallback is `-1`. +- Video only. Optional on the wire, omitted when 0 (like `stalled`). + Additive, so older players ignore it. +- Selection order in `js/watch/src/video/source.ts`: decode support, then + keep the highest preference among supported renditions, then `stalled`, + then the existing target and bitrate pick within what is left. Preference + is about decodability only: a stalled source does not move capable viewers + onto a lower tier, which is usually derived from it and likely stalled too. +- A manual `target.name` still wins, as it does for `stalled`. The quality + picker keeps listing every tier. +- `Video::ranked` sorts by preference (highest first), then by picture and + bitrate as today. RTMP play, FLV export, and moq-transcode take the first + rendition they support, so they need no change. Update the + [JS rendition ranking](/quest/m1/js-ranked.md) Plan if it is still open. +- WHEP needs its own step: `Session::handle_media` (`rs/moq-rtc`) takes the + peer's first negotiated payload type, then `pick_video` filters `ranked()` + to that codec, so a peer offering the fallback's codec first would get the + fallback. Choose across every negotiated video codec so the highest + supported preference wins. +- Update `rs/hang` `VideoConfig`, `js/hang` `VideoConfigSchema`, + `drafts/draft-lcurley-moq-hang.md` (next to `stalled`, with a + source-plus-fallback example), and `doc/concept/hang.md`. No new docs page. +- Tests: a supported source wins over a lower-preference rendition with a + higher bitrate, over a bitrate budget that fits only the lower one, and + while the source is stalled and the lower one is not; an unsupported + source selects the lower preference; three tiers resolve to the highest + supported one; a manual `target.name` selects a lower preference; `ranked` + orders a larger lower-preference rendition after a smaller source; a WHEP + peer offering the fallback's codec first still gets the source; a + moq-transcode source at preference 1 under a budget that fits only a rung + still selects the rung. +- Out of scope: moq-ffi, libmoq, and the bindings until a native player needs + the field. moq-transcode producing same-size codec fallbacks; the consumer + publishes its own. + +## Related + +- [JS rendition ranking](/quest/m1/js-ranked.md) - mirrors `Video::ranked` in `@moq/hang`, which sorts by preference first once this quest lands +- [Audio rendition pick](/quest/m1/audio-ranked.md) - audio ranking, where `preference` could join later