From e71a47d492cf625f36d1ef4e8913c72994fec32c Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 21 Sep 2026 18:08:08 -0700 Subject: [PATCH 1/2] chore: claim quest/main/media-release-review From 822512ad09c5d201395c704027509452287903b1 Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Mon, 21 Sep 2026 18:25:02 -0700 Subject: [PATCH 2/2] chore(quest): complete the media release review Records that the audio, video, transcode, and NVENC contracts landed on main at f1894110b with no release blockers, fixes the stale backlog and docs the review turned up, adds the missing Unreleased changelog entries, and marks RateError non-exhaustive like every other public error enum. Co-Authored-By: Claude Opus 5 --- doc/bin/cli.md | 5 ++-- doc/lib/rs/moq-video.md | 2 +- quest/future/audio-loss-recovery.md | 3 ++- quest/future/intra-refresh/encode-config.md | 7 ++--- quest/future/intra-refresh/v4l2-refresh.md | 4 +-- quest/future/sip-stack.md | 7 ++--- quest/future/video-vaapi.md | 4 +-- quest/main/README.md | 5 ++-- quest/main/media-release-review.md | 27 ------------------- ...bandwidth-grant-in-moq-audio-instead-of.md | 25 +++++++++-------- quest/next/audio-jitter-target/README.md | 4 +-- quest/next/audio-jitter-target/native.md | 12 ++++----- quest/next/capture-alsa-link.md | 9 ++----- quest/next/jitter-flush-clock.md | 4 +-- quest/next/keyframe-trigger.md | 19 ++++++------- quest/next/nvenc-recovery.md | 4 +-- quest/next/play-audio-rendition-gap.md | 2 +- quest/next/qos/stats/encoder-feedback.md | 8 +++--- rs/moq-audio/CHANGELOG.md | 10 +++++++ rs/moq-mux/CHANGELOG.md | 2 ++ rs/moq-nvenc/CHANGELOG.md | 12 +++++++++ rs/moq-transcode/CHANGELOG.md | 5 ++++ rs/moq-transcode/README.md | 6 ++--- rs/moq-transcode/src/lib.rs | 4 ++- rs/moq-video/CHANGELOG.md | 15 +++++++++++ rs/moq-video/DESIGN-native-codecs.md | 17 ++++++------ rs/moq-video/README.md | 5 ++-- rs/moq-video/src/rate.rs | 1 + 28 files changed, 124 insertions(+), 104 deletions(-) delete mode 100644 quest/main/media-release-review.md diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 1efcaeea54..70f1493158 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -136,8 +136,9 @@ moq ... transcode --rung 720:2500000 --rung 360:600000 --encoder nvenc --decoder Publishes `cam.hang/transcode.hang` whose catalog references the source's rendition and adds lower rungs that are decoded and encoded only while someone -watches them. On NVIDIA the whole pipeline stays on the GPU. Requires the -`transcode` feature. +watches them. On NVIDIA the whole pipeline stays on the GPU; `--frames cpu` +forces decoded frames into CPU memory instead of the default `native`. +Requires the `transcode` feature. The ladder is sized against the source picture and follows it, so a source that changes resolution mid-stream (a window capture renegotiated by a resize, a diff --git a/doc/lib/rs/moq-video.md b/doc/lib/rs/moq-video.md index 1a8d80af4a..0d6d5007b7 100644 --- a/doc/lib/rs/moq-video.md +++ b/doc/lib/rs/moq-video.md @@ -20,7 +20,7 @@ ffmpeg, no GStreamer, no system codec to install. Highlights: -- **Automatic backend selection**, hardware first. Linux GPU libraries are `dlopen`ed at runtime, so one binary starts anywhere and warns when it falls back to software. openh264 is statically linked as the H.264 fallback; H.265 is hardware-only; AV1 decodes via NVDEC. The VAAPI encoder is compile-verified but not yet validated on hardware. +- **Automatic backend selection**, hardware first. Linux GPU libraries are `dlopen`ed at runtime, so one binary starts anywhere and warns when it falls back to software. openh264 (the default-on `openh264` feature) is statically linked as the H.264 fallback; H.265 is hardware-only; AV1 decodes via NVDEC. The VAAPI encoder is compile-verified but not yet validated on hardware. - **Publish on demand.** `encode::publish_capture` advertises the track up front and opens the camera only while someone subscribes. - **GPU ownership where the platform allows.** Matching codec backends consume their native GPU surfaces directly. The renderer imports `CVPixelBuffer` and supported DMA-BUF formats. Linux/NVIDIA producers can import dedicated Vulkan RGBA8 slots into CUDA with timeline-semaphore ordering and completion-driven slot return. Vulkan/CUDA surfaces deliberately have no CPU pixel fallback; other surfaces use the typed `Surface::into_i420()` and configured `Surface::to_rgba(config)` when needed. - **Live bitrate control** where the selected backend supports it, without forcing a keyframe. An unsupported backend keeps its opening rate. diff --git a/quest/future/audio-loss-recovery.md b/quest/future/audio-loss-recovery.md index 5c6d1079e2..d8594d96b4 100644 --- a/quest/future/audio-loss-recovery.md +++ b/quest/future/audio-loss-recovery.md @@ -3,7 +3,8 @@ ## Goal Decide whether a concrete MoQ audio consumer benefits from in-band Opus FEC, -with a tested loss/latency policy before exposing a replacement public option. +with a tested loss/latency policy before exposing any additive option on +`encode::Settings`. This does not block 0.1. ## Plan diff --git a/quest/future/intra-refresh/encode-config.md b/quest/future/intra-refresh/encode-config.md index 8bc9f9143c..42cfbc81ff 100644 --- a/quest/future/intra-refresh/encode-config.md +++ b/quest/future/intra-refresh/encode-config.md @@ -25,9 +25,10 @@ without replacing an API after 0.1. error for `Refresh` (supported or refused, never a silent fallback to keyframes). The test-only probe backend accepts it so the producer logic is testable without hardware. -- Group boundaries in refresh mode come from counting: backends report - `keyframe = false` for a sweep start, so the producer marks the first frame - of each cycle by frame count from the last cut and opens the group there. +- Group boundaries in refresh mode come from counting: the splitter sees no + IDR at a sweep start (only a recovery-point SEI where the backend emits + one), so the producer marks the first frame of each cycle by frame count + from the last cut and tells the importer to open the group there. A backend reports the sweep length it actually configured, which can be shorter than the cycle (NVENC needs it strictly shorter), and the producer publishes `warmup` as that length over the framerate. diff --git a/quest/future/intra-refresh/v4l2-refresh.md b/quest/future/intra-refresh/v4l2-refresh.md index fc7c411568..a536e88bf6 100644 --- a/quest/future/intra-refresh/v4l2-refresh.md +++ b/quest/future/intra-refresh/v4l2-refresh.md @@ -17,8 +17,8 @@ claiming a boundary that was not encoded. `V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB` as macroblocks per frame derived from the picture size and the cycle) from the sys crate, set `GOP_SIZE` to zero for an unbounded GOP, and refuse the mode when the driver - rejects both controls, unlike the self-disabling `keyframes` fallback used - for forced keyframes. + rejects both controls, the same way a missing `FORCE_KEY_FRAME` control + refuses cuts with `CutUnsupported` at open. - A cut in refresh mode restarts the sweep. If the driver cannot honor it, report rejection through the error path settled in main; this does not independently require a Result from cut(). Never open a group out of phase diff --git a/quest/future/sip-stack.md b/quest/future/sip-stack.md index 4d812ad5dc..261b4a1e24 100644 --- a/quest/future/sip-stack.md +++ b/quest/future/sip-stack.md @@ -21,9 +21,10 @@ consumable by an embedder. - The media leg is plain negotiated RTP: `moq-rtc`/str0m is ICE/DTLS-first and its reusable session internals are crate-private, so it is precedent, not a base. Consider publicizing moq-rtc's codec bridges rather than - duplicating the RTP-to-hang mapping, and reuse `moq-audio`'s Opus and - resampler for the G.711<->Opus transcode (mono 8 kHz; G.711 companding is - new code, nothing in the repository has it). + duplicating the RTP-to-hang mapping, and reuse `moq-audio`'s Opus codec + for the G.711<->Opus transcode, with rate conversion through + `decode::Output` and `encode::Input` since the resampler is private (mono + 8 kHz; G.711 companding is new code, nothing in the repository has it). - RTP wall-clock normalization off RTCP sender reports, as moq-rtc does. - The embedder decides paths and auth; the crate's API is "answer this INVITE, give me the caller as Opus, take Opus to play" plus diff --git a/quest/future/video-vaapi.md b/quest/future/video-vaapi.md index 7cd8b5348e..864b4b3988 100644 --- a/quest/future/video-vaapi.md +++ b/quest/future/video-vaapi.md @@ -12,8 +12,8 @@ release first. Four gaps, one external dependency. **Decode.** The H.264 decoder landed (moq-vaapi 0.0.4, `decode/backend/vaapi.rs`), -with `decode::Config::gpu_frames` exporting DMA-BUF surfaces the renderer -imports without a download. H.265 decode is still missing, so a Linux box +with the default `decode::Config::output` of `Output::Native` handing out +DMA-BUF surfaces the renderer imports without a download. H.265 decode is still missing, so a Linux box without NVDEC has no hardware path for it. **The encoder.** Ours is a 111-line CPU-only adapter whose own header says it diff --git a/quest/main/README.md b/quest/main/README.md index 3f26121854..70c9d6072a 100644 --- a/quest/main/README.md +++ b/quest/main/README.md @@ -41,8 +41,8 @@ Their package boundaries are explicit: The media crates are also 0.0.x, so their changes target main. Adapt callers in other packages without breaking their published APIs, C layouts, or wire -formats. Do not bump versions as part of these quests. The media review records -when the four crates are ready for a separately requested 0.1 release. +formats. Do not bump versions as part of these quests. The media review +found the four crates ready for a separately requested 0.1 release. Their package boundaries are explicit: @@ -90,7 +90,6 @@ do not add another media abstraction or a renderer crate during stabilization. - [GPU conversion and NVENC](/quest/main/video-gpu-encode.md) - validate the GPU conversion, resize and NVENC path for imported frames on NVIDIA hardware -- [Media release review](/quest/main/media-release-review.md) - verify the settled contracts before separately authorizing 0.1 releases ## Related diff --git a/quest/main/media-release-review.md b/quest/main/media-release-review.md deleted file mode 100644 index 6f3a77b96b..0000000000 --- a/quest/main/media-release-review.md +++ /dev/null @@ -1,27 +0,0 @@ -# [S] Verify the four media contracts before 0.1 - -## Goal - -Record that the agreed audio, video, transcode, and NVENC API changes have -landed, or return concrete remaining blockers, before a separate version-bump -request. Completing this review does not bump or publish packages. - -## Plan - -Review the final public surface and examples after the prerequisite quests. -Check that deferred backend, surround, refresh, color, and performance work can -use the extension points without another planned replacement of these APIs. -Search the living backlog for obsolete signatures and remove completed work. - -Verify independent crate feature builds, default and backend-free refusal, -ownership/cancellation tests, docs, and migrated callers on the exact release -candidate. Use the Nix check/test recipes and platform CI; record skipped -hardware cases separately from compilation. Confirm package manifests do not -reintroduce mandatory native compilation through feature unification. - -No published FFI/C layout or wire break is authorized by this stabilization -line. Any newly discovered break outside these four 0.0.x crates needs its own -maintainer decision and appropriate branch. No performance claim is accepted -without measurements; unresolved optimization work alone does not block 0.1. - -Public API/wire impact: review only. diff --git a/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md b/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md index 7afd14d5b5..38e28ccf8d 100644 --- a/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md +++ b/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md @@ -16,37 +16,36 @@ off the same way. Today audio reserves but never follows. The capture driver takes the reservation once the layout reveals the encoded rate -(`rs/moq-audio/src/encode/capture.rs:340`, `:471-474`) and nothing reads its +(`rs/moq-audio/src/encode/capture.rs`) and nothing reads its grant; `Options::bandwidth` documents that -(`rs/moq-audio/src/encode/producer.rs:49-62`). Video already follows through -`rate::Control` (`rs/moq-video/src/encode/producer.rs:467-476`). +(`rs/moq-audio/src/encode/producer.rs`). Video already follows through +`moq_mux::rate::Control` (`rs/moq-video/src/encode/producer.rs`). - Consume `moq_mux::rate`, keeping one shared policy; this quest adds audio adaptation without growing a second implementation. - The follow loop lives in `moq_audio::encode::Producer`, not the capture - driver: `Producer::new` (`producer.rs:255`) already takes `Options` with the - allocator (`:62`), so it reserves the configured bitrate against + driver: `Producer::new` already takes `Options` with the + allocator, so it reserves the configured bitrate against `self.demand()`, holds a `Reservation::consumer()` plus a `rate::Control`, and feeds each grant to `Encoder::set_bitrate` - (`rs/moq-audio/src/encode/encoder.rs:411`). Capture and moq-ffi - (`rs/moq-ffi/src/audio.rs:254`) build the Producer, so both adapt without + (`rs/moq-audio/src/encode/encoder.rs`). Capture and moq-ffi + (`rs/moq-ffi/src/audio.rs`) build the Producer, so both adapt without their own loop; the capture driver's `_reservation` goes away. Public entry - points for a manual ceiling stay `Producer::set_bitrate` (`producer.rs:295`) + points for a manual ceiling stay `Producer::set_bitrate` and `Encoder::set_bitrate`. - Floor: `set_opus_bitrate` refuses anything outside `opus::bitrate_floor(codec_rate, frame_size).max(500)` to - `300_000 * channels` (`encoder.rs:345-346`, - `rs/moq-audio/src/opus.rs:137-143`). `Policy::min` defaults to a tenth of + `300_000 * channels` (`encoder.rs`, `rs/moq-audio/src/opus.rs`). `Policy::min` defaults to a tenth of the ceiling (`rs/moq-mux/src/rate.rs`); for Opus it is the codec floor, so a grant below it clamps there and never errors. The reservation's ceiling is the configured bitrate; only the policy target moves. - PCM: `pcm::bitrate(sample_rate, channels)` is `pub(crate)` - (`rs/moq-audio/src/pcm.rs:9`), `Config::bitrate` is refused for it - (`encoder.rs:269-272`) and so is `set_bitrate` (`encoder.rs:412-414`). A PCM + (`rs/moq-audio/src/pcm.rs`), `Settings::bitrate` is refused for it + and so is `set_bitrate` (`encoder.rs`). A PCM Producer reserves its fixed rate and runs no policy. That is the same reserve-only usage passthrough imports use, so nothing new is added for it. - Priority is unchanged: `PRIORITY` puts audio at 80 and video at 60 - (`rs/hang/src/catalog/priority.rs:21-26`), so the allocator fills audio's + (`rs/hang/src/catalog/priority.rs`), so the allocator fills audio's reservation before video sees a bit and audio is squeezed only once the link cannot carry audio alone. Worth doing for that tail, not worth blocking on. diff --git a/quest/next/audio-jitter-target/README.md b/quest/next/audio-jitter-target/README.md index 460825a0b3..2535d3d799 100644 --- a/quest/next/audio-jitter-target/README.md +++ b/quest/next/audio-jitter-target/README.md @@ -48,8 +48,8 @@ the branch still gets wrong is recorded in the first gap between observed timestamps and the rise is immediate and unclamped, so a tune-in across a stale group sets the target to seconds. -Native has no jitter buffer at all. `rs/moq-audio`'s decode `Config` -(`rs/moq-audio/src/decode/decoder.rs:60-80`) carries `max_age`, how far +Native has no jitter buffer at all. `rs/moq-audio`'s `decode::Options` +(`rs/moq-audio/src/decode/consumer.rs`) carries `max_age`, how far playback may drift from the live edge before skipping a stalled group, and `start`, where to begin on a track that already holds groups. Nothing pads the buffer against uneven arrivals. diff --git a/quest/next/audio-jitter-target/native.md b/quest/next/audio-jitter-target/native.md index 6ad5517f50..a58c43b3c9 100644 --- a/quest/next/audio-jitter-target/native.md +++ b/quest/next/audio-jitter-target/native.md @@ -13,18 +13,18 @@ Video keeps its own path. ## Plan -`rs/moq-audio` has no jitter buffer at all. `decode::Config` -(`rs/moq-audio/src/decode/decoder.rs:60-80`) carries `max_age`, how far +`rs/moq-audio` has no jitter buffer at all. `decode::Options` +(`rs/moq-audio/src/decode/consumer.rs`) carries `max_age`, how far playback may drift from the live edge before skipping a stalled group, applied -to the subscription and clamped to the track's retention in -`decode/consumer.rs:113`, and `start`, where to begin on a track that already -holds groups. Neither adds latency: the consumer skips only when newer data is +to the subscription and clamped to the track's retention, and `start`, where +to begin on a track that already holds groups; `decode::Config` selects the +backend only. Neither adds latency: the consumer skips only when newer data is already that far ahead. - Measure arrivals at the same point the browser does, on the container consumer before the age budget can skip a group, so both languages estimate from the same observation. -- Add the knob to `decode::Config`, additive on the `#[non_exhaustive]` struct, +- Add the knob to `decode::Options`, additive on the `#[non_exhaustive]` struct, so it targets `main`. The spec quest settles the final name and shape before this implementation starts; `delay` is the recommendation, matching the browser while `max_age` remains the live-edge skip budget. Do not reopen or diff --git a/quest/next/capture-alsa-link.md b/quest/next/capture-alsa-link.md index 87548ac293..57bcf399d0 100644 --- a/quest/next/capture-alsa-link.md +++ b/quest/next/capture-alsa-link.md @@ -17,13 +17,8 @@ vaapi/nvidia pattern and load the system library at runtime instead, falling through to the next host when it is missing, so a build with the feature on still links and starts driverless. -The current feature matrix is the contract to change: `capture` and -`playback` pull cpal with ALSA always linked, while -`pipewire = ["cpal/pipewire"]` and `pulseaudio = ["cpal/pulseaudio"]` activate -cpal without either. Make the host flags require `capture` or `playback` so -they never activate cpal alone, and document the supported combinations in -the moq-audio feature table with the same change. If cpal cannot load ALSA at -runtime in-tree, split that half into its own upstream quest holding the cpal +`capture` and `playback` pull cpal with ALSA always linked; the host flags +alone no longer activate it. If cpal cannot load ALSA at runtime in-tree, split that half into its own upstream quest holding the cpal release as a plain-text `Required` condition, and this quest requires it. Verify by building in the Nix shell, then running the shipped binary on a diff --git a/quest/next/jitter-flush-clock.md b/quest/next/jitter-flush-clock.md index 7a751ba47a..d9885dca89 100644 --- a/quest/next/jitter-flush-clock.md +++ b/quest/next/jitter-flush-clock.md @@ -40,8 +40,8 @@ clock-free estimates from the media span of each emitted batch. - Enforce the never-lower rule at the publisher, not only by convention. `js/publish/src/catalog.ts:25-29` already refuses a decrease and a zero for audio and video renditions; Rust does not. Add `moq_mux::Error::JitterDecreased` - and return it from `Rendition::set`, `Rendition::update`, and - `Rendition::estimate` (`rs/moq-mux/src/catalog/tracks.rs:444`, `:497`, `:486`) + and return it from `Rendition::set`, `Rendition::replace`, and + `Rendition::estimate` (`rs/moq-mux/src/catalog/tracks.rs`) when the resolved jitter would drop below what the catalog already carries. Extend the `MillisCeil` ceiling serialization (`rs/hang/src/catalog/millis.rs`) to `TextConfig` (`rs/hang/src/catalog/text/mod.rs:124`), which audio and video diff --git a/quest/next/keyframe-trigger.md b/quest/next/keyframe-trigger.md index 2074881c35..01407a2f61 100644 --- a/quest/next/keyframe-trigger.md +++ b/quest/next/keyframe-trigger.md @@ -3,19 +3,20 @@ ## Goal An application publishing through the built-in capture path can ask for a -keyframe. Every encoder backend already forces an IDR and it is tested, but -nothing above the backend can reach it. +keyframe. `Encoder::cut()`, `Sink::cut()`, and the ffi/libmoq `cut` already +force one, refusing with `CutUnsupported` when a backend cannot, but the +turnkey capture paths have no way in. ## Plan -`Encoder::encode` takes `keyframe: bool` and each backend honors it (NVENC via -the `FORCEIDR` picture flag with `repeatSPSPPS` so the IDR carries its -parameter sets, deliberately not `pictureType` which `enablePTD` ignores; -openh264, VAAPI, VideoToolbox and Media Foundation the same way). What is -missing is a caller-facing trigger: +`Backend::encode(frame, cut)` honors a cut (NVENC via the `FORCEIDR` picture +flag with `repeatSPSPPS` so the IDR carries its parameter sets, deliberately +not `pictureType` which `enablePTD` ignores; openh264, VAAPI, VideoToolbox and +Media Foundation the same way) and `can_cut()` answers at open whether it can. +What is missing is a caller-facing trigger on the turnkey paths: -- `publish_capture` forces a keyframe on the first frame and otherwise rides - the backend's GOP cadence, with no way in. +- `publish_capture` relies on every backend opening with a keyframe and + otherwise rides the GOP cadence; its `Options` carry no trigger. - `js/publish`'s encode path already calls `encoder.encode(frame, { keyFrame })`, but `lastKeyframe` is a closure-local `let` with no external trigger. `Config.keyframeInterval` is cadence, not on demand. diff --git a/quest/next/nvenc-recovery.md b/quest/next/nvenc-recovery.md index 6d71b49db2..c65cb61035 100644 --- a/quest/next/nvenc-recovery.md +++ b/quest/next/nvenc-recovery.md @@ -7,8 +7,8 @@ panic, and a rejected rate change leaves the last accepted settings intact. ## Plan -Registration can succeed before mapping fails in safe/buffer.rs, and several -destructors call expect on driver cleanup. Session::reconfigure mutates retained +Registration rollback and non-panicking destructors landed with the ownership +work (#3834, #3835, #3838). Session::reconfigure still mutates retained bitrate/VBV fields before the driver accepts the change; a rejected zero-rate update can corrupt the basis of the next proportional update. diff --git a/quest/next/play-audio-rendition-gap.md b/quest/next/play-audio-rendition-gap.md index eb368a6e04..eab0d8da4d 100644 --- a/quest/next/play-audio-rendition-gap.md +++ b/quest/next/play-audio-rendition-gap.md @@ -30,7 +30,7 @@ Worth weighing, and the choice is A/V policy: largest change to `Playback`. - Hand the replacement the retired sink rather than opening a new one, so the ring is already at depth. Keeps one task per kind, but only works when the - rate and channel count match. + sample rate and `Layout` match. Retiring a rendition mid-track is the case to reproduce: a transcode ladder resizing under a source that changed resolution is the one that happens in diff --git a/quest/next/qos/stats/encoder-feedback.md b/quest/next/qos/stats/encoder-feedback.md index c471316267..31e71a6e1a 100644 --- a/quest/next/qos/stats/encoder-feedback.md +++ b/quest/next/qos/stats/encoder-feedback.md @@ -11,8 +11,9 @@ prefix. Keyframe requests stay out. ## Plan -- `encode::Config` in `rs/moq-video` and `rs/moq-audio` gains - `feedback: Option`, a handle built from an +- `encode::Options` in `rs/moq-video` and `rs/moq-audio` (the producer + options, beside `bandwidth`) gains `feedback: Option`, + a handle built from an `origin::Consumer` and a prefix: it consumes announcements under the prefix, keeps one `moq_stats::Consumer` per `.stats` broadcast requesting `/subscriber.json`, and folds the reports into one @@ -22,7 +23,8 @@ prefix. Keyframe requests stay out. that goes backwards is a restarted viewer and resets that baseline. Viewers that stop reporting age out on the stats interval, so one stall long ago never lowers the target forever. -- `rate.rs` takes that signal beside the bandwidth estimate: a stalled share +- `moq_mux::rate::Control` takes that signal beside the bandwidth estimate: + a stalled share above a threshold steps the target down like a bandwidth drop, recovery follows the existing attack curve, and the estimate stays the ceiling. Audio follows the same signal with its narrower ladder. diff --git a/rs/moq-audio/CHANGELOG.md b/rs/moq-audio/CHANGELOG.md index b2763125d8..8e6ee9cb47 100644 --- a/rs/moq-audio/CHANGELOG.md +++ b/rs/moq-audio/CHANGELOG.md @@ -15,6 +15,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 replace ambiguous channel counts with `Layout` and add explicit decoder backend selection. - [**breaking**] `encode::Producer::finish` borrows (`&mut self`) instead of consuming, so a later `abort(self)` can still run after a clean end. Writes after finish fail with `Closed`. +- [**breaking**] `encode::Producer::track` is replaced by the watch-only `demand()`, + `publish_capture` takes `PublicationOptions`, `Resampler` is no longer exported, and + `Frame` and `encode::Encoded` are non-exhaustive with `new` constructors. +- [**breaking**] The `fec` flag is removed from `encode::Settings` and `Options`; it never + produced redundancy. +- [**breaking**] `aec::Canceller` is `aec::Control`, `playback::Engine::canceller` returns + `Result`, and `capture::Config::aec` takes a `Control`. An engine owns one AEC reference and a + control attaches to one live microphone; conflicts fail with `Error::Busy`. +- [**breaking**] `playback::Sink::write` returns `playback::Write`, reporting accepted and + dropped input sample frames instead of silently discarding overflow. ## [0.0.25](https://github.com/moq-dev/moq/compare/moq-audio-v0.0.24...moq-audio-v0.0.25) - 2026-09-17 diff --git a/rs/moq-mux/CHANGELOG.md b/rs/moq-mux/CHANGELOG.md index dd03c3b711..ffd20a8b3d 100644 --- a/rs/moq-mux/CHANGELOG.md +++ b/rs/moq-mux/CHANGELOG.md @@ -48,6 +48,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- `rate::{Policy, Control}`, the media bitrate policy audio, video, and transcode senders share + (moved from `moq_video::encode::rate`). An inverted policy starts at its ceiling. - Propagate rendition labels through single-track media imports. - `Clock::with_wall` / `Clock::new_at` for imports and synthetic sources, `Clock::source` translating a source's own zero onto the broadcast mapping (first frame anchors live, a diff --git a/rs/moq-nvenc/CHANGELOG.md b/rs/moq-nvenc/CHANGELOG.md index ba1e16b2f1..56fa2ad1d3 100644 --- a/rs/moq-nvenc/CHANGELOG.md +++ b/rs/moq-nvenc/CHANGELOG.md @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- [**breaking**] `Encoder::load` validates the driver table and returns a non-exhaustive + `LoadError`, as does `Encoder::initialize_with_cuda`; the public `ENCODE_API` and `EncodeAPI` + are removed and loading never panics. +- [**breaking**] Buffers and registrations own the encoder lifetime, `Session::encode_picture` + consumes its buffers and returns a `Submission` that holds them through completion, and raw + registration and configuration entry points are `unsafe`. `EncoderOutput` and the raw picture + parameters are removed. +- Failed resource mapping unregisters the registration before releasing its owner; + `EncodeError::cleanup` exposes a rollback failure beside the primary error. + ## [0.0.5](https://github.com/moq-dev/moq/compare/moq-nvenc-v0.0.4...moq-nvenc-v0.0.5) - 2026-09-13 ### Fixed diff --git a/rs/moq-transcode/CHANGELOG.md b/rs/moq-transcode/CHANGELOG.md index 0f3e1bc3a5..4a163f2909 100644 --- a/rs/moq-transcode/CHANGELOG.md +++ b/rs/moq-transcode/CHANGELOG.md @@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 remove the deprecated `nvenc` and `nvdec` feature aliases. - Preserve a source catalog's `archive` entry on the derivative instead of synthesizing a timeline for the transcode output. +- [**breaking**] `Config::resize` is `moq_video::resize::Config` with a + native-or-CPU `output`, replacing the acceleration knob; `moq transcode + --resize-acceleration` becomes `--frames native|cpu`. +- [**breaking**] Source frame rates are `Option`, unknown + when the source catalog carries none. ## [0.0.19](https://github.com/moq-dev/moq/compare/moq-transcode-v0.0.18...moq-transcode-v0.0.19) - 2026-09-17 diff --git a/rs/moq-transcode/README.md b/rs/moq-transcode/README.md index a865bb5956..fc539bcfd6 100644 --- a/rs/moq-transcode/README.md +++ b/rs/moq-transcode/README.md @@ -44,10 +44,8 @@ let mut config = moq_transcode::Config::default(); // renditions are referenced through its parent. config.source = Some(moq_net::path::RelativeOwned::from(".".to_string())); -let output = origin.create_broadcast( - format!("{path}/transcode.hang"), - moq_net::broadcast::Route::new().with_announce(true), -)?; +let output = origin.create_broadcast(format!("{path}/transcode.hang"))?; +output.announce(Default::default())?; moq_transcode::run(source, output, config).await?; ``` diff --git a/rs/moq-transcode/src/lib.rs b/rs/moq-transcode/src/lib.rs index 4d1b335ba4..a9cda09c0d 100644 --- a/rs/moq-transcode/src/lib.rs +++ b/rs/moq-transcode/src/lib.rs @@ -19,7 +19,9 @@ //! Linux, VideoToolbox on macOS, Media Foundation on Windows), with the default //! `openh264` feature providing H.264 software fallback. On an NVIDIA GPU the whole pipeline is //! GPU-resident: NVDEC decodes and scales in hardware and NVENC encodes the -//! CUDA frame in place, with no CPU copies. Other decoders scale on the CPU. +//! CUDA frame in place, with no CPU copies. macOS and Windows also resize on +//! the GPU; set [`Config::resize`]'s output to `Output::Cpu` to decode to CPU +//! pixels and resize there. pub mod active; pub mod ladder; diff --git a/rs/moq-video/CHANGELOG.md b/rs/moq-video/CHANGELOG.md index b4cac02915..abcadd022f 100644 --- a/rs/moq-video/CHANGELOG.md +++ b/rs/moq-video/CHANGELOG.md @@ -32,6 +32,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `max_age` beside the decoder config. `gpu_frames` and `resize::Acceleration` are replaced by `moq_video::Output`, which `resize::Config::output` shares. +- [**breaking**] `encode::Config::gop` is the non-exhaustive `encode::Gop` + enum (`Gop::Keyframe { interval }`, `Gop::keyframe_every(duration, rate)`) + instead of a bare `u32`; an interval of 0 is refused with `Error::InvalidGop`. +- [**breaking**] `Encoder::keyframe` and `Sink::keyframe` are `cut()` and + return `Result`; a backend that cannot force a group boundary refuses at + open with `Error::CutUnsupported`. +- [**breaking**] Frame rates are the reduced rational `moq_video::Rate` + (`encode::Config::new(width, height, rate)`, `capture::Config::framerate`), + and capture reads return timestamped `Frame`s stamped before queue + replacement rather than bare `Surface`s. +- [**breaking**] `encode::rate::{Policy, Control}` moved to `moq_mux::rate`. +- [**breaking**] The synchronous `encode::Encoder` and `decode::Decoder` are + `!Send` and `!Sync`; the async `Sink` and `Consumer` stay `Send` and own + codec execution. +- [**breaking**] `RateError` is non-exhaustive. ## [0.0.25](https://github.com/moq-dev/moq/compare/moq-video-v0.0.24...moq-video-v0.0.25) - 2026-09-17 diff --git a/rs/moq-video/DESIGN-native-codecs.md b/rs/moq-video/DESIGN-native-codecs.md index dcfdc6b1ba..3ee8a755c7 100644 --- a/rs/moq-video/DESIGN-native-codecs.md +++ b/rs/moq-video/DESIGN-native-codecs.md @@ -224,17 +224,18 @@ objc2-core-media = "..." objc2-core-video = "..." [target.'cfg(target_os = "linux")'.dependencies] -# Hardware encoders are always-on for Linux (cfg-gated, no feature). Both -# dlopen their drivers at runtime, so they link on a GPU-less builder. -moq-nvenc = { path = "../moq-nvenc" } # in-tree fork, dlopen-only -moq-vaapi = "0.0.2" # standalone; vendored cros-libva + cros-codecs +# NVENC rides the default-on `nvidia` feature and VAAPI the opt-in `vaapi` +# feature. Both dlopen their drivers at runtime, so they link on a GPU-less builder. +moq-nvenc = { path = "../moq-nvenc", optional = true } # in-tree fork, dlopen-only +moq-vaapi = { version = "0.0.2", optional = true } # standalone; vendored cros-libva + cros-codecs [dependencies] openh264 = { version = "...", optional = true } # default software fallback ``` -Hardware encoders are always-on (VideoToolbox on macOS, Media Foundation on -Windows, NVENC + VAAPI on Linux); the runtime fallback chain skips whichever +Hardware encoders are cfg-gated on macOS and Windows (VideoToolbox, Media +Foundation) and feature-gated on Linux (NVENC behind the default-on `nvidia`, +VAAPI behind the opt-in `vaapi`); the runtime fallback chain skips whichever driver is absent. None is a build-time hard dep on the driver, so the binary still builds and runs on a box with no GPU. The default `openh264` feature provides the software fallback, so a GPU-less box still encodes (it's also what @@ -436,7 +437,7 @@ licenses (drm, drm-fourcc, etc.) once the vaapi graph resolves. For the "single binary reaches the GPU at runtime" goal, NVENC must not hard-link the driver. The stock `nvidia-video-codec-sdk` emits `cargo:rustc-link-lib=nvidia-encode` / `nvcuvid`, which would make an -`--features nvenc` binary (a) impossible to link on a GPU-less builder and (b) +`--features nvidia` binary (a) impossible to link on a GPU-less builder and (b) fail to even load on a machine without the NVIDIA driver (`DT_NEEDED libnvidia-encode.so.1`), before `backend::open`'s software fallback could run. @@ -450,7 +451,7 @@ So `nvenc` dlopens everything at runtime, like `cudarc` does for CUDA: `GetMaxSupportedVersion`), and the fork resolves those two via `dlopen` instead of linking them (there is no `build.rs`, so nothing links). -Result, verified on a GPU-less Linux box: `--features nvenc` builds, links, and the +Result, verified on a GPU-less Linux box: `--features nvidia` builds, links, and the test suite runs and passes (NVENC unavailable -> falls back to openh264), and the binary has no `libnvidia-encode` / `libcuda` `DT_NEEDED`. So one portable `moq-cli` can carry NVENC and use it only where the driver is present. diff --git a/rs/moq-video/README.md b/rs/moq-video/README.md index f520e9ab4e..f2aafe02ed 100644 --- a/rs/moq-video/README.md +++ b/rs/moq-video/README.md @@ -45,8 +45,9 @@ let mut config = moq_video::capture::Config::default(); config.source = moq_video::capture::Source::Display(None); let mut capture = moq_video::capture::open(&config).await?; -while let Some(surface) = capture.read().await? { - // Encode, render, or inspect the newest captured surface. +while let Some(frame) = capture.read().await? { + // Encode, render, or inspect the newest captured frame; `frame.surface` + // holds the pixels and `frame.timestamp` the capture time. } ``` diff --git a/rs/moq-video/src/rate.rs b/rs/moq-video/src/rate.rs index 5e2e387fef..a434a13fc5 100644 --- a/rs/moq-video/src/rate.rs +++ b/rs/moq-video/src/rate.rs @@ -9,6 +9,7 @@ pub const MAX_FRAMES_PER_SECOND: u32 = 1_000_000; /// An invalid video frame rate. #[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)] +#[non_exhaustive] pub enum RateError { /// The numerator or denominator was zero. #[error("frame rate numerator and denominator must be non-zero")]