diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 1efcaeea54..70f1493158 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -136,8 +136,9 @@ moq ... transcode --rung 720:2500000 --rung 360:600000 --encoder nvenc --decoder Publishes `cam.hang/transcode.hang` whose catalog references the source's rendition and adds lower rungs that are decoded and encoded only while someone -watches them. On NVIDIA the whole pipeline stays on the GPU. Requires the -`transcode` feature. +watches them. On NVIDIA the whole pipeline stays on the GPU; `--frames cpu` +forces decoded frames into CPU memory instead of the default `native`. +Requires the `transcode` feature. The ladder is sized against the source picture and follows it, so a source that changes resolution mid-stream (a window capture renegotiated by a resize, a diff --git a/doc/lib/rs/moq-video.md b/doc/lib/rs/moq-video.md index 1a8d80af4a..0d6d5007b7 100644 --- a/doc/lib/rs/moq-video.md +++ b/doc/lib/rs/moq-video.md @@ -20,7 +20,7 @@ ffmpeg, no GStreamer, no system codec to install. Highlights: -- **Automatic backend selection**, hardware first. Linux GPU libraries are `dlopen`ed at runtime, so one binary starts anywhere and warns when it falls back to software. openh264 is statically linked as the H.264 fallback; H.265 is hardware-only; AV1 decodes via NVDEC. The VAAPI encoder is compile-verified but not yet validated on hardware. +- **Automatic backend selection**, hardware first. Linux GPU libraries are `dlopen`ed at runtime, so one binary starts anywhere and warns when it falls back to software. openh264 (the default-on `openh264` feature) is statically linked as the H.264 fallback; H.265 is hardware-only; AV1 decodes via NVDEC. The VAAPI encoder is compile-verified but not yet validated on hardware. - **Publish on demand.** `encode::publish_capture` advertises the track up front and opens the camera only while someone subscribes. - **GPU ownership where the platform allows.** Matching codec backends consume their native GPU surfaces directly. The renderer imports `CVPixelBuffer` and supported DMA-BUF formats. Linux/NVIDIA producers can import dedicated Vulkan RGBA8 slots into CUDA with timeline-semaphore ordering and completion-driven slot return. Vulkan/CUDA surfaces deliberately have no CPU pixel fallback; other surfaces use the typed `Surface::into_i420()` and configured `Surface::to_rgba(config)` when needed. - **Live bitrate control** where the selected backend supports it, without forcing a keyframe. An unsupported backend keeps its opening rate. diff --git a/quest/future/audio-loss-recovery.md b/quest/future/audio-loss-recovery.md index 5c6d1079e2..d8594d96b4 100644 --- a/quest/future/audio-loss-recovery.md +++ b/quest/future/audio-loss-recovery.md @@ -3,7 +3,8 @@ ## Goal Decide whether a concrete MoQ audio consumer benefits from in-band Opus FEC, -with a tested loss/latency policy before exposing a replacement public option. +with a tested loss/latency policy before exposing any additive option on +`encode::Settings`. This does not block 0.1. ## Plan diff --git a/quest/future/intra-refresh/encode-config.md b/quest/future/intra-refresh/encode-config.md index 8bc9f9143c..42cfbc81ff 100644 --- a/quest/future/intra-refresh/encode-config.md +++ b/quest/future/intra-refresh/encode-config.md @@ -25,9 +25,10 @@ without replacing an API after 0.1. error for `Refresh` (supported or refused, never a silent fallback to keyframes). The test-only probe backend accepts it so the producer logic is testable without hardware. -- Group boundaries in refresh mode come from counting: backends report - `keyframe = false` for a sweep start, so the producer marks the first frame - of each cycle by frame count from the last cut and opens the group there. +- Group boundaries in refresh mode come from counting: the splitter sees no + IDR at a sweep start (only a recovery-point SEI where the backend emits + one), so the producer marks the first frame of each cycle by frame count + from the last cut and tells the importer to open the group there. A backend reports the sweep length it actually configured, which can be shorter than the cycle (NVENC needs it strictly shorter), and the producer publishes `warmup` as that length over the framerate. diff --git a/quest/future/intra-refresh/v4l2-refresh.md b/quest/future/intra-refresh/v4l2-refresh.md index fc7c411568..a536e88bf6 100644 --- a/quest/future/intra-refresh/v4l2-refresh.md +++ b/quest/future/intra-refresh/v4l2-refresh.md @@ -17,8 +17,8 @@ claiming a boundary that was not encoded. `V4L2_CID_MPEG_VIDEO_CYCLIC_INTRA_REFRESH_MB` as macroblocks per frame derived from the picture size and the cycle) from the sys crate, set `GOP_SIZE` to zero for an unbounded GOP, and refuse the mode when the driver - rejects both controls, unlike the self-disabling `keyframes` fallback used - for forced keyframes. + rejects both controls, the same way a missing `FORCE_KEY_FRAME` control + refuses cuts with `CutUnsupported` at open. - A cut in refresh mode restarts the sweep. If the driver cannot honor it, report rejection through the error path settled in main; this does not independently require a Result from cut(). Never open a group out of phase diff --git a/quest/future/sip-stack.md b/quest/future/sip-stack.md index 4d812ad5dc..261b4a1e24 100644 --- a/quest/future/sip-stack.md +++ b/quest/future/sip-stack.md @@ -21,9 +21,10 @@ consumable by an embedder. - The media leg is plain negotiated RTP: `moq-rtc`/str0m is ICE/DTLS-first and its reusable session internals are crate-private, so it is precedent, not a base. Consider publicizing moq-rtc's codec bridges rather than - duplicating the RTP-to-hang mapping, and reuse `moq-audio`'s Opus and - resampler for the G.711<->Opus transcode (mono 8 kHz; G.711 companding is - new code, nothing in the repository has it). + duplicating the RTP-to-hang mapping, and reuse `moq-audio`'s Opus codec + for the G.711<->Opus transcode, with rate conversion through + `decode::Output` and `encode::Input` since the resampler is private (mono + 8 kHz; G.711 companding is new code, nothing in the repository has it). - RTP wall-clock normalization off RTCP sender reports, as moq-rtc does. - The embedder decides paths and auth; the crate's API is "answer this INVITE, give me the caller as Opus, take Opus to play" plus diff --git a/quest/future/video-vaapi.md b/quest/future/video-vaapi.md index 7cd8b5348e..864b4b3988 100644 --- a/quest/future/video-vaapi.md +++ b/quest/future/video-vaapi.md @@ -12,8 +12,8 @@ release first. Four gaps, one external dependency. **Decode.** The H.264 decoder landed (moq-vaapi 0.0.4, `decode/backend/vaapi.rs`), -with `decode::Config::gpu_frames` exporting DMA-BUF surfaces the renderer -imports without a download. H.265 decode is still missing, so a Linux box +with the default `decode::Config::output` of `Output::Native` handing out +DMA-BUF surfaces the renderer imports without a download. H.265 decode is still missing, so a Linux box without NVDEC has no hardware path for it. **The encoder.** Ours is a 111-line CPU-only adapter whose own header says it diff --git a/quest/main/README.md b/quest/main/README.md index 3f26121854..70c9d6072a 100644 --- a/quest/main/README.md +++ b/quest/main/README.md @@ -41,8 +41,8 @@ Their package boundaries are explicit: The media crates are also 0.0.x, so their changes target main. Adapt callers in other packages without breaking their published APIs, C layouts, or wire -formats. Do not bump versions as part of these quests. The media review records -when the four crates are ready for a separately requested 0.1 release. +formats. Do not bump versions as part of these quests. The media review +found the four crates ready for a separately requested 0.1 release. Their package boundaries are explicit: @@ -90,7 +90,6 @@ do not add another media abstraction or a renderer crate during stabilization. - [GPU conversion and NVENC](/quest/main/video-gpu-encode.md) - validate the GPU conversion, resize and NVENC path for imported frames on NVIDIA hardware -- [Media release review](/quest/main/media-release-review.md) - verify the settled contracts before separately authorizing 0.1 releases ## Related diff --git a/quest/main/media-release-review.md b/quest/main/media-release-review.md deleted file mode 100644 index 6f3a77b96b..0000000000 --- a/quest/main/media-release-review.md +++ /dev/null @@ -1,27 +0,0 @@ -# [S] Verify the four media contracts before 0.1 - -## Goal - -Record that the agreed audio, video, transcode, and NVENC API changes have -landed, or return concrete remaining blockers, before a separate version-bump -request. Completing this review does not bump or publish packages. - -## Plan - -Review the final public surface and examples after the prerequisite quests. -Check that deferred backend, surround, refresh, color, and performance work can -use the extension points without another planned replacement of these APIs. -Search the living backlog for obsolete signatures and remove completed work. - -Verify independent crate feature builds, default and backend-free refusal, -ownership/cancellation tests, docs, and migrated callers on the exact release -candidate. Use the Nix check/test recipes and platform CI; record skipped -hardware cases separately from compilation. Confirm package manifests do not -reintroduce mandatory native compilation through feature unification. - -No published FFI/C layout or wire break is authorized by this stabilization -line. Any newly discovered break outside these four 0.0.x crates needs its own -maintainer decision and appropriate branch. No performance claim is accepted -without measurements; unresolved optimization work alone does not block 0.1. - -Public API/wire impact: review only. diff --git a/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md b/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md index 7afd14d5b5..38e28ccf8d 100644 --- a/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md +++ b/quest/next/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md @@ -16,37 +16,36 @@ off the same way. Today audio reserves but never follows. The capture driver takes the reservation once the layout reveals the encoded rate -(`rs/moq-audio/src/encode/capture.rs:340`, `:471-474`) and nothing reads its +(`rs/moq-audio/src/encode/capture.rs`) and nothing reads its grant; `Options::bandwidth` documents that -(`rs/moq-audio/src/encode/producer.rs:49-62`). Video already follows through -`rate::Control` (`rs/moq-video/src/encode/producer.rs:467-476`). +(`rs/moq-audio/src/encode/producer.rs`). Video already follows through +`moq_mux::rate::Control` (`rs/moq-video/src/encode/producer.rs`). - Consume `moq_mux::rate`, keeping one shared policy; this quest adds audio adaptation without growing a second implementation. - The follow loop lives in `moq_audio::encode::Producer`, not the capture - driver: `Producer::new` (`producer.rs:255`) already takes `Options` with the - allocator (`:62`), so it reserves the configured bitrate against + driver: `Producer::new` already takes `Options` with the + allocator, so it reserves the configured bitrate against `self.demand()`, holds a `Reservation::consumer()` plus a `rate::Control`, and feeds each grant to `Encoder::set_bitrate` - (`rs/moq-audio/src/encode/encoder.rs:411`). Capture and moq-ffi - (`rs/moq-ffi/src/audio.rs:254`) build the Producer, so both adapt without + (`rs/moq-audio/src/encode/encoder.rs`). Capture and moq-ffi + (`rs/moq-ffi/src/audio.rs`) build the Producer, so both adapt without their own loop; the capture driver's `_reservation` goes away. Public entry - points for a manual ceiling stay `Producer::set_bitrate` (`producer.rs:295`) + points for a manual ceiling stay `Producer::set_bitrate` and `Encoder::set_bitrate`. - Floor: `set_opus_bitrate` refuses anything outside `opus::bitrate_floor(codec_rate, frame_size).max(500)` to - `300_000 * channels` (`encoder.rs:345-346`, - `rs/moq-audio/src/opus.rs:137-143`). `Policy::min` defaults to a tenth of + `300_000 * channels` (`encoder.rs`, `rs/moq-audio/src/opus.rs`). `Policy::min` defaults to a tenth of the ceiling (`rs/moq-mux/src/rate.rs`); for Opus it is the codec floor, so a grant below it clamps there and never errors. The reservation's ceiling is the configured bitrate; only the policy target moves. - PCM: `pcm::bitrate(sample_rate, channels)` is `pub(crate)` - (`rs/moq-audio/src/pcm.rs:9`), `Config::bitrate` is refused for it - (`encoder.rs:269-272`) and so is `set_bitrate` (`encoder.rs:412-414`). A PCM + (`rs/moq-audio/src/pcm.rs`), `Settings::bitrate` is refused for it + and so is `set_bitrate` (`encoder.rs`). A PCM Producer reserves its fixed rate and runs no policy. That is the same reserve-only usage passthrough imports use, so nothing new is added for it. - Priority is unchanged: `PRIORITY` puts audio at 80 and video at 60 - (`rs/hang/src/catalog/priority.rs:21-26`), so the allocator fills audio's + (`rs/hang/src/catalog/priority.rs`), so the allocator fills audio's reservation before video sees a bit and audio is squeezed only once the link cannot carry audio alone. Worth doing for that tail, not worth blocking on. diff --git a/quest/next/audio-jitter-target/README.md b/quest/next/audio-jitter-target/README.md index 460825a0b3..2535d3d799 100644 --- a/quest/next/audio-jitter-target/README.md +++ b/quest/next/audio-jitter-target/README.md @@ -48,8 +48,8 @@ the branch still gets wrong is recorded in the first gap between observed timestamps and the rise is immediate and unclamped, so a tune-in across a stale group sets the target to seconds. -Native has no jitter buffer at all. `rs/moq-audio`'s decode `Config` -(`rs/moq-audio/src/decode/decoder.rs:60-80`) carries `max_age`, how far +Native has no jitter buffer at all. `rs/moq-audio`'s `decode::Options` +(`rs/moq-audio/src/decode/consumer.rs`) carries `max_age`, how far playback may drift from the live edge before skipping a stalled group, and `start`, where to begin on a track that already holds groups. Nothing pads the buffer against uneven arrivals. diff --git a/quest/next/audio-jitter-target/native.md b/quest/next/audio-jitter-target/native.md index 6ad5517f50..a58c43b3c9 100644 --- a/quest/next/audio-jitter-target/native.md +++ b/quest/next/audio-jitter-target/native.md @@ -13,18 +13,18 @@ Video keeps its own path. ## Plan -`rs/moq-audio` has no jitter buffer at all. `decode::Config` -(`rs/moq-audio/src/decode/decoder.rs:60-80`) carries `max_age`, how far +`rs/moq-audio` has no jitter buffer at all. `decode::Options` +(`rs/moq-audio/src/decode/consumer.rs`) carries `max_age`, how far playback may drift from the live edge before skipping a stalled group, applied -to the subscription and clamped to the track's retention in -`decode/consumer.rs:113`, and `start`, where to begin on a track that already -holds groups. Neither adds latency: the consumer skips only when newer data is +to the subscription and clamped to the track's retention, and `start`, where +to begin on a track that already holds groups; `decode::Config` selects the +backend only. Neither adds latency: the consumer skips only when newer data is already that far ahead. - Measure arrivals at the same point the browser does, on the container consumer before the age budget can skip a group, so both languages estimate from the same observation. -- Add the knob to `decode::Config`, additive on the `#[non_exhaustive]` struct, +- Add the knob to `decode::Options`, additive on the `#[non_exhaustive]` struct, so it targets `main`. The spec quest settles the final name and shape before this implementation starts; `delay` is the recommendation, matching the browser while `max_age` remains the live-edge skip budget. Do not reopen or diff --git a/quest/next/capture-alsa-link.md b/quest/next/capture-alsa-link.md index 87548ac293..57bcf399d0 100644 --- a/quest/next/capture-alsa-link.md +++ b/quest/next/capture-alsa-link.md @@ -17,13 +17,8 @@ vaapi/nvidia pattern and load the system library at runtime instead, falling through to the next host when it is missing, so a build with the feature on still links and starts driverless. -The current feature matrix is the contract to change: `capture` and -`playback` pull cpal with ALSA always linked, while -`pipewire = ["cpal/pipewire"]` and `pulseaudio = ["cpal/pulseaudio"]` activate -cpal without either. Make the host flags require `capture` or `playback` so -they never activate cpal alone, and document the supported combinations in -the moq-audio feature table with the same change. If cpal cannot load ALSA at -runtime in-tree, split that half into its own upstream quest holding the cpal +`capture` and `playback` pull cpal with ALSA always linked; the host flags +alone no longer activate it. If cpal cannot load ALSA at runtime in-tree, split that half into its own upstream quest holding the cpal release as a plain-text `Required` condition, and this quest requires it. Verify by building in the Nix shell, then running the shipped binary on a diff --git a/quest/next/jitter-flush-clock.md b/quest/next/jitter-flush-clock.md index 7a751ba47a..d9885dca89 100644 --- a/quest/next/jitter-flush-clock.md +++ b/quest/next/jitter-flush-clock.md @@ -40,8 +40,8 @@ clock-free estimates from the media span of each emitted batch. - Enforce the never-lower rule at the publisher, not only by convention. `js/publish/src/catalog.ts:25-29` already refuses a decrease and a zero for audio and video renditions; Rust does not. Add `moq_mux::Error::JitterDecreased` - and return it from `Rendition::set`, `Rendition::update`, and - `Rendition::estimate` (`rs/moq-mux/src/catalog/tracks.rs:444`, `:497`, `:486`) + and return it from `Rendition::set`, `Rendition::replace`, and + `Rendition::estimate` (`rs/moq-mux/src/catalog/tracks.rs`) when the resolved jitter would drop below what the catalog already carries. Extend the `MillisCeil` ceiling serialization (`rs/hang/src/catalog/millis.rs`) to `TextConfig` (`rs/hang/src/catalog/text/mod.rs:124`), which audio and video diff --git a/quest/next/keyframe-trigger.md b/quest/next/keyframe-trigger.md index 2074881c35..01407a2f61 100644 --- a/quest/next/keyframe-trigger.md +++ b/quest/next/keyframe-trigger.md @@ -3,19 +3,20 @@ ## Goal An application publishing through the built-in capture path can ask for a -keyframe. Every encoder backend already forces an IDR and it is tested, but -nothing above the backend can reach it. +keyframe. `Encoder::cut()`, `Sink::cut()`, and the ffi/libmoq `cut` already +force one, refusing with `CutUnsupported` when a backend cannot, but the +turnkey capture paths have no way in. ## Plan -`Encoder::encode` takes `keyframe: bool` and each backend honors it (NVENC via -the `FORCEIDR` picture flag with `repeatSPSPPS` so the IDR carries its -parameter sets, deliberately not `pictureType` which `enablePTD` ignores; -openh264, VAAPI, VideoToolbox and Media Foundation the same way). What is -missing is a caller-facing trigger: +`Backend::encode(frame, cut)` honors a cut (NVENC via the `FORCEIDR` picture +flag with `repeatSPSPPS` so the IDR carries its parameter sets, deliberately +not `pictureType` which `enablePTD` ignores; openh264, VAAPI, VideoToolbox and +Media Foundation the same way) and `can_cut()` answers at open whether it can. +What is missing is a caller-facing trigger on the turnkey paths: -- `publish_capture` forces a keyframe on the first frame and otherwise rides - the backend's GOP cadence, with no way in. +- `publish_capture` relies on every backend opening with a keyframe and + otherwise rides the GOP cadence; its `Options` carry no trigger. - `js/publish`'s encode path already calls `encoder.encode(frame, { keyFrame })`, but `lastKeyframe` is a closure-local `let` with no external trigger. `Config.keyframeInterval` is cadence, not on demand. diff --git a/quest/next/nvenc-recovery.md b/quest/next/nvenc-recovery.md index 6d71b49db2..c65cb61035 100644 --- a/quest/next/nvenc-recovery.md +++ b/quest/next/nvenc-recovery.md @@ -7,8 +7,8 @@ panic, and a rejected rate change leaves the last accepted settings intact. ## Plan -Registration can succeed before mapping fails in safe/buffer.rs, and several -destructors call expect on driver cleanup. Session::reconfigure mutates retained +Registration rollback and non-panicking destructors landed with the ownership +work (#3834, #3835, #3838). Session::reconfigure still mutates retained bitrate/VBV fields before the driver accepts the change; a rejected zero-rate update can corrupt the basis of the next proportional update. diff --git a/quest/next/play-audio-rendition-gap.md b/quest/next/play-audio-rendition-gap.md index eb368a6e04..eab0d8da4d 100644 --- a/quest/next/play-audio-rendition-gap.md +++ b/quest/next/play-audio-rendition-gap.md @@ -30,7 +30,7 @@ Worth weighing, and the choice is A/V policy: largest change to `Playback`. - Hand the replacement the retired sink rather than opening a new one, so the ring is already at depth. Keeps one task per kind, but only works when the - rate and channel count match. + sample rate and `Layout` match. Retiring a rendition mid-track is the case to reproduce: a transcode ladder resizing under a source that changed resolution is the one that happens in diff --git a/quest/next/qos/stats/encoder-feedback.md b/quest/next/qos/stats/encoder-feedback.md index c471316267..31e71a6e1a 100644 --- a/quest/next/qos/stats/encoder-feedback.md +++ b/quest/next/qos/stats/encoder-feedback.md @@ -11,8 +11,9 @@ prefix. Keyframe requests stay out. ## Plan -- `encode::Config` in `rs/moq-video` and `rs/moq-audio` gains - `feedback: Option`, a handle built from an +- `encode::Options` in `rs/moq-video` and `rs/moq-audio` (the producer + options, beside `bandwidth`) gains `feedback: Option`, + a handle built from an `origin::Consumer` and a prefix: it consumes announcements under the prefix, keeps one `moq_stats::Consumer` per `.stats` broadcast requesting `/subscriber.json`, and folds the reports into one @@ -22,7 +23,8 @@ prefix. Keyframe requests stay out. that goes backwards is a restarted viewer and resets that baseline. Viewers that stop reporting age out on the stats interval, so one stall long ago never lowers the target forever. -- `rate.rs` takes that signal beside the bandwidth estimate: a stalled share +- `moq_mux::rate::Control` takes that signal beside the bandwidth estimate: + a stalled share above a threshold steps the target down like a bandwidth drop, recovery follows the existing attack curve, and the estimate stays the ceiling. Audio follows the same signal with its narrower ladder. diff --git a/rs/moq-audio/CHANGELOG.md b/rs/moq-audio/CHANGELOG.md index b2763125d8..8e6ee9cb47 100644 --- a/rs/moq-audio/CHANGELOG.md +++ b/rs/moq-audio/CHANGELOG.md @@ -15,6 +15,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 replace ambiguous channel counts with `Layout` and add explicit decoder backend selection. - [**breaking**] `encode::Producer::finish` borrows (`&mut self`) instead of consuming, so a later `abort(self)` can still run after a clean end. Writes after finish fail with `Closed`. +- [**breaking**] `encode::Producer::track` is replaced by the watch-only `demand()`, + `publish_capture` takes `PublicationOptions`, `Resampler` is no longer exported, and + `Frame` and `encode::Encoded` are non-exhaustive with `new` constructors. +- [**breaking**] The `fec` flag is removed from `encode::Settings` and `Options`; it never + produced redundancy. +- [**breaking**] `aec::Canceller` is `aec::Control`, `playback::Engine::canceller` returns + `Result`, and `capture::Config::aec` takes a `Control`. An engine owns one AEC reference and a + control attaches to one live microphone; conflicts fail with `Error::Busy`. +- [**breaking**] `playback::Sink::write` returns `playback::Write`, reporting accepted and + dropped input sample frames instead of silently discarding overflow. ## [0.0.25](https://github.com/moq-dev/moq/compare/moq-audio-v0.0.24...moq-audio-v0.0.25) - 2026-09-17 diff --git a/rs/moq-mux/CHANGELOG.md b/rs/moq-mux/CHANGELOG.md index dd03c3b711..ffd20a8b3d 100644 --- a/rs/moq-mux/CHANGELOG.md +++ b/rs/moq-mux/CHANGELOG.md @@ -48,6 +48,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added +- `rate::{Policy, Control}`, the media bitrate policy audio, video, and transcode senders share + (moved from `moq_video::encode::rate`). An inverted policy starts at its ceiling. - Propagate rendition labels through single-track media imports. - `Clock::with_wall` / `Clock::new_at` for imports and synthetic sources, `Clock::source` translating a source's own zero onto the broadcast mapping (first frame anchors live, a diff --git a/rs/moq-nvenc/CHANGELOG.md b/rs/moq-nvenc/CHANGELOG.md index ba1e16b2f1..56fa2ad1d3 100644 --- a/rs/moq-nvenc/CHANGELOG.md +++ b/rs/moq-nvenc/CHANGELOG.md @@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- [**breaking**] `Encoder::load` validates the driver table and returns a non-exhaustive + `LoadError`, as does `Encoder::initialize_with_cuda`; the public `ENCODE_API` and `EncodeAPI` + are removed and loading never panics. +- [**breaking**] Buffers and registrations own the encoder lifetime, `Session::encode_picture` + consumes its buffers and returns a `Submission` that holds them through completion, and raw + registration and configuration entry points are `unsafe`. `EncoderOutput` and the raw picture + parameters are removed. +- Failed resource mapping unregisters the registration before releasing its owner; + `EncodeError::cleanup` exposes a rollback failure beside the primary error. + ## [0.0.5](https://github.com/moq-dev/moq/compare/moq-nvenc-v0.0.4...moq-nvenc-v0.0.5) - 2026-09-13 ### Fixed diff --git a/rs/moq-transcode/CHANGELOG.md b/rs/moq-transcode/CHANGELOG.md index 0f3e1bc3a5..4a163f2909 100644 --- a/rs/moq-transcode/CHANGELOG.md +++ b/rs/moq-transcode/CHANGELOG.md @@ -13,6 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 remove the deprecated `nvenc` and `nvdec` feature aliases. - Preserve a source catalog's `archive` entry on the derivative instead of synthesizing a timeline for the transcode output. +- [**breaking**] `Config::resize` is `moq_video::resize::Config` with a + native-or-CPU `output`, replacing the acceleration knob; `moq transcode + --resize-acceleration` becomes `--frames native|cpu`. +- [**breaking**] Source frame rates are `Option`, unknown + when the source catalog carries none. ## [0.0.19](https://github.com/moq-dev/moq/compare/moq-transcode-v0.0.18...moq-transcode-v0.0.19) - 2026-09-17 diff --git a/rs/moq-transcode/README.md b/rs/moq-transcode/README.md index a865bb5956..fc539bcfd6 100644 --- a/rs/moq-transcode/README.md +++ b/rs/moq-transcode/README.md @@ -44,10 +44,8 @@ let mut config = moq_transcode::Config::default(); // renditions are referenced through its parent. config.source = Some(moq_net::path::RelativeOwned::from(".".to_string())); -let output = origin.create_broadcast( - format!("{path}/transcode.hang"), - moq_net::broadcast::Route::new().with_announce(true), -)?; +let output = origin.create_broadcast(format!("{path}/transcode.hang"))?; +output.announce(Default::default())?; moq_transcode::run(source, output, config).await?; ``` diff --git a/rs/moq-transcode/src/lib.rs b/rs/moq-transcode/src/lib.rs index 4d1b335ba4..a9cda09c0d 100644 --- a/rs/moq-transcode/src/lib.rs +++ b/rs/moq-transcode/src/lib.rs @@ -19,7 +19,9 @@ //! Linux, VideoToolbox on macOS, Media Foundation on Windows), with the default //! `openh264` feature providing H.264 software fallback. On an NVIDIA GPU the whole pipeline is //! GPU-resident: NVDEC decodes and scales in hardware and NVENC encodes the -//! CUDA frame in place, with no CPU copies. Other decoders scale on the CPU. +//! CUDA frame in place, with no CPU copies. macOS and Windows also resize on +//! the GPU; set [`Config::resize`]'s output to `Output::Cpu` to decode to CPU +//! pixels and resize there. pub mod active; pub mod ladder; diff --git a/rs/moq-video/CHANGELOG.md b/rs/moq-video/CHANGELOG.md index b4cac02915..abcadd022f 100644 --- a/rs/moq-video/CHANGELOG.md +++ b/rs/moq-video/CHANGELOG.md @@ -32,6 +32,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `max_age` beside the decoder config. `gpu_frames` and `resize::Acceleration` are replaced by `moq_video::Output`, which `resize::Config::output` shares. +- [**breaking**] `encode::Config::gop` is the non-exhaustive `encode::Gop` + enum (`Gop::Keyframe { interval }`, `Gop::keyframe_every(duration, rate)`) + instead of a bare `u32`; an interval of 0 is refused with `Error::InvalidGop`. +- [**breaking**] `Encoder::keyframe` and `Sink::keyframe` are `cut()` and + return `Result`; a backend that cannot force a group boundary refuses at + open with `Error::CutUnsupported`. +- [**breaking**] Frame rates are the reduced rational `moq_video::Rate` + (`encode::Config::new(width, height, rate)`, `capture::Config::framerate`), + and capture reads return timestamped `Frame`s stamped before queue + replacement rather than bare `Surface`s. +- [**breaking**] `encode::rate::{Policy, Control}` moved to `moq_mux::rate`. +- [**breaking**] The synchronous `encode::Encoder` and `decode::Decoder` are + `!Send` and `!Sync`; the async `Sink` and `Consumer` stay `Send` and own + codec execution. +- [**breaking**] `RateError` is non-exhaustive. ## [0.0.25](https://github.com/moq-dev/moq/compare/moq-video-v0.0.24...moq-video-v0.0.25) - 2026-09-17 diff --git a/rs/moq-video/DESIGN-native-codecs.md b/rs/moq-video/DESIGN-native-codecs.md index dcfdc6b1ba..3ee8a755c7 100644 --- a/rs/moq-video/DESIGN-native-codecs.md +++ b/rs/moq-video/DESIGN-native-codecs.md @@ -224,17 +224,18 @@ objc2-core-media = "..." objc2-core-video = "..." [target.'cfg(target_os = "linux")'.dependencies] -# Hardware encoders are always-on for Linux (cfg-gated, no feature). Both -# dlopen their drivers at runtime, so they link on a GPU-less builder. -moq-nvenc = { path = "../moq-nvenc" } # in-tree fork, dlopen-only -moq-vaapi = "0.0.2" # standalone; vendored cros-libva + cros-codecs +# NVENC rides the default-on `nvidia` feature and VAAPI the opt-in `vaapi` +# feature. Both dlopen their drivers at runtime, so they link on a GPU-less builder. +moq-nvenc = { path = "../moq-nvenc", optional = true } # in-tree fork, dlopen-only +moq-vaapi = { version = "0.0.2", optional = true } # standalone; vendored cros-libva + cros-codecs [dependencies] openh264 = { version = "...", optional = true } # default software fallback ``` -Hardware encoders are always-on (VideoToolbox on macOS, Media Foundation on -Windows, NVENC + VAAPI on Linux); the runtime fallback chain skips whichever +Hardware encoders are cfg-gated on macOS and Windows (VideoToolbox, Media +Foundation) and feature-gated on Linux (NVENC behind the default-on `nvidia`, +VAAPI behind the opt-in `vaapi`); the runtime fallback chain skips whichever driver is absent. None is a build-time hard dep on the driver, so the binary still builds and runs on a box with no GPU. The default `openh264` feature provides the software fallback, so a GPU-less box still encodes (it's also what @@ -436,7 +437,7 @@ licenses (drm, drm-fourcc, etc.) once the vaapi graph resolves. For the "single binary reaches the GPU at runtime" goal, NVENC must not hard-link the driver. The stock `nvidia-video-codec-sdk` emits `cargo:rustc-link-lib=nvidia-encode` / `nvcuvid`, which would make an -`--features nvenc` binary (a) impossible to link on a GPU-less builder and (b) +`--features nvidia` binary (a) impossible to link on a GPU-less builder and (b) fail to even load on a machine without the NVIDIA driver (`DT_NEEDED libnvidia-encode.so.1`), before `backend::open`'s software fallback could run. @@ -450,7 +451,7 @@ So `nvenc` dlopens everything at runtime, like `cudarc` does for CUDA: `GetMaxSupportedVersion`), and the fork resolves those two via `dlopen` instead of linking them (there is no `build.rs`, so nothing links). -Result, verified on a GPU-less Linux box: `--features nvenc` builds, links, and the +Result, verified on a GPU-less Linux box: `--features nvidia` builds, links, and the test suite runs and passes (NVENC unavailable -> falls back to openh264), and the binary has no `libnvidia-encode` / `libcuda` `DT_NEEDED`. So one portable `moq-cli` can carry NVENC and use it only where the driver is present. diff --git a/rs/moq-video/README.md b/rs/moq-video/README.md index f520e9ab4e..f2aafe02ed 100644 --- a/rs/moq-video/README.md +++ b/rs/moq-video/README.md @@ -45,8 +45,9 @@ let mut config = moq_video::capture::Config::default(); config.source = moq_video::capture::Source::Display(None); let mut capture = moq_video::capture::open(&config).await?; -while let Some(surface) = capture.read().await? { - // Encode, render, or inspect the newest captured surface. +while let Some(frame) = capture.read().await? { + // Encode, render, or inspect the newest captured frame; `frame.surface` + // holds the pixels and `frame.timestamp` the capture time. } ``` diff --git a/rs/moq-video/src/rate.rs b/rs/moq-video/src/rate.rs index 5e2e387fef..a434a13fc5 100644 --- a/rs/moq-video/src/rate.rs +++ b/rs/moq-video/src/rate.rs @@ -9,6 +9,7 @@ pub const MAX_FRAMES_PER_SECOND: u32 = 1_000_000; /// An invalid video frame rate. #[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)] +#[non_exhaustive] pub enum RateError { /// The numerator or denominator was zero. #[error("frame rate numerator and denominator must be non-zero")]