Skip to content

docs(quest): plan media API stabilization before 0.1 - #3803

Merged
kixelated merged 2 commits into
mainfrom
codex/audit-zero-packages
Sep 21, 2026
Merged

kixelated merged 2 commits into
mainfrom
codex/audit-zero-packages

Conversation

@kixelated

@kixelated kixelated commented Sep 20, 2026

Copy link
Copy Markdown
Collaborator

Problem

The public contracts of moq-audio, moq-video, moq-transcode, and moq-nvenc should be settled before their first 0.1 releases. Existing backlog items mixed those API replacements with later backend, optimization, and hardware work, which would either delay stabilization or require another breaking release.

main now also uses m0 for the Pronto GPU path, so this plan must join that immediate-priority graph without replacing its existing quests.

Approach

Add 14 focused media API quests and a final release-review gate to the existing m0 questline. Preserve the three Pronto GPU quests as an independent m0 deliverable, under the combined m0: immediate media scope. Keep API-preserving correctness, performance, codec, and hardware-study work in m2/m3, update their dependencies, and remove the obsolete decoder-drain quest because backend flush and consumer/transcode draining already landed.

The audit covers configuration, optional native compilation, frames and timing, group structure, codec thread ownership, NVENC resource ownership and fallible loading, publication authority, AEC ownership, playback outcomes, and controls that currently promise behavior they do not provide. It is based on source inspection, not new hardware or benchmark certification.

Public API impact by package

This PR is planning-only and changes no executable API today. The quests plan these pre-0.1 Rust changes:

  • moq-audio 0.0.25: separate source PCM layout/rate from codec settings and subscription/output policy; replace channel counts with an extensible layout contract; establish decoder backend-selection construction; remove ineffective fec; replace Producer::track with demand authority; make the standalone Resampler private; simplify publish_capture around PublicationOptions; split exclusive AEC attachment from cloneable controls; return a typed accepted/dropped frame outcome from playback writes; and make frame/packet construction extensible.
  • moq-video 0.0.25: consolidate conversion entry points and return typed pixels with retained size/color; make frame and encoded records constructor-based and extensible; separate decoder output from consumer subscription policy with explicit native-or-CPU output; make direct synchronous codecs thread-confined while worker-owned sinks remain async; preserve capture timestamps and rational rates; replace integer GOP configuration with an extensible group contract and consistently rename forced boundaries to cut; make OpenH264 optional-but-default, rendering opt-in, and remove the nvenc/nvdec feature aliases; remove encode::rate after its shared replacement exists.
  • moq-transcode 0.0.19: adopt the rational rate, output, group/cut, and feature contracts in its public configuration and observations; remove the nvenc/nvdec aliases; preserve the validated Ladder, packetizing behavior, and coalescing active cursor.
  • moq-nvenc 0.0.5: narrow and seal the safe facade around real consumers; encode resource, session, configuration, and completion lifetimes in owned types; make unsafe escape obligations explicit; and replace panicking driver/function-table initialization with fallible loading and extensible public errors.
  • moq-mux 0.9.16: add only the shared rate::{Policy, Control} namespace. Existing published exports remain source-compatible.
  • moq-ffi, libmoq, and language bindings: implementation callers adapt internally, but published signatures, record layouts, channel-count conventions, sentinel behavior, and C layouts remain unchanged.

The cut rejection point remains an explicit maintainer decision in the quest. The recommended simple contract queues cut() infallibly and reports a backend refusal from the next encode, matching current deferred codec work, unless an immediate acknowledgement requirement is identified.

Wire impact

None. The plan changes no MoQ messages, framing, authentication, catalog schema, media container schema, codec packetization, or version negotiation. Removing the ineffective audio FEC flag does not change currently emitted packets. Rational rates and timestamps convert to the existing published representations at binding and catalog boundaries, and current keyframe grouping remains unchanged until separately scoped refresh work.

Alternatives

  • Defer stabilization until each backend feature is implemented. That couples API decisions to slower hardware work and risks another breaking release.
  • Create another frame abstraction or renderer crate. The plan retains the existing Frame/Surface hierarchy and native/wgpu interoperability.
  • Replace m0's Pronto graph with the release gate. The reconciled graph preserves both independent immediate deliverables instead.
  • Keep compatibility aliases in the 0.0.x crates. The plan removes obsolete names before 0.1 instead of carrying shims.

Follow-ups

  • Execute the 14 API quests, then run the media release-review gate on the exact release candidates before separately authorizing version bumps or publication.
  • Preserve docs(quest): settle 0.0.x package APIs before release #3801 as a separate release-audit PR for archive, E2EE, socket, and uring APIs; this PR does not edit or pre-resolve that branch.
  • Keep surround, refresh, color, additional codecs, native backend studies, allocation optimization, and hardware proof in their existing m2/m3/m4 quests.

Validation: nix develop --command just check passed after rebasing and reconciling the current m0 graph, including all 418 quest documents and repository Markdown checks. No runtime test suite is selected by this planning-only diff.

(written by GPT-5)

@kixelated
kixelated marked this pull request as ready for review September 20, 2026 17:46
@coderabbitai

coderabbitai Bot commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 5845af7e-dec8-46ad-8800-e2977f8f2461

📥 Commits

Reviewing files that changed from the base of the PR and between fc4ac91 and bdbab42.

📒 Files selected for processing (54)
  • quest/README.md
  • quest/m0/README.md
  • quest/m0/audio-aec.md
  • quest/m0/audio-config.md
  • quest/m0/audio-fec.md
  • quest/m0/audio-playback.md
  • quest/m0/audio-publication.md
  • quest/m0/media-features.md
  • quest/m0/media-rate-policy.md
  • quest/m0/media-release-review.md
  • quest/m0/nvenc-loading.md
  • quest/m0/nvenc-resources.md
  • quest/m0/video-frames.md
  • quest/m0/video-gop.md
  • quest/m0/video-output.md
  • quest/m0/video-thread-ownership.md
  • quest/m0/video-timing.md
  • quest/m1/README.md
  • quest/m2/2147-moq-video-10-bit-hevc-and-av1-support-in-the-nvidia-codec.md
  • quest/m2/2848-follow-the-bandwidth-grant-in-moq-audio-instead-of.md
  • quest/m2/README.md
  • quest/m2/audio-buffers.md
  • quest/m2/audio-capture-format.md
  • quest/m2/audio-codecs/README.md
  • quest/m2/audio-codecs/decode-backend.md
  • quest/m2/audio-codecs/decode-mediacodec.md
  • quest/m2/audio-codecs/encode-backend.md
  • quest/m2/audio-codecs/layout.md
  • quest/m2/audio-opus-input.md
  • quest/m2/capture-ergonomics.md
  • quest/m2/color-model.md
  • quest/m2/decoded-frames.md
  • quest/m2/intra-refresh/README.md
  • quest/m2/intra-refresh/encode-config.md
  • quest/m2/intra-refresh/v4l2-refresh.md
  • quest/m2/ladder/README.md
  • quest/m2/mobile/ffi-video-consumer.md
  • quest/m2/nvenc-recovery.md
  • quest/m2/nvenc-reuse.md
  • quest/m2/obs-moq-video/adapter.md
  • quest/m2/obs-moq-video/macos.md
  • quest/m2/obs-moq-video/rate-control.md
  • quest/m2/obs-moq-video/source.md
  • quest/m2/obs-moq-video/windows.md
  • quest/m2/transcode-resources.md
  • quest/m2/transcode-source.md
  • quest/m2/video-render-resources.md
  • quest/m3/README.md
  • quest/m3/audio-loss-recovery.md
  • quest/m3/audio-opus-backend.md
  • quest/m3/decode-drain.md
  • quest/m3/mobile-capture-android.md
  • quest/m3/mobile-capture-ios.md
  • quest/m3/video-codec-coverage.md

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

kixelated and others added 2 commits September 20, 2026 18:49
Co-Authored-By: GPT-6 <noreply@openai.com>
Record each audited crate's public surface and the additive-only moq-mux plus unchanged binding boundaries.

Co-Authored-By: GPT-5 <noreply@openai.com>
@kixelated
kixelated force-pushed the codex/audit-zero-packages branch from 478b7b6 to bdbab42 Compare September 21, 2026 01:49
@moq-bot

moq-bot Bot commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

No issues found.
Verdict: approve
(Written by Muse Spark)

New%20session%20-%202026-09-21T01%3A50%3A19.383Z
opencode session  |  github run

@kixelated
kixelated merged commit d3f7e5a into main Sep 21, 2026
4 checks passed
@kixelated
kixelated deleted the codex/audit-zero-packages branch September 21, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant