feat(audio): a decode backend seam that prefers the platform codec

doc/bin/cli.md

@@ -87,11 +87,11 @@ moq --connect https://relay.example.com/anon --broadcast my-stream.hang play
 moq ... play --delay 500ms          # trade latency for a jittery link
 ```
 
-Decodes H.264, H.265, and AV1 video and Opus, PCM, and AAC-LC audio using
-the platform hardware decoder where available. `--video-name` and
-`--audio-name` pick a rendition.
-HE-AAC signaled only in band (implicit SBR, as over MPEG-TS) plays as its
-half-rate AAC-LC core.
+Decodes H.264, H.265, and AV1 video using the platform hardware decoder where
+available, and Opus, PCM, and AAC-LC (mono or stereo) audio in software. The
+log names the decoder each track opened. `--video-name` and `--audio-name`
+pick a rendition. HE-AAC signaled only in band (implicit SBR, as over MPEG-TS)
+plays as its half-rate AAC-LC core.
 
 Playback runs on a clock it owns. `--delay` (default 100 ms) is how far it
 trails the live edge, which is both the jitter a late frame may absorb and the

doc/lib/rs/moq-audio.md

@@ -33,9 +33,20 @@ policy. Decoding likewise separates low-level `decode::Config`, PCM
 | `playback` | One output device mixing every track in a call, with click-free volume ramps |
 | `aec` | Acoustic echo cancellation (a port of WebRTC's), so a laptop with no headset doesn't feed itself back |
 
-AAC decoding refuses HE-AAC its config declares. HE-AAC signaled only in band
-(implicit SBR, as over MPEG-TS) goes undetected and plays as its half-rate
-AAC-LC core.
+`decode` picks a backend per track the way `moq-video` does: a platform decoder
+first, then software. `decode::Config::kind` forces one (`Kind::Software`, or
+`Kind::Named` with a name below), and `Decoder::name()` reports what opened.
+
+| Backend | Decodes | Hosts |
+| --- | --- | --- |
+| `libopus` | Opus, mono or stereo | all |
+| `pcm` | PCM | all |
+| `symphonia` | AAC-LC, mono or stereo (the default-on `aac` feature) | all |
+
+No platform decoder is wired in yet, so multichannel AAC and HE-AAC declared in
+its config are refused at construction on every host. HE-AAC signaled only in
+band plays as its half-rate LC core. Linux has no OS audio decoder, so it will
+stay that way there.
 
 Highlights:
 

quest/m1/audio-codecs/README.md

@@ -44,7 +44,6 @@ its own decode and encode quest so verification stays per host.
 ## Quests
 
 - [TS export PCE](/quest/m1/audio-codecs/ts-export-pce.md) - a TS export of a PCE-described AAC track writes channel_config 0 and the PCE instead of a count-derived config
-- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - `decode::backend` selects a platform decoder before symphonia, mirroring moq-video
 - [AudioToolbox decode](/quest/m1/audio-codecs/decode-audiotoolbox.md) - macOS and iOS decode HE-AAC, multichannel AAC, and what else the framework offers
 - [Opus surround](/quest/m1/audio-codecs/opus-surround.md) - mapping family 1 decodes on every host through the multistream decoder
 - [Encode seam](/quest/m1/audio-codecs/encode-backend.md) - `encode::backend` and `Codec::Aac`, so a native publisher can produce AAC-LC

quest/m1/audio-codecs/decode-audiotoolbox.md

@@ -10,17 +10,23 @@ documents as unsupported.
 ## Plan
 
 An `AudioConverter` from the packetized format to interleaved `f32` at the
-codec's native rate and layout, behind the decode seam as the first platform
-candidate on `target_os = "macos"` and `"ios"`. `objc2-audio-toolbox` is the
-binding, alongside the `objc2-core-audio-types` the crate already carries.
+codec's native rate and layout, behind the decode seam
+(`rs/moq-audio/src/decode/backend`) as the first platform candidate on
+`target_os = "macos"` and `"ios"`. `objc2-audio-toolbox` is the binding,
+alongside the `objc2-core-audio-types` the crate already carries.
+
+- With a platform tier in place, `Auto` falling past a refusing platform
+  decoder to software should warn, as moq-video's `select` does; the seam
+  only aggregates the refusals into its error today.
 
 - Build the `AudioStreamBasicDescription` and magic cookie from the catalog
   description; the converter reports the output layout, which maps to
   `Layout` from the AudioChannelLayout tag rather than a count.
 - HE-AAC: the converter reads SBR in band and reports the doubled rate; the
   seam passes it through. No config-level guessing.
 - Priming and remainder: AudioToolbox reports `kAudioConverterPrimeInfo`;
-  trim it so timestamps line up with symphonia's output on the same stream.
+  report it as the backend's startup delay, which the front end trims, so
+  timestamps line up with symphonia's output on the same stream.
 - Every codec the backend advertises has a fixture and a decode test, and the
   test asserts the layout order matches the canonical one (the LFE and centre
   end up where `Layout` says).
@@ -29,10 +35,6 @@ binding, alongside the `objc2-core-audio-types` the crate already carries.
 - Docs: `doc/bin/obs.md` drops the HE-AAC and multichannel caveat on macOS,
   and the backend table names what this host decodes.
 
-## Required
-
-- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the candidate order this backend joins
-
 ## Related
 
 - [Media Foundation decode](/quest/m2/audio-decode-mediafoundation.md) - the same shape on Windows

quest/m1/audio-codecs/encode-backend.md

@@ -9,7 +9,7 @@ host with no AAC encoder refuses it at construction.
 
 ## Plan
 
-Mirror the decode seam: `encode::backend` with a crate-private `Backend`
+Mirror the decode seam (`rs/moq-audio/src/decode/backend`): `encode::backend` with a crate-private `Backend`
 trait (`encode`, `flush`, `set_bitrate`, `name`), an `open(codec, config)`
 that walks platform candidates before software ones, using the public settings
 and selection contract settled in main. Opus and PCM retain their behavior. This
@@ -35,10 +35,6 @@ quest adds AAC through platform encoders; no software AAC dependency is selected
 - Regression: the selection order with a stub backend; `Codec::Aac` refused on
   a host with no backend; the Opus and PCM paths unchanged.
 
-## Required
-
-- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the naming and shape this mirrors
-
 ## Related
 
 - [OBS audio publishing](/quest/m1/obs-moq-video/audio-publish.md) - the OBS encoder adapter can offer AAC once this lands

quest/m2/audio-decode-mediacodec.md

@@ -23,10 +23,6 @@ behind a new optional audio `mediacodec` feature and the decode seam, on `target
 - The binding ships in the moq-ffi Android slice, which is how Kotlin and Dart
   reach it.
 
-## Required
-
-- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the candidate order this backend joins
-
 ## Related
 
 - [Android capture](/quest/m2/mobile-capture-android.md) - the video MediaCodec family this sits beside

quest/m2/audio-decode-mediafoundation.md

@@ -22,10 +22,6 @@ ones. Behind the decode seam as the first candidate on `target_os =
 - Verification runs on a Windows host; the per-PR CI only compiles the
   platform code, and `just rs windows` runs nightly.
 
-## Required
-
-- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the candidate order this backend joins
-
 ## Related
 
 - [Runtime QA hosts](/quest/m2/runtime-qa-hosts.md) - where the Windows run happens

rs/moq-audio/src/decode/backend/libopus.rs

@@ -0,0 +1,126 @@
+//! Opus through libopus, the software decoder for every host.
+
+use unsafe_libopus::{
+	OPUS_OK, OPUS_RESET_STATE, OpusDecoder, opus_decode_float, opus_decoder_create, opus_decoder_ctl_impl,
+	opus_decoder_destroy, varargs,
+};
+
+use super::Backend;
+use crate::decode::Decoded;
+use crate::{Error, Layout, opus};
+
+pub(super) const NAME: &str = "libopus";
+
+/// Opus packets cap at 120 ms (RFC 6716 §2.1.4).
+const MAX_FRAME_MS: usize = 120;
+
+pub(super) struct Libopus {
+	inner: *mut OpusDecoder,
+	sample_rate: u32,
+	layout: Layout,
+	pre_skip: usize,
+	max_frame_size: usize,
+	in_dtx: bool,
+}
+
+// SAFETY: the decoder is owned exclusively and libopus keeps no thread-local state.
+unsafe impl Send for Libopus {}
+
+impl Libopus {
+	/// Parses the OpusHead `description` if present; falls back to the catalog's
+	/// declared sample rate / channel count.
+	pub(super) fn open(catalog: &hang::catalog::AudioConfig) -> Result<Box<dyn Backend>, Error> {
+		let (sample_rate, channel_count, pre_skip) = if let Some(desc) = &catalog.description {
+			let mut buf = desc.as_ref();
+			match moq_mux::codec::opus::Config::parse(&mut buf) {
+				Ok(head) => (head.sample_rate, head.channel_count, head.pre_skip),
+				Err(_) => (catalog.sample_rate, catalog.channel_count, 0),
+			}
+		} else {
+			(catalog.sample_rate, catalog.channel_count, 0)
+		};
+
+		opus::validate_rate(sample_rate)?;
+		let channels = opus::validate_channels(channel_count)?;
+		let layout = Layout::from_channels(channel_count)?;
+
+		let mut err = 0i32;
+		// SAFETY: out-pointer is valid; inner is checked for null below.
+		let inner = unsafe { opus_decoder_create(sample_rate as i32, channels, &mut err) };
+		if err != OPUS_OK || inner.is_null() {
+			return Err(opus::error(err, "opus_decoder_create"));
+		}
+
+		Ok(Box::new(Self {
+			inner,
+			sample_rate,
+			layout,
+			// OpusHead counts pre-skip at 48 kHz whatever rate the decoder runs at.
+			pre_skip: (pre_skip as usize * sample_rate as usize) / 48_000,
+			max_frame_size: (sample_rate as usize * MAX_FRAME_MS) / 1000,
+			in_dtx: false,
+		}))
+	}
+}
+
+impl Backend for Libopus {
+	/// Empty packets invoke packet-loss concealment. Loss during DTX remains
+	/// classified as DTX, while loss during active audio remains active.
+	fn decode(&mut self, packet: &[u8]) -> Result<Decoded, Error> {
+		let channels = self.layout.channels() as usize;
+		let mut out = vec![0.0f32; self.max_frame_size * channels];
+		// SAFETY: `inner` owns a live OpusDecoder; packet/out slices are bounded by
+		// the lengths we pass.
+		let samples = unsafe {
+			opus_decode_float(
+				&mut *self.inner,
+				packet.as_ptr(),
+				packet.len() as i32,
+				out.as_mut_ptr(),
+				self.max_frame_size as i32,
+				0,
+			)
+		};
+		if samples < 0 {
+			return Err(opus::decode_error(samples));
+		}
+		out.truncate(samples as usize * channels);
+
+		let activity = opus::activity(packet, self.in_dtx);
+		self.in_dtx = activity.is_dtx();
+		Ok(Decoded { samples: out, activity })
+	}
+
+	fn reset(&mut self) -> Result<(), Error> {
+		// SAFETY: `inner` owns a live decoder and OPUS_RESET_STATE takes no arguments.
+		let rc = unsafe { opus_decoder_ctl_impl(self.inner, OPUS_RESET_STATE, varargs![]) };
+		if rc != OPUS_OK {
+			return Err(opus::error(rc, "OPUS_RESET_STATE"));
+		}
+		self.in_dtx = false;
+		Ok(())
+	}
+
+	fn sample_rate(&self) -> u32 {
+		self.sample_rate
+	}
+
+	fn layout(&self) -> Layout {
+		self.layout
+	}
+
+	fn delay(&self) -> usize {
+		self.pre_skip
+	}
+
+	fn name(&self) -> &str {
+		NAME
+	}
+}
+
+impl Drop for Libopus {
+	fn drop(&mut self) {
+		// SAFETY: `inner` is a live OpusDecoder that nothing else aliases.
+		unsafe { opus_decoder_destroy(self.inner) };
+	}
+}