quest(audio-codecs): Audio codecs

doc/bin/cli.md

@@ -87,9 +87,11 @@ moq --connect https://relay.example.com/anon --broadcast my-stream.hang play
 moq ... play --delay 500ms          # trade latency for a jittery link
 ```
 
-Decodes H.264, H.265, and AV1 video and Opus, PCM, and AAC-LC audio using
-the platform hardware decoder where available. `--video-name` and
-`--audio-name` pick a rendition.
+Decodes H.264, H.265, and AV1 video using the platform hardware decoder where
+available, and Opus, PCM, and AAC-LC (mono or stereo) audio in software. The
+log names the decoder each track opened. `--video-name` and `--audio-name`
+pick a rendition. HE-AAC signaled only in band (implicit SBR, as over MPEG-TS)
+plays as its half-rate AAC-LC core.
 
 Playback runs on a clock it owns. `--delay` (default 100 ms) is how far it
 trails the live edge, which is both the jitter a late frame may absorb and the

doc/lib/c/index.md

@@ -35,7 +35,8 @@ and `target/include/moq.h`.
 - **Threading.** Any function from any thread. Raw publish calls block until the codec takes the frame, which paces a publisher.
 - **Connection health.** `moq_session_stats()` reports available metrics with per-field validity flags. `moq_session_snapshot()` samples those metrics and the negotiated draft name together from the same connection. Its protocol string is backed by static storage. Both return an offline error between reconnects and leave the destination untouched. `moq_session_bandwidth()` mints an allocator over the send estimate; `moq_bandwidth_reserve` claims a share for an app-owned track, and `moq_encode_video` / `moq_encode_audio` take the same handle so the built-in video encoder follows the grant.
 - **Raw playback.** Raw audio and video consumers start at the newest cached group when opened, so rebuilding a live decoder skips the retained backlog.
-- **Raw audio encode.** `moq_audio_encoder_output.codec` names the codec (only `"opus"` today), and `frame_duration_us` sets the Opus frame length: 2500, 5000, 10000, 20000, 40000, or 60000, with 0 meaning the 20 ms default.
+- **Raw audio encode.** `moq_audio_encoder_output.codec` names the codec: `"opus"`, `"pcm"`, or `"aac"`. `frame_duration_us` sets the Opus frame length: 2500, 5000, 10000, 20000, 40000, or 60000, with 0 meaning the codec's default (20 ms for Opus, 1024 samples for AAC). AAC-LC encodes through the platform's encoder, so a host without one refuses it.
+- **Audio channel layouts.** A `channels` count also names the speaker layout, by the WAVE convention: 1 is mono, 2 stereo, 3 2.1, 4 quad, 5 5.0, 6 5.1, 7 6.1, and 8 7.1, interleaved front left, front right, center, LFE, back, then side. `moq_decode_audio` remixes to the count you ask for; past 8 channels the samples pass through but can't be remixed.
 - **Raw decode output.** `moq_video_decoder_output` selects the decoded CPU pixel format (`MOQ_VIDEO_PIXEL_FORMAT_I420` or `_RGBA`) and target size (`width`/`height`, both zero for native; otherwise even and non-zero). Unknown formats and invalid sizes fail `moq_decode_video` before subscribing; accepted requests deliver exactly that layout or fail on the terminal callback.
 - **Encoded video metadata.** `moq_video_init.hint` is a zero-initialized `moq_video_hint` with `has_*` flags for coded dimensions, bitrate (bits per second), frame rate, and latency preference. Hints seed a video codec track's catalog; detected dimensions take precedence.
 - **Client config.** A zeroed `moq_client_config` means the defaults for every knob, which is what lets a new one be appended without disturbing callers. Fields cover protocol (`versions`), TLS (`tls_fingerprints`, `tls_roots`, `tls_cert`/`_key`, `tls_host_name`), transport (`bind`, `connect_timeout_us`, the Happy Eyeballs delays, `websocket_enabled`/`_delay_us`), and tuning (reconnect backoff, `quic_*`). Every duration is in microseconds. A knob whose default isn't zero carries a `has_*` flag, so setting `backoff_timeout_us = 0` needs `has_backoff_timeout = true` to mean "retry forever" rather than "use the default". `moq_client_defaults()` reports what a NULL config dials with.

doc/lib/go/index.md

@@ -115,6 +115,18 @@ one: `FetchGroup`/`FetchMediaGroup`, `Dynamic()` with `Requests(ctx)`,
 `AppendDatagram`/`Datagrams(ctx)`, `SetCatalogSection`, `Demand()` for `Used`/`Unused`,
 `Session().Stats()`. `moq.IsAuthError` and `moq.IsShutdown` classify errors. `moq.ProtocolError(err)` is the structured protocol failure (scope, verbatim code, kind) when the peer sent one.
 
+`EncodeAudio` encodes raw PCM inside the binding. Its codec is `OpusAudioCodec()`
+or `AacAudioCodec()`, and `AudioEncoderOutput.FrameDurationUs` sets the Opus
+frame length: 2500, 5000, 10000, 20000 (the default), 40000, or 60000. 0 takes
+the codec's own frame, which AAC needs. AAC-LC encodes through the platform's
+encoder, so a host without one refuses it.
+
+Audio `Channels` also names the speaker layout, by the WAVE convention: 1 is
+mono, 2 stereo, 3 2.1, 4 quad, 5 5.0, 6 5.1, 7 6.1, and 8 7.1, interleaved
+front left, front right, center, LFE, back, then side. Decoding remixes to the
+count you ask for; past 8 channels the samples pass through but can't be
+remixed.
+
 `DecodeVideo` picks the decoded CPU pixel layout: `VideoDecoderOutput.Format`
 is I420 when nil, or `VideoPixelFormatRgba` for four bytes a pixel, and every
 `VideoDecodedFrame` repeats the layout it was decoded to. `Resize` is best

doc/lib/kt/index.md

@@ -91,8 +91,17 @@ connection's send estimate; pass it to `encodeVideo` / `encodeAudio` or
 native side.
 
 `encodeAudio` encodes raw PCM inside the binding. Its codec is an object,
-`AudioCodec.opus()`, and `AudioEncoderOutput.frameDurationUs` sets the Opus
-frame length: 2500, 5000, 10000, 20000 (the default), 40000, or 60000.
+`AudioCodec.opus()` or `AudioCodec.aac()`, and
+`AudioEncoderOutput.frameDurationUs` sets the Opus frame length: 2500, 5000,
+10000, 20000 (the default), 40000, or 60000. 0 takes the codec's own frame,
+which AAC needs. AAC-LC encodes through the platform's encoder, so a host
+without one refuses it.
+
+Audio `channels` also names the speaker layout, by the WAVE convention: 1 is
+mono, 2 stereo, 3 2.1, 4 quad, 5 5.0, 6 5.1, 7 6.1, and 8 7.1, interleaved
+front left, front right, center, LFE, back, then side. Decoding remixes to the
+count you ask for; past 8 channels the samples pass through but can't be
+remixed.
 
 `decodeVideo` picks the decoded CPU pixel layout: `VideoDecoderOutput.format`
 is `VideoPixelFormat.I420` when null, or `VideoPixelFormat.RGBA` for four bytes

doc/lib/py/index.md

@@ -107,8 +107,17 @@ Each server request reports a `moq.Transport` enum, including QUIC, Iroh,
 WebSocket, TCP, and Unix sockets.
 
 `encode_audio` encodes raw PCM inside the binding. Its codec is an object,
-`moq.AudioCodec.opus()`, and `AudioEncoderOutput.frame_duration_us` sets the
-Opus frame length: 2500, 5000, 10000, 20000 (the default), 40000, or 60000.
+`moq.AudioCodec.opus()` or `moq.AudioCodec.aac()`, and
+`AudioEncoderOutput.frame_duration_us` sets the Opus frame length: 2500, 5000,
+10000, 20000 (the default), 40000, or 60000. 0 takes the codec's own frame,
+which AAC needs. AAC-LC encodes through the platform's encoder, so a host
+without one refuses it.
+
+Audio `channels` also names the speaker layout, by the WAVE convention: 1 is
+mono, 2 stereo, 3 2.1, 4 quad, 5 5.0, 6 5.1, 7 6.1, and 8 7.1, interleaved
+front left, front right, center, LFE, back, then side. Decoding remixes to the
+count you ask for; past 8 channels the samples pass through but can't be
+remixed.
 
 `decode_video` picks the decoded CPU pixel layout: `VideoDecoderOutput.format`
 is `VideoPixelFormat.I420` when unset, or `VideoPixelFormat.RGBA` for four

doc/lib/rs/moq-audio.md

@@ -13,20 +13,62 @@ the far end. Everything is Rust, so there is no C toolchain, CMake step, or
 codec to install.
 
 `Layout` names speaker meaning separately from a channel count. `Mono` is center,
-`Stereo` is left then right, and `Discrete(n)` preserves unnamed channels without
-inventing speaker positions. Encoding keeps source PCM in `encode::Input` and
+`Stereo` is left then right, and the surround layouts up to `SevenPointOne`
+interleave in the SMPTE/WAVE order (front left, front right, center, LFE, back,
+side). A catalog carries only a count, which reads as that count's WAVE default
+(`Layout::from_channels`: 6 is 5.1, 8 is 7.1). `Discrete(n)` preserves unnamed
+channels without inventing speaker positions, so it passes through but never
+remixes. Decoding and playback downmix with the ITU-R BS.775 coefficients and
+upmix by leaving the extra speakers silent; the playback mix runs in whatever
+layout the output device opened. Encoding keeps source PCM in `encode::Input` and
 codec requirements in `encode::Settings`; `encode::Options` adds publication
 policy. Decoding likewise separates low-level `decode::Config`, PCM
 `decode::Output`, and subscription `decode::Options`.
 
 | Module | Does |
 | --- | --- |
 | `capture` | Microphones via CoreAudio, WASAPI, ALSA (and PipeWire/PulseAudio hosts), plus macOS system audio |
-| `encode` | PCM to Opus (with DTX and voice-activity signaling) or raw PCM for the lowest latency |
+| `encode` | PCM to Opus (with DTX and voice-activity signaling), raw PCM for the lowest latency, or AAC-LC through a platform encoder |
 | `decode` | Opus, PCM, and AAC-LC back to PCM, resampled to the rate you want |
 | `playback` | One output device mixing every track in a call, with click-free volume ramps |
 | `aec` | Acoustic echo cancellation (a port of WebRTC's), so a laptop with no headset doesn't feed itself back |
 
+`decode` picks a backend per track the way `moq-video` does: a platform decoder
+first, then software. `decode::Config::kind` forces one (`Kind::Software`, or
+`Kind::Named` with a name below), and `Decoder::name()` reports what opened.
+
+| Backend | Decodes | Hosts |
+| --- | --- | --- |
+| `libopus` | Opus, mono or stereo, and surround up to 7.1 (channel mapping family 1) | all |
+| `pcm` | PCM | all |
+| `symphonia` | AAC-LC, mono or stereo (the default-on `aac` feature) | all |
+
+No platform decoder is wired in yet, so multichannel AAC and HE-AAC declared in
+its config are refused at construction on every host. HE-AAC signaled only in
+band plays as its half-rate LC core. Linux has no OS audio decoder, so it will
+stay that way there. Surround Opus is pure Rust, so it is the one multichannel
+path every host has. Opus mapping families other than 0 and 1 (ambisonics, and
+255's unpositioned channels) are refused, since they declare no speakers.
+
+`encode` selects the same way, through `encode::Settings::kind`, and
+`Encoder::name()` reports what opened.
+
+| Backend | Encodes | Hosts |
+| --- | --- | --- |
+| `libopus` | Opus, mono or stereo | all |
+| `pcm` | PCM | all |
+
+`encode::Codec::Aac` is AAC-LC (`mp4a.40.2`) at the input's rate and layout:
+mono, stereo, 3.0, 4.0, 5.0, 5.1, or 7.1, the layouts with an AAC
+channelConfiguration. Frames are 1024 samples, so `Settings::from_input` sets
+`frame_duration` to match. The catalog's AudioSpecificConfig is built from the
+settings when the track is registered, and since it has no field for the
+encoder's delay, packets are stamped that much earlier so the first input
+sample still lands at the first timestamp. There is no software AAC encoder,
+and no platform encoder is wired in yet, so `Codec::Aac` is refused at
+construction on every host for now. Linux has no OS encoder, so it will stay
+that way there.
+
 Highlights:
 
 - **`encode::Publication`** advertises the track and opens the microphone only while someone listens. Stop, swap devices, and restart without changing the track subscribers know; read a level meter for the UI.

doc/lib/swift/index.md

@@ -95,8 +95,17 @@ divides the connection's send estimate; pass it to `encodeVideo` /
 (scope, verbatim code, kind) when the peer sent one.
 
 `encodeAudio` encodes raw PCM inside the binding. Its codec is an object,
-`AudioCodec.opus()`, and `AudioEncoderOutput.frameDurationUs` sets the Opus
-frame length: 2500, 5000, 10000, 20000 (the default), 40000, or 60000.
+`AudioCodec.opus()` or `AudioCodec.aac()`, and
+`AudioEncoderOutput.frameDurationUs` sets the Opus frame length: 2500, 5000,
+10000, 20000 (the default), 40000, or 60000. 0 takes the codec's own frame,
+which AAC needs. AAC-LC encodes through the platform's encoder, so a host
+without one refuses it.
+
+Audio `channels` also names the speaker layout, by the WAVE convention: 1 is
+mono, 2 stereo, 3 2.1, 4 quad, 5 5.0, 6 5.1, 7 6.1, and 8 7.1, interleaved
+front left, front right, center, LFE, back, then side. Decoding remixes to the
+count you ask for; past 8 channels the samples pass through but can't be
+remixed.
 
 `decodeVideo` picks the decoded CPU pixel layout: `VideoDecoderOutput.format`
 is `.i420` when unset, or `.rgba` for four bytes a pixel, and every frame

go/wrapper/publish.go

@@ -193,7 +193,7 @@ func (b *BroadcastProducer) PublishContainerStream(format ContainerFormat) (*Con
 
 // EncodeAudio publishes a raw-audio track with an in-process encoder.
 //
-// Select the codec with OpusAudioCodec (currently the only constructor).
+// Select the codec with OpusAudioCodec or AacAudioCodec.
 // Pass bandwidth to reserve this track's bitrate against the session's
 // allocator so a co-resident video encoder sizes itself against what is left.
 func (b *BroadcastProducer) EncodeAudio(name string, input AudioEncoderInput, output AudioEncoderOutput, bandwidth *Bandwidth) (*AudioProducer, error) {

go/wrapper/types.go

@@ -8,14 +8,14 @@ import ffi "moq.dev/moq-ffi/moq"
 type (
 	// Audio describes one audio rendition in a broadcast catalog: codec, sample rate, channel count, and container.
 	Audio = ffi.MoqAudio
-	// AudioCodec selects the audio encoder codec. Build one with OpusAudioCodec;
+	// AudioCodec selects the audio encoder codec. Build one with OpusAudioCodec or AacAudioCodec;
 	// adding a codec later adds a constructor, not a breaking enum change.
 	AudioCodec = ffi.MoqAudioCodec
 	// AudioDecoderOutput configures the PCM format, sample rate, and channels DecodeAudio delivers.
 	AudioDecoderOutput = ffi.MoqAudioDecoderOutput
 	// AudioEncoderInput declares the PCM sample format, sample rate, and channel count of frames written to an audio producer.
 	AudioEncoderInput = ffi.MoqAudioEncoderInput
-	// AudioEncoderOutput configures the Opus encoder: codec, optional sample rate, channels, bitrate, and frame duration.
+	// AudioEncoderOutput configures the encoder: codec, optional sample rate, channels, bitrate, and frame duration.
 	AudioEncoderOutput = ffi.MoqAudioEncoderOutput
 	// AudioSampleFormat is a raw PCM sample layout, mirroring WebCodecs AudioData.format.
 	AudioSampleFormat = ffi.MoqAudioSampleFormat
@@ -184,6 +184,12 @@ func OpusAudioCodec() *AudioCodec {
 	return ffi.MoqAudioCodecOpus()
 }
 
+// AacAudioCodec selects AAC-LC through the platform's encoder for EncodeAudio.
+// A host without one refuses it. Leave FrameDurationUs at 0 for AAC's own frame.
+func AacAudioCodec() *AudioCodec {
+	return ffi.MoqAudioCodecAac()
+}
+
 // VideoPixelFormat values: the raw pixel layout fed to the in-process encoder,
 // and the one the in-process decoder delivers.
 const (

js/hang/src/util/aac.test.ts

@@ -48,9 +48,11 @@ describe("audioSpecificConfig", () => {
 		expect(audioSpecificConfig(48000, 8)).toEqual(new Uint8Array([0x11, 0xb8]));
 	});
 
-	// Unsupported channel counts fall back to stereo (config 2).
-	it("unsupported channel count falls back to stereo", () => {
-		expect(audioSpecificConfig(48000, 7)).toEqual(audioSpecificConfig(48000, 2));
+	// A count with no channel configuration is refused rather than described as stereo.
+	it("refuses a channel count with no configuration", () => {
+		for (const count of [0, 7, 9]) {
+			expect(() => audioSpecificConfig(48000, count)).toThrow();
+		}
 	});
 
 	// Non-table sample rates use the 5-byte explicit-frequency form (freqIndex 0xF).

js/hang/src/util/aac.ts

@@ -45,19 +45,19 @@ export function pickRate(rate: number): number {
 const AAC_LC = 2; // audioObjectType for AAC-LC
 
 // Map a channel count to its AAC channelConfiguration (ISO 14496-3 Table 1.19). Configs 1..=6 are
-// identity (5.1 is config 6 / 6 channels); 8 channels is config 7 (7.1). Anything else has no valid
-// config, so fall back to stereo (matching the Rust muxer in rs/moq-mux/src/codec/aac).
+// identity (5.1 is config 6 / 6 channels); 8 channels is config 7 (7.1). Anything else has no
+// config, and describing it as stereo would mislabel every channel past the first two.
 function channelConfig(channelCount: number): number {
-	if (channelCount >= 1 && channelCount <= 6) return channelCount;
+	if (Number.isInteger(channelCount) && channelCount >= 1 && channelCount <= 6) return channelCount;
 	if (channelCount === 8) return 7;
-	return 2;
+	throw new Error(`no AAC channel configuration for ${channelCount} channels`);
 }
 
 /**
  * Build the AAC-LC AudioSpecificConfig that decoders need when frames are raw (no ADTS header).
  *
  * Standard sample rates produce the 2-byte form; non-table rates fall back to the 5-byte form
- * with an explicit 24-bit frequency. Mirrors the Rust muxer so JS and Rust agree on the bytes.
+ * with an explicit 24-bit frequency. Throws for a channel count no configuration names.
  */
 export function audioSpecificConfig(sampleRate: number, channelCount: number): Uint8Array {
 	const config = channelConfig(channelCount);

kt/moq/src/jvmAndAndroidMain/kotlin/dev/moq/Aliases.kt

@@ -134,7 +134,7 @@ typealias FetchGroupOptions = uniffi.moq.MoqFetchGroupOptions
 typealias TrackInfo = uniffi.moq.MoqTrackInfo
 /** One audio frame: PCM payload bytes plus a presentation timestamp. */
 typealias AudioFrame = uniffi.moq.MoqAudioFrame
-/** Selects the audio encoder codec. Build one with `AudioCodec.opus()`. */
+/** Selects the audio encoder codec. Build one with `AudioCodec.opus()` or `AudioCodec.aac()`. */
 typealias AudioCodec = uniffi.moq.MoqAudioCodec
 /** A raw PCM sample format, mirroring WebCodecs `AudioData.format`. */
 typealias AudioSampleFormat = uniffi.moq.MoqAudioSampleFormat

py/moq-rs/README.md

@@ -154,7 +154,7 @@ client = moq.Client(
   - `.publish_audio(format, init, *, label=None, track=None) → MediaProducer`. `init` is required: an OpusHead or AudioSpecificConfig resolves the whole rendition. `track` names the track; otherwise a unique name is derived from the format.
   - `.publish_video(format, init=b"", *, label=None, hint=None, track=None) → MediaProducer`. `init` may be empty for a format that resolves in band; a `VideoHint` pins catalog fields the stream can't reveal (bitrate) or publishes the catalog before the first keyframe. `track` names the track as in `publish_audio`.
   - `.encode_video(input, output, *, bandwidth=None) → VideoProducer`. Encode raw `VideoFrame`s inside the binding; `.write(frame)` each one.
-  - `.encode_audio(name, input, output, *, bandwidth=None) → AudioProducer`. Encode raw PCM `AudioFrame`s; the codec is `output.codec`, e.g. `AudioCodec.opus()`, with `output.frame_duration_us` setting the Opus frame length.
+  - `.encode_audio(name, input, output, *, bandwidth=None) → AudioProducer`. Encode raw PCM `AudioFrame`s; the codec is `output.codec`, e.g. `AudioCodec.opus()` or `AudioCodec.aac()`, with `output.frame_duration_us` setting the Opus frame length (0 takes the codec's own frame, which AAC needs).
   - `.finish()`
 - **`BroadcastDynamic`**. Async source of tracks requested by subscribers.
   - `await .requested_track() → TrackRequest`. Call `.accept()` on it for a `TrackProducer`, or `.abort(code)` to reject.

py/moq-rs/moq/publish.py

@@ -749,8 +749,8 @@ def encode_audio(
     ) -> AudioProducer:
         """Publish a raw-audio track with an in-process encoder.
 
-        Select the codec with ``moq.AudioCodec.opus()`` (currently the only
-        constructor), placed in ``output``.
+        Select the codec with ``moq.AudioCodec.opus()`` or
+        ``moq.AudioCodec.aac()``, placed in ``output``.
 
         Pass ``bandwidth`` to reserve this track's bitrate against the session's
         allocator so a co-resident video encoder sizes itself against what is left.