From dcdddf47e94f2d251100b46b76d421b58df9ddfd Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 24 Sep 2026 19:29:15 -0700 Subject: [PATCH 1/3] quest: claim he-aac-refusal From f83c8fe63b25e3de8a556cd4a25f67855ccc512f Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Thu, 24 Sep 2026 20:34:35 -0700 Subject: [PATCH 2/3] chore(quest): abandon he-aac-refusal Detecting implicit SBR needs a full Huffman walk of the channel elements, not a small sniff. Document the half-rate LC core on symphonia instead. Co-Authored-By: Claude Opus 5.5 --- quest/m1/audio-codecs/README.md | 12 ++++----- quest/m1/audio-codecs/aac-pce.md | 1 - quest/m1/audio-codecs/decode-backend.md | 6 +++-- quest/m1/audio-codecs/he-aac-refusal.md | 34 ------------------------- 4 files changed, 10 insertions(+), 43 deletions(-) delete mode 100644 quest/m1/audio-codecs/he-aac-refusal.md diff --git a/quest/m1/audio-codecs/README.md b/quest/m1/audio-codecs/README.md index 372157036a..19f8e74376 100644 --- a/quest/m1/audio-codecs/README.md +++ b/quest/m1/audio-codecs/README.md @@ -16,8 +16,10 @@ publisher can produce AAC. Platform first, exactly like video: AudioToolbox on macOS and iOS, Media Foundation on Windows, MediaCodec on Android, and symphonia (AAC-LC mono/stereo) as the software fallback that openh264 is for H.264. Linux has no -OS audio decoder, so HE-AAC and multichannel AAC stay refused there, stated in -the docs and rejected at construction. A platform backend claims every catalog +OS audio decoder, so multichannel AAC stays refused there, stated in the docs +and rejected at construction. HE-AAC signaled only in band (implicit SBR, as +over MPEG-TS) plays as its half-rate LC core on symphonia; detecting it needs a +full element walk, so the docs state it instead of refusing it. A platform backend claims every catalog codec its framework opens, so AC-3, E-AC-3, MP3, and FLAC ride along on the hosts that have them; each still needs a fixture before the backend advertises it. @@ -37,13 +39,11 @@ mono/stereo. The core configuration and layout contracts land in main. These quests implement surround and backend dispatch on that contract; each platform then lands as -its own decode and encode quest so verification stays per host. The -HE-AAC refusal and the PCE parse are defects in what ships today and are -ready now. +its own decode and encode quest so verification stays per host. The PCE +parse is a defect in what ships today and is ready now. ## Quests -- [HE-AAC refusal](/quest/m1/audio-codecs/he-aac-refusal.md) - implicit-SBR HE-AAC over TS is refused instead of half-decoded as the LC core - [AAC PCE](/quest/m1/audio-codecs/aac-pce.md) - a channel_config of 0 parses the program config element instead of guessing stereo - [Layout](/quest/m1/audio-codecs/layout.md) - the settled `Layout` carries up to 7.1 through decode, resample, playback, and the FFI - [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - `decode::backend` selects a platform decoder before symphonia, mirroring moq-video diff --git a/quest/m1/audio-codecs/aac-pce.md b/quest/m1/audio-codecs/aac-pce.md index db99ec93f9..073f64b473 100644 --- a/quest/m1/audio-codecs/aac-pce.md +++ b/quest/m1/audio-codecs/aac-pce.md @@ -30,5 +30,4 @@ Warn-then-continue is banned: supported or refused. ## Related -- [HE-AAC refusal](/quest/m1/audio-codecs/he-aac-refusal.md) - reads the same first block - [Layout](/quest/m1/audio-codecs/layout.md) - what the parsed PCE eventually maps to diff --git a/quest/m1/audio-codecs/decode-backend.md b/quest/m1/audio-codecs/decode-backend.md index ecfb6f64ba..490f5eac84 100644 --- a/quest/m1/audio-codecs/decode-backend.md +++ b/quest/m1/audio-codecs/decode-backend.md @@ -22,8 +22,10 @@ stats and `moq play` surface. AAC-LC mono/stereo only; the platform backends that follow advertise what their framework opens and has a fixture for. - Move today's Opus, PCM, and symphonia code behind the trait without changing - behavior; the HE-AAC sniff from [HE-AAC refusal](/quest/m1/audio-codecs/he-aac-refusal.md) - lands in the symphonia backend. + behavior. Document per host that symphonia plays implicit-SBR HE-AAC as its + half-rate LC core: finding the in-band SBR element needs a full Huffman walk + of the channel elements, and symphonia detects it internally without + exposing or refusing it. - A backend's output rate and layout are what it produced, not what the catalog said (HE-AAC doubles the rate); `Consumer` already resamples and remixes to the requested output, so that stays the seam's contract. diff --git a/quest/m1/audio-codecs/he-aac-refusal.md b/quest/m1/audio-codecs/he-aac-refusal.md deleted file mode 100644 index f7995cc0f7..0000000000 --- a/quest/m1/audio-codecs/he-aac-refusal.md +++ /dev/null @@ -1,34 +0,0 @@ -# [S] Refuse implicit-SBR HE-AAC instead of half-decoding it - -## Goal - -An HE-AAC stream that signals AAC-LC and carries SBR only in band is refused -with a clear error, the same way explicitly signaled and backward-compatible -HE-AAC already are. Today it decodes as the LC core at half the sample rate -with no indication anything is wrong. - -## Plan - -ADTS carries a 2-bit profile, so HE-AAC over MPEG-TS (SRT, `moq import ts`) -always arrives as `mp4a.40.2` with a synthesized LC AudioSpecificConfig. The -config-level checks in `rs/moq-audio/src/aac.rs` cannot see it, and the code -says so. The stream itself can: the first raw data block carries an `ID_FIL` -element with `EXT_SBR_DATA` (or `EXT_SBR_DATA_CRC`), and every later frame -does too. - -- Sniff the fill elements of the first packet before handing it to symphonia. - An SBR extension makes the track `Error::Unsupported` with the same wording - as the config-level refusal, naming the host's decoder as the reason. -- Symphonia already walks the element tree, so the sniff is a small parser - over the same syntax, not a decode. It runs once per track. -- Regression: an ADTS fixture with implicit SBR is refused; the existing - explicit and backward-compatible fixtures keep their errors; an LC fixture - with an unrelated fill element still decodes. -- When a platform decoder handles the track ([Decode seam](/quest/m1/audio-codecs/decode-backend.md)), - the sniff belongs to the symphonia backend only; the OS decoders read SBR in - band themselves. - -## Related - -- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the sniff becomes the software backend's contract -- [AAC PCE](/quest/m1/audio-codecs/aac-pce.md) - the other place the catalog lies about an AAC stream From 5c61a75e76dca7a8b25e032b578d3b190fe1e4cf Mon Sep 17 00:00:00 2001 From: Luke Curley Date: Fri, 25 Sep 2026 01:09:38 -0700 Subject: [PATCH 3/3] docs(audio): state that implicit-SBR HE-AAC plays as its LC core Co-Authored-By: Claude Opus 5.5 --- doc/bin/cli.md | 2 ++ doc/lib/rs/moq-audio.md | 4 ++++ 2 files changed, 6 insertions(+) diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 1376686f9d..cbd4d9d03a 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -90,6 +90,8 @@ moq ... play --delay 500ms # trade latency for a jittery link Decodes H.264, H.265, and AV1 video and Opus, PCM, and AAC-LC audio using the platform hardware decoder where available. `--video-name` and `--audio-name` pick a rendition. +HE-AAC signaled only in band (implicit SBR, as over MPEG-TS) plays as its +half-rate AAC-LC core. Playback runs on a clock it owns. `--delay` (default 100 ms) is how far it trails the live edge, which is both the jitter a late frame may absorb and the diff --git a/doc/lib/rs/moq-audio.md b/doc/lib/rs/moq-audio.md index 82f65a3306..5fee2602e7 100644 --- a/doc/lib/rs/moq-audio.md +++ b/doc/lib/rs/moq-audio.md @@ -27,6 +27,10 @@ policy. Decoding likewise separates low-level `decode::Config`, PCM | `playback` | One output device mixing every track in a call, with click-free volume ramps | | `aec` | Acoustic echo cancellation (a port of WebRTC's), so a laptop with no headset doesn't feed itself back | +AAC decoding refuses HE-AAC its config declares. HE-AAC signaled only in band +(implicit SBR, as over MPEG-TS) goes undetected and plays as its half-rate +AAC-LC core. + Highlights: - **`encode::Publication`** advertises the track and opens the microphone only while someone listens. Stop, swap devices, and restart without changing the track subscribers know; read a level meter for the UI.