diff --git a/doc/bin/cli.md b/doc/bin/cli.md index 1376686f9d..cbd4d9d03a 100644 --- a/doc/bin/cli.md +++ b/doc/bin/cli.md @@ -90,6 +90,8 @@ moq ... play --delay 500ms # trade latency for a jittery link Decodes H.264, H.265, and AV1 video and Opus, PCM, and AAC-LC audio using the platform hardware decoder where available. `--video-name` and `--audio-name` pick a rendition. +HE-AAC signaled only in band (implicit SBR, as over MPEG-TS) plays as its +half-rate AAC-LC core. Playback runs on a clock it owns. `--delay` (default 100 ms) is how far it trails the live edge, which is both the jitter a late frame may absorb and the diff --git a/doc/lib/rs/moq-audio.md b/doc/lib/rs/moq-audio.md index 82f65a3306..5fee2602e7 100644 --- a/doc/lib/rs/moq-audio.md +++ b/doc/lib/rs/moq-audio.md @@ -27,6 +27,10 @@ policy. Decoding likewise separates low-level `decode::Config`, PCM | `playback` | One output device mixing every track in a call, with click-free volume ramps | | `aec` | Acoustic echo cancellation (a port of WebRTC's), so a laptop with no headset doesn't feed itself back | +AAC decoding refuses HE-AAC its config declares. HE-AAC signaled only in band +(implicit SBR, as over MPEG-TS) goes undetected and plays as its half-rate +AAC-LC core. + Highlights: - **`encode::Publication`** advertises the track and opens the microphone only while someone listens. Stop, swap devices, and restart without changing the track subscribers know; read a level meter for the UI. diff --git a/quest/m1/audio-codecs/README.md b/quest/m1/audio-codecs/README.md index c1366d831b..6682cbc9c2 100644 --- a/quest/m1/audio-codecs/README.md +++ b/quest/m1/audio-codecs/README.md @@ -16,8 +16,10 @@ publisher can produce AAC. Platform first, exactly like video: AudioToolbox on macOS and iOS, Media Foundation on Windows, MediaCodec on Android, and symphonia (AAC-LC mono/stereo) as the software fallback that openh264 is for H.264. Linux has no -OS audio decoder, so HE-AAC and multichannel AAC stay refused there, stated in -the docs and rejected at construction. A platform backend claims every catalog +OS audio decoder, so multichannel AAC stays refused there, stated in the docs +and rejected at construction. HE-AAC signaled only in band (implicit SBR, as +over MPEG-TS) plays as its half-rate LC core on symphonia; detecting it needs a +full element walk, so the docs state it instead of refusing it. A platform backend claims every catalog codec its framework opens, so AC-3, E-AC-3, MP3, and FLAC ride along on the hosts that have them; each still needs a fixture before the backend advertises it. @@ -41,7 +43,6 @@ its own decode and encode quest so verification stays per host. ## Quests -- [HE-AAC refusal](/quest/m1/audio-codecs/he-aac-refusal.md) - implicit-SBR HE-AAC over TS is refused instead of half-decoded as the LC core - [TS export PCE](/quest/m1/audio-codecs/ts-export-pce.md) - a TS export of a PCE-described AAC track writes channel_config 0 and the PCE instead of a count-derived config - [Layout](/quest/m1/audio-codecs/layout.md) - the settled `Layout` carries up to 7.1 through decode, resample, playback, and the FFI - [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - `decode::backend` selects a platform decoder before symphonia, mirroring moq-video diff --git a/quest/m1/audio-codecs/decode-backend.md b/quest/m1/audio-codecs/decode-backend.md index ecfb6f64ba..490f5eac84 100644 --- a/quest/m1/audio-codecs/decode-backend.md +++ b/quest/m1/audio-codecs/decode-backend.md @@ -22,8 +22,10 @@ stats and `moq play` surface. AAC-LC mono/stereo only; the platform backends that follow advertise what their framework opens and has a fixture for. - Move today's Opus, PCM, and symphonia code behind the trait without changing - behavior; the HE-AAC sniff from [HE-AAC refusal](/quest/m1/audio-codecs/he-aac-refusal.md) - lands in the symphonia backend. + behavior. Document per host that symphonia plays implicit-SBR HE-AAC as its + half-rate LC core: finding the in-band SBR element needs a full Huffman walk + of the channel elements, and symphonia detects it internally without + exposing or refusing it. - A backend's output rate and layout are what it produced, not what the catalog said (HE-AAC doubles the rate); `Consumer` already resamples and remixes to the requested output, so that stays the seam's contract. diff --git a/quest/m1/audio-codecs/he-aac-refusal.md b/quest/m1/audio-codecs/he-aac-refusal.md deleted file mode 100644 index c3edf29e21..0000000000 --- a/quest/m1/audio-codecs/he-aac-refusal.md +++ /dev/null @@ -1,33 +0,0 @@ -# [S] Refuse implicit-SBR HE-AAC instead of half-decoding it - -## Goal - -An HE-AAC stream that signals AAC-LC and carries SBR only in band is refused -with a clear error, the same way explicitly signaled and backward-compatible -HE-AAC already are. Today it decodes as the LC core at half the sample rate -with no indication anything is wrong. - -## Plan - -ADTS carries a 2-bit profile, so HE-AAC over MPEG-TS (SRT, `moq import ts`) -always arrives as `mp4a.40.2` with a synthesized LC AudioSpecificConfig. The -config-level checks in `rs/moq-audio/src/aac.rs` cannot see it, and the code -says so. The stream itself can: the first raw data block carries an `ID_FIL` -element with `EXT_SBR_DATA` (or `EXT_SBR_DATA_CRC`), and every later frame -does too. - -- Sniff the fill elements of the first packet before handing it to symphonia. - An SBR extension makes the track `Error::Unsupported` with the same wording - as the config-level refusal, naming the host's decoder as the reason. -- Symphonia already walks the element tree, so the sniff is a small parser - over the same syntax, not a decode. It runs once per track. -- Regression: an ADTS fixture with implicit SBR is refused; the existing - explicit and backward-compatible fixtures keep their errors; an LC fixture - with an unrelated fill element still decodes. -- When a platform decoder handles the track ([Decode seam](/quest/m1/audio-codecs/decode-backend.md)), - the sniff belongs to the symphonia backend only; the OS decoders read SBR in - band themselves. - -## Related - -- [Decode seam](/quest/m1/audio-codecs/decode-backend.md) - the sniff becomes the software backend's contract