Skip to content

Convert Ogg, Opus, WMA and audio above 96 kHz, and carry them onto drives - #111

Open
jwilkins wants to merge 1 commit into
mainfrom
claude/conversions
Open

jwilkins wants to merge 1 commit into
mainfrom
claude/conversions

Conversation

@jwilkins

Copy link
Copy Markdown
Owner

Closes the TODO entry "Formats nothing here converts". Ogg Vorbis, Opus and WMA used to be recognised as unplayable and then left alone, because nothing in the build decoded them. A file above 96 kHz was reported, and refused by the drive writer, because nothing here resampled.

All four are now decoded, and all four get the same treatment. At import they are offered a conversion to FLAC. On a drive they are carried as a 320 kbps MP3, the same way a FLAC bound for a nexus deck already is.

What changed

Decoding (booth-cli, audio::decode):

  • Vorbis: symphonia's own Ogg demuxer and Vorbis decoder. These are two new features, so there is no new dependency.
  • Opus: rusty-opus. symphonia demuxes Opus but has no decoder for it. Booth now trims the 312-sample encoder delay at the start, which symphonia leaves on the first packet.
  • WMA: ffmpeg, when it's installed, through a new audio::ffmpeg module. Nothing in Rust decodes WMA. Booth looks for ffmpeg on PATH, then in Homebrew's two folders (an app started from the Finder doesn't get the shell's PATH), or wherever BOOTH_FFMPEG points. Without ffmpeg, a WMA is reported with the command to install it and is not offered a conversion.

Tags: Ogg, Opus and WMA files now arrive with their names, and a conversion keeps them.

  • Ogg and Opus: the Vorbis comments, as symphonia finds them, under the same keys a FLAC uses.
  • WMA: ffmpeg's metadata listing.
  • Both are read-only. TagKind::writable() keeps the three callers that only ruled out WAV from trying to write them.

Resampling for the library: a new resample::for_library, built on rubato's FFT resampler (already compiled in, via rusty-chromaprint). It converts to 88.2 or 96 kHz (compat::library_rate), so the conversion is a halving or a quartering. The quick resampler that feeds the speech and genre models is unchanged.

Conversions (the app's Convert job):

  • Files above 96 kHz are resampled through for_library.
  • The FLAC is 24-bit from a float WAV or a hi-res file, and 16-bit with dither from a lossy one.
  • Until now every conversion was 16-bit, even though the float WAV's fix text has always said "24-bit FLAC".
  • The converted track gets its new rate at once, so it leaves the attention list straight away.

Drive writer:

  • It takes any format Booth can decode, and re-encodes whatever the target player doesn't open.
  • The refusal above 96 kHz is gone.
  • The preflight now says "skipped" only about files nothing here can decode, such as WMA with no ffmpeg. Everything else it can decode gets the "goes on as mp3" warning instead.

Import: the app now imports the new formats (discover::LIBRARY, collect_library), and so does the library-folder orphan check. The booth-cli pipelines keep the old list, because normalize writes its output in the input's format and has no Ogg, Opus or WMA writer, so one such file in a folder would stop the whole command.

Why it's right

Everything below is measured: each decoder against a reference decode of the same file, each resampler on test tones.

Decoding, on a 30-second CC record:

Distance from the reference Time for 30 s
Vorbis, symphonia vs ffmpeg 137.9 dB (rounding level) 45 ms
Opus, rusty-opus vs libopus float 83.1 dB 96 ms
Opus, opus-decoder vs libopus 87.0 dB 12,633 ms
Opus, ffmpeg's built-in decoder vs libopus 70.2 dB —

Both Rust Opus decoders are closer to libopus than ffmpeg's own decoder is. opus-decoder is 130 times slower than rusty-opus, which would mean about two and a half minutes to decode one six-minute track, so rusty-opus it is. Without the pre-skip trim the decode is exactly 312 samples late. A test holds both the frame count and the phase of a tone that starts at zero.

Resampling, halving tones at 192 kHz:

Audible band 40 kHz Ultrasonic tones folded back 6 min stereo
rubato FFT ±0.0000 dB ±0.0000 dB −149 to −158 dB 2.1 s
Booth's resample ±0.0001 dB −0.34 dB −111 to −123 dB 3.9 s

A test holds the audible band within 0.001 dB and every folded tone below −120 dB. Another holds a click in the same place after resampling, so cues still land on the same beat.

LAME at hi-res rates: I gave it 176.4, 192, 384 and 96 kHz, with the encoder settings the writer uses. It accepted all four and wrote 44.1 or 48 kHz MP3s of the right length. So the writer's refusal above 96 kHz had no technical cause.

Fixtures: six small synthetic files, made with ffmpeg; the commands are in their README.

  • tones.*: a different tone in each channel, which catches a swapped or mixed-down decode.
  • beat.*: 128 BPM, because writing a drive means finding a grid, and a pure tone has none.

The new formats test decodes all of them, reads their tags, and writes a drive carrying Ogg, Opus, WMA and a 192 kHz file. All four land as MP3s at 44.1 or 48 kHz.

The other ways, and why they lost

  • libopus through symphonia-adapter-libopus. That is the reference decoder, but it means building C with cmake, including inside the universal macOS build. rusty-opus has no build step and no dependencies, and it is closer to libopus than ffmpeg is.
  • Make ffmpeg the decoder for Opus too. It would work, but it would make Opus, which is common, depend on something the user has to install. Only WMA, which nothing else decodes, goes through ffmpeg.
  • Strengthen Booth's own resampler instead of adding rubato. rubato is already compiled in, measures 30 dB better, and is faster.
  • Carry 192 kHz files to a CDJ-3000 as 96 kHz FLAC rather than MP3. That would change the writer's one rule for everything a player can't take. Converting in the library gives the lossless result once, and the drive then copies the FLAC.
  • Stop flagging Ogg, Opus and WMA, since the writer now handles them. Every drive would then carry a fresh lossy-to-lossy transcode. A conversion done once avoids that. The same goes for hi-res files, which a CDJ-3000 could play at 96 kHz rather than as a 48 kHz MP3.
  • Widen the shared format list for every booth-cli command. That would break normalize on any folder containing one of these files.

Worth a second opinion

  • rusty-opus's README carries an unrelated advert (a "RAG converter" link). I read the crate itself: no build script, no dependencies, no process, network or environment access, and unsafe only in its documented SIMD kernels. The code looks fine, but the advert is a reason to keep an eye on who maintains it.
  • WMA conversion and drive writes depend on ffmpeg, found once per run and remembered. If someone installs ffmpeg while the app is open, Booth won't see it until a restart. CI now installs ffmpeg so the WMA tests decode a real file. Without it, they check that the missing ffmpeg is explained.
  • 16-bit FLAC for lossy sources is a judgement. It's below the decode's own noise and keeps the files smaller. WMA Lossless exists, and a 24-bit one would lose bits. I didn't check for it, because ffmpeg reports the codec and I chose not to spend a second call on a rare case.
  • Two files that become the same MP3 collide on the drive. A re-encoded copy is named after its file with .mp3 on the end, and nothing checks the name is free. I saw it happen with three fixtures called beat.ogg, beat.opus and beat.wma by the same artist. It was already true for a FLAC and an MP3 of the same name going to a nexus deck. This PR makes it likelier, so it's now in the README's "not done yet" list and in TODO.md.
  • The orphan check now finds converted originals. Once an Ogg is converted, the original Ogg still in the library folder shows up as an orphan, as a float WAV's original already does.
  • The website's known-gaps row still says these files aren't converted. That's still true of 0.1.1, which is what people download, so I'd change it at the release.
  • flacenc won't write FLAC above 96 kHz. Conversions never need to, but anything that someday wants a hi-res FLAC would need a different encoder.

Gates pass: cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings, and env -u BOOTH_WHISPER_MODEL -u BOOTH_WHISPER_BIN -u BOOTH_GENRE_MODEL cargo test --workspace, which runs 32 test binaries, including the new formats.


Generated by Claude Code

…ives

Ogg Vorbis, Opus and WMA were recognised as files no player opens and
then left alone, because nothing in the build decoded them, and a file
above 96 kHz was reported and refused by the drive writer, because
nothing here resampled. All four are now decoded, offered a conversion
to FLAC at import, and carried onto a drive as a 320 kbps mp3 like a
FLAC bound for a nexus deck.

Decoding, each measured on a 30-second CC record against a reference:

- Vorbis, through symphonia's own Ogg demuxer and Vorbis decoder: 137.9
  dB from ffmpeg's decode, which is rounding.
- Opus, through rusty-opus, because symphonia demuxes Opus and has no
  decoder: 83.1 dB from libopus's float decode, where ffmpeg's built-in
  Opus decoder is 70.2 dB from it, in 96 ms for the 30 seconds. The
  other pure-Rust decoder, opus-decoder, was 87.0 dB and took 12.6 s.
  Symphonia leaves the 312-sample pre-skip on the first packet, which
  would put every cue on an Opus file 6.5 ms late; it is taken off.
- WMA, through ffmpeg when it is installed, since nothing in Rust
  decodes it: on PATH, in Homebrew's two folders, or BOOTH_FFMPEG. A
  WMA without ffmpeg is reported with the command to install it, and is
  not offered a conversion that would fail.

Their tags are read too, so a file arrives with its name and a
conversion keeps it: Vorbis comments as symphonia finds them in the
Ogg, under the keys a FLAC's use, and ASF through ffmpeg's metadata
listing. They are read and not written.

Resampling for the library goes through rubato's FFT resampler, already
compiled in by rusty-chromaprint, to 88.2 or 96 kHz so it is a halving
or a quartering. Halving a 192 kHz signal it keeps 40 kHz flat to
0.0001 dB and folds ultrasonic tones back at -149 to -158 dB, where the
quick resampler the models are fed by gets -111 dB and is 0.34 dB down
at 40 kHz; it is also twice as fast. The quick one is left alone.

The drive writer refused anything above 96 kHz outright, though LAME
takes 176.4, 192 and 384 kHz input and writes 44.1 or 48 kHz from it,
measured with the same encoder settings. It now carries those as mp3s
and takes any format something here decodes; what nothing decodes is
still refused, and the preflight says "skipped" only about that.

A conversion is 24-bit FLAC from a float WAV or a hi-res file and
16-bit with dither from a lossy one. Every conversion was 16-bit until
now, though the float WAV's fix has always said 24.

The library takes the new formats on import; the command-line
pipelines do not, because normalize writes its output in the format it
was given and has no Ogg, Opus or WMA writer.

flacenc refuses to write FLAC above 96 kHz, which conversions never
need, so the test's 192 kHz input is a WAV. CI installs ffmpeg so the
WMA tests decode a file; without it they check the absence is
explained. The fixtures are synthetic and made with ffmpeg, with the
commands beside them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019tjgDHGCsZrZcZPVL4Dmon

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants