Repository navigation
feat(qta): read Apple Voice Memos recordings (#646) - #762
Conversation
Register .qta as a read-only audio format: it is QuickTime audio, so FFmpeg's mov demuxer reads it and it goes to every audio target in one step. Settings schema 55 teaches existing audio donuts the extension. iOS 26 memos are reported to pair an AAC track with a 4-channel APAC track FFmpeg has no decoder for. MediaInfo now drops audio streams whose codec lacks the D flag in `ffmpeg -codecs` of the FFmpeg in use (cached per executable; nothing is dropped when the query fails or no stream decodes) and keeps each stream's audio ordinal, which every mapping uses instead of 0:a:0. Files whose audio all decodes get the same arguments as before. Joining copies only when every file's chosen ordinal matches. Closes KnifeLemon#646
KnifeLemon
left a comment
There was a problem hiding this comment.
Thanks, @SeoJCalive! Reading .qta as QuickTime audio, and mapping the first track FFmpeg can actually decode, both look right. Every call site in the diff uses the track's place among all audio tracks, the -codecs parsing skips encoder-only and identify-only entries, and the migration follows the v48/v50/v54 steps. Merging.
A few things for a follow-up, none blocking:
-
Three places still map
0:a:0, although the description says all of them changed. Today's behaviour, so not a regression:ClipPreview.cs:158: the waveform (ReadLoudnessAsync)ClipPreview.cs:200:SilenceArgumentsMediaEncoding.Clip.cs:125: clip music[1:a:0](ClipMusicdoesn't carry the track number)
With an APAC-first file the waveform comes out empty, and "remove silence" or adding it as music fails.
-
The first
ffmpeg -codecscan run on the UI thread:MediaInfo.ProbeAsynccallsDecodableCodecsOfsynchronously after an await withoutConfigureAwait(false). Callers from the clip window (VideoFrames.ProbeAsync,ClipPreview.ReadSoundLengthAsync) can freeze the first open for as long as the query takes. -
The codec cache: one timeout caches
nullfor the whole session, andGetOrAddcan start severalffmpeg -codecsat once on a batch. Same pattern asEncodersOf, so fixing both together makes sense. -
A stream without
codec_namecounts as decodable. When the decodable set is known, it could go with the undecodable ones. -
docs/ENGINES.md: the audio-track line ("Video to audio takes the first") and Merge videos could mention that tracks FFmpeg can't decode are skipped.
Schema note for whoever merges next: this takes 55, so #705 moves to 56.
|
@KnifeLemon Thanks for the review and the merge! You're right on all five, and the description overstated item 1: those three spots still map
I'm starting on it now. |
fix(qta): follow-up to the #762 review (track choice in clips, codec query caching)
Waveform, silence detection and clip music take the first audio track FFmpeg can decode. The codec and encoder list queries run off the caller's thread, run once per executable even when callers overlap, and are asked again after a failure. Audio without a codec name counts as undecodable when the decodable set is known. ENGINES.md mentions the skipped tracks.
Closes #646
What it does
.qtais a read-only audio format (FormatRegistry:qta, audio,Writable: false). Apple Voice Memos saves some recordings as.qta, which is QuickTime audio, so FFmpeg's mov demuxer reads it and it goes to every audio target (MP3, M4A, WAV and the other 19) in one FFmpeg step, like DSS or APE. Nothing writes.qta.apple_apacbut has no decoder for it, so if that track came first,-map 0:a:0would fail withno decoder found for: apple_apac. NowMediaInfokeeps only the audio streams FFmpeg can decode and remembers each one's place among the file's audio streams (AudioStreamInfo.Ordinal); every place that mapped0:a:0/[0:a:0](audio targets, multi-track video targets, clips, clip preview audio, joining, live photo) maps that number instead.ffmpeg -codecs(theDflag) of the same FFmpeg Filee runs, cached per executable like the encoder list. If the query fails or a stream has no codec name, nothing is skipped (today's behaviour).sample_fmt" as the signal: an AAC track that starts late in an MPEG-TS has nosample_fmteither, yet decodes fine. A test pins that case..qta; a.qtathe user put on their own donut stays there. (The issue comment said 49; main had already reached 54.)docs/ENGINES.mdmentions.qtaand the skipped APAC track. The README count goes 268 → 269 in all 15 READMEs.Not checked with a real recording
I couldn't get a real
.qta: Voice Memos recordings I made on both a Mac and an iPhone came out as.m4a, and no sample was attached to the issue. So the APAC part is tested with a file built in the test: a normal two-track AAC QuickTime file whose first track is relabelledapac(sample entry) with itsesdsbox renamed, which ffprobe then reports asapple_apacand FFmpeg refuses to decode, the same error a real APAC track gives. If anyone has an iOS 26 Voice Memos.qta, a quick conversion to MP3 would confirm it.Tests
QtaFormatTests(Core): registry entry and audio profile; the 54 → 55 library step runs once; a library saved at schema 54 learns.qtaon the next start and doesn't learn it again after the user removes it; a user's own donut withQTAkeeps it.MediaTests(with FFmpeg): a.qtareaches every audio target in one step; a QuickTime AAC.qtaconverts to MP3, WAV and M4A; the first of several decodable tracks is used; an APAC-first.qtaconverts to MP3, WAV and M4A from the AAC track (1 channel, 1 s, not the 4-channel 2 s APAC one); an APAC-only file fails with the codec name and writes nothing; a video with AAC then APAC converts to MKV with only the AAC track; the late-start MPEG-TS keeps its first AAC.AudioTracksTests,ClipTests,MediaJoiningTestsandLivePhotoCreationTestscheck the chosen number reaches each mapping, the output option numbers stay 0, 1, …, and joining refuses to copy when the numbers differ;MediaToolRealTestsjoins two real clips like that and checks the sound lasts the whole result.HardwareEncodingTestsparses the codec list and checks a missing or hanging FFmpeg gives up.0:a:0at each mapping, numbering output options by input track, dropping the joining gate, going back to thesample_fmtrule, ignoring theDflag, and treating a failed query as "nothing decodes". The matching test fails every time. The one exception is the ffplay preview call inClipPreview, which passes the same number but has no test that runs ffplay.Ran locally (Linux, FFmpeg 9.0.2 from
build/fetch-engines.ps1), compared with the same commands before the change:dotnet format --verify-no-changesis clean.Alac_speex_and_dts_are_written_and_read: this FFmpeg build has no libspeex encoder).Engines_page_shows_versions_descriptions_and_download_speed, which depends on the engines installed on this PC).filee-cli convertof the APAC-first.qtato MP3, WAV and M4A: each exits 0 with one 1-channel 1.0 s audio stream.