fix(record_ios): Prevent M4A silent frame drops when encoder buffer is full - #8
fix(record_ios): Prevent M4A silent frame drops when encoder buffer is full#8raulmabe-labhouse wants to merge 1 commit into
Conversation
Avoid silent frame drops when AVAssetWriterInput.isReadyForMoreMediaData is false under CPU/memory pressure (Low Power Mode, background throttling). Previously, frames were dropped with no error. Now buffers are queued and drained asynchronously when the encoder becomes ready. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses truncated M4A recordings on iOS by avoiding silent frame drops when AVAssetWriterInput.isReadyForMoreMediaData is false, introducing a queue-and-drain approach for AAC encoding backpressure.
Changes:
- Queue
CMSampleBuffers when the encoder input isn’t ready instead of dropping them. - Add a serial
drainQueueand drain loop to append queued buffers when possible. - Update
stop()/release()to coordinate with the drain queue and clear pending buffers.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let input = input, | ||
| let writer = writer, |
There was a problem hiding this comment.
In write(...), input and writer are unwrapped but never used anymore (they’re only used for existence checks). This will produce unused-variable warnings; consider changing the guard to boolean nil checks or remove these bindings and rely on the drainQueue-side guard.
| let input = input, | |
| let writer = writer, | |
| input != nil, | |
| writer != nil, |
| private func enqueueAndDrain(sampleBuffer: CMSampleBuffer) { | ||
| guard errorMessage == nil, | ||
| let input = input, | ||
| let writer = writer else { return } | ||
|
|
||
| pendingSampleBuffers.append(sampleBuffer) | ||
| drainPendingBuffers() |
There was a problem hiding this comment.
pendingSampleBuffers can grow without bound if the encoder stays backpressured for an extended period (e.g., prolonged background throttling). That can lead to unbounded memory growth and app termination; consider adding a maximum queue size / duration and setting an explicit error (or a defined dropping policy) once exceeded.
| while !pendingSampleBuffers.isEmpty && input.isReadyForMoreMediaData && writer.status != .failed { | ||
| let sampleBuffer = pendingSampleBuffers.removeFirst() | ||
| let success = input.append(sampleBuffer) | ||
| if !success { | ||
| if writer.status == .failed { | ||
| errorMessage = writer.error?.localizedDescription ?? "Writer failed" | ||
| } else if errorMessage == nil { | ||
| errorMessage = "Failed to append sample buffer" | ||
| } | ||
| pendingSampleBuffers.insert(sampleBuffer, at: 0) | ||
| return | ||
| } |
There was a problem hiding this comment.
Using Array.removeFirst() (and insert(..., at: 0) on failure) is O(n) per buffer and can become a hot path when the queue is large. Consider a ring buffer / deque approach (head index) to make draining and re-queueing O(1).
| self.drainPendingBuffers() | ||
| input.markAsFinished() | ||
| writer.finishWriting { [weak self] in |
There was a problem hiding this comment.
stop() calls drainPendingBuffers() only once and then immediately markAsFinished() / finishWriting(). If input.isReadyForMoreMediaData is false at that moment, queued buffers remain undrained but the writer is still finalized, so audio can still be truncated. Consider waiting/draining until the pending queue is empty (e.g., via requestMediaDataWhenReady on drainQueue, or a readiness loop with a timeout) before marking the input finished.
| func write(buffer: AVAudioPCMBuffer, framePosition: Int64) { | ||
| guard errorMessage == nil, | ||
| let input = input, | ||
| let writer = writer, | ||
| let pcmFormat = pcmFormat else { return } | ||
|
|
||
| guard input.isReadyForMoreMediaData else { return } | ||
|
|
||
|
|
||
| let pts = CMTimeMake(value: framePosition, timescale: Int32(pcmFormat.sampleRate)) | ||
|
|
||
| guard let sampleBuffer = buffer.toCMSampleBuffer(presentationTime: pts) else { | ||
| if errorMessage == nil { | ||
| errorMessage = "Failed to create CMSampleBuffer" | ||
| } | ||
| return | ||
| } | ||
|
|
||
| let success = input.append(sampleBuffer) | ||
| if !success { | ||
| if writer.status == .failed { | ||
| errorMessage = writer.error?.localizedDescription ?? "Writer failed" | ||
| } else if errorMessage == nil { | ||
| errorMessage = "Failed to append sample buffer" | ||
|
|
||
| drainQueue.async { [weak self] in | ||
| self?.enqueueAndDrain(sampleBuffer: sampleBuffer) | ||
| } |
There was a problem hiding this comment.
errorMessage is read/written from multiple threads/queues (audio callback thread in write, drainQueue during draining, and the finishWriting completion queue). This introduces a data race and can lead to inconsistent state; consider confining all state mutations/reads (including errorMessage, writer, input) to drainQueue (or protect with a lock/atomic) and make getError/getOutputPath read from the same synchronization context.
Summary
Fixes silent audio frame drops in the M4A writer when the AAC encoder buffer fills under CPU/memory pressure. Previously, any buffer passed while
AVAssetWriterInput.isReadyForMoreMediaDatawas false was silently discarded, producing M4A files shorter than the actual recording duration.Problem
Original code (line 84):
When the AAC encoder cannot keep up with incoming PCM data (e.g. Low Power Mode, background processing throttling, CPU-intensive background tasks),
isReadyForMoreMediaDatareturnsfalse. Thewrite()method returned early with no error and no log. Audio frames were permanently lost.Impact:
recorder_pauseevent fires; users have no indication anything went wrongSolution
Implement a queue-and-drain pattern, following Apple recommended approach for real-time encoding:
AVAudioPCMBuffer→CMSampleBufferimmediately (copies data; safe for async handling)isReadyForMoreMediaDatabecomes truemarkAsFinished()Changes
drainQueue(serialDispatchQueue) for thread-safe drain operationspendingSampleBuffersto hold buffers when encoder is busywrite(): convert to CMSampleBuffer, enqueue, and dispatch async drainenqueueAndDrain()anddrainPendingBuffers()for the drain loopstop(): runs on drainQueue to ensure full flush before finalizationrelease(): clears queue under lock to avoid leaksTesting
Recommended reproduction (before fix):
Before: M4A shorter than WAV and elapsed time
After: M4A and WAV should match
Related
doc/investigations/recording-duration-bug-2026-02.md(audioapp repo)Made with Cursor