Repository navigation
Conversation
Webcams deliver 4K30 as MJPEG. Qt decoded each frame to RGB and then encoded it as 10-bit 4:4:4 H.264, which most players cannot open and which is far slower to encode. Decode with libjpeg-turbo straight to video-range YUV 4:2:0 on the thread pool, keeping frames in capture order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Qt's recorder always runs libx264 at its medium preset, which manages about 17 fps at 4K, and never uses VAAPI for frames in system memory, so 4K takes stopped within seconds. Encode on a worker thread with FFmpeg: the GPU's h264_vaapi encoder when available, otherwise libx264 at veryfast. Takes keep the same timing model and MP4 output. Fixes omacom#8 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Level 5.1 above 1080p, and 4.2 otherwise, on VAAPI and libx264. No B-frames, so the software fallback stays low-delay. A failed GPU open frees the codec before that fallback, including when the frame buffer or video stream cannot be created.
The capture thread copies the compressed bytes and decodes them on a pool of two threads, so a worker never maps the preview frame and a 4K decode cannot occupy every core. One bad JPEG is skipped. Fifteen in a row stop the take. VAAPI is not flushed when that stop happens before any frame, which faults. The file records the encoder name, and the level check reads ffprobe's numeric JSON field.
[Medium impact] The PR changes the recording and finalizing path used by Monologue takes. The PR needs fixes before merging because the encoder timeout, unsupported microphone rates, and native-frame colors remain unresolved. Findings
SummaryReplaces Qt recording with an FFmpeg worker and adds a bounded JPEG decode queue.
|
| connect(&m_flushTimeout, &QTimer::timeout, this, [this] { | ||
| if (!m_failed) { m_failed = true; emit failed("The encoder stopped responding, so the take was stopped."); } | ||
| emitFinished(); |
There was a problem hiding this comment.
Timeout leaves the encoder running
When the timeout fires, it emits finished before the encoder has stopped. Backend::writerFinished() then starts reading the MP4 while the worker may still be writing it or may not have written its trailer. Deleting the writer also waits for that worker on the GUI thread, so a stuck encoder can freeze the window.
Keep worker shutdown off the GUI thread, and emit finished only after the file is closed.
| m_audio->sample_rate = format.sampleRate(); | ||
| av_channel_layout_default(&m_audio->ch_layout, format.channelCount()); | ||
| m_audio->bit_rate = 192000; | ||
| m_audio->time_base = {1, format.sampleRate()}; | ||
| if (m_format->oformat->flags & AVFMT_GLOBALHEADER) m_audio->flags |= AV_CODEC_FLAG_GLOBAL_HEADER; | ||
| int result = avcodec_open2(m_audio, codec, nullptr); |
There was a problem hiding this comment.
Some microphones cannot record
If a microphone reports a 192 kHz preferred format, openAudio() tries to open AAC at that unsupported rate and the whole take fails to start. The backend passes preferredFormat() through unchanged, and the new audio conversion uses the same input and output rate.
Choose a rate supported by AAC and resample the captured audio to it.
| m_video->color_range = AVCOL_RANGE_MPEG; | ||
| m_video->colorspace = AVCOL_SPC_BT470BG; |
There was a problem hiding this comment.
If a camera supplies native BT.709 or full-range YUV frames, the encoder marks them as limited-range BT.601 without converting their values to match. Non-JPEG frames bypass decodeJpeg(), and the encoder never reads their colorRange() or colorSpace(). Recordings can therefore have wrong colors or clipped dark and bright areas.
Honor the source settings when converting pixels or writing the output color settings.
| if (!frame.map(QVideoFrame::ReadOnly)) { noteBadFrame(); return; } | ||
| const QByteArray jpeg(reinterpret_cast<const char *>(frame.bits(0)), frame.mappedBytes(0)); | ||
| frame.unmap(); | ||
| if (jpeg.isEmpty()) { noteBadFrame(); return; } |
There was a problem hiding this comment.
Map failures update m_badDecodes immediately, but successful JPEGs reset it only after their background decode finishes. After 14 bad decodes, a good JPEG still waiting to finish followed by one map failure stops the take, even though the bad frames were not consecutive.
Put map and empty-buffer failures into m_frames as failed results too, so all counting happens in capture order.
4276b75 to
d33f7e5
Compare
|
Tested this PR (d33f7e5) on an Intel MacBook Pro 13" 2019 (T2, i5-8259U, Iris Plus 655, intel-media-driver 26.3.5 / iHD), Omarchy, kernel 7.2.11-t2, Hyprland 0.56.2, Qt 6.12.0, FFmpeg 9.0.2, with a Logitech BRIO (4K) on a Thunderbolt dock. 0.3.1 (packaged): every take stopped within ~3 s, at 4K and also at 1080p/720p fed through a v4l2loopback device. The log showed This PR:
The VAAPI path works well on Intel Gen9 here. A format/resolution picker (#14/#18) would let people on older CPUs pick 1080p from the camera directly instead of going through v4l2loopback. On one earlier run of this build (before I added logging), a ~4 s take ended with "The encoded clip did not match the selected resolution or audio mode". I couldn't reproduce it afterwards, so I don't know the cause. |
|
Thanks! This builds on #11, and the 4:2:0 decode it carries is now covered by #32, which keeps Qt's recorder and lets it choose the hardware encoder. Closing in favour of that. If the H.264 level / no-B-frames change or the bad-JPEG handling still matter on top of #32, they'd be welcome as a small follow-up. |
Fixes #8. Also addresses the encoder half of #3: its 4:4:4 output and software-only encoding.
This is #11 plus two commits. The first two commits are Merovex's, unchanged, from
fix-4k-recording. If #11 takes the last two, this pull request can close.What changed
From #11:
yuv420pinstead of 10-bit 4:4:4.QMediaRecorder. A worker thread encodes H.264/AAC into the MP4 withh264_vaapiwhen available, otherwiselibx264atveryfast.Writerkeeps its timing model, failure messages, and backpressure limits.libjpeg-turbo(README, PKGBUILD).On top of that:
h264_vaapiandlibx264. A failed GPU open frees the codec before thelibx264fallback. Same-size frames are copied or point-sampled instead of bilinear-scaled, and the file records the encoder name.The frame-grid drops in #9 are not in this pull request. That fix is #10. Short takes on this branch can still come in under 30 fps for that reason.
Verification
The first two commits were checked by Merovex on a Beelink SER8 (Ryzen 7 8745HS, Radeon 780M), Omarchy 4.0.4, Qt 6.11.2, FFmpeg 9.0.1, Logitech MX Brio at 3840×2160 MJPEG 30 fps. Details are on #11: tests passed, and a 59 s 4K take completed where takes had been stopping after 1–3 s.
The last two commits were checked on an Intel Core Ultra X7 358H with Arc B390 (FFmpeg 9.0.2):
./bin/test: 24 backend tests and 3 UI tests, including the bad-JPEG cases, level 4.2 with no B-frames at 320×240, and a forcedlibx264fallback../bin/test-camera: a 4.0 s 3840×2160 clip, High,yuv420p, level 5.1, no B-frames, encoder tagh264_vaapi, with AAC for the whole take. The frame count was 100, about 25 fps, which is the separate Takes silently lose about a third of their frames when camera jitter straddles the frame grid #9 drop.