You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Two Tier 2 tests that mix real-time audio failed once each on 2026-10-15 in local rust/tools/linux-validate/run.sh runs on a loaded shared machine, then passed on rerun with no code change. Neither run touched the code under test (#828/#832 and #833/#835).
debug: Audio timestamp for 'vlc' exceeded TS_SMOOTHING_THRESHOLD, diff=128596000 ns, expected 18243622000, input 18372218000
file 2: 3.008 s of 3.0 s in the mix, heard from 0.014 s to 1.114 s of the file, mix 3.093 s - 6.229 s
file 2: the last 1.886 s are missing from the mix
playlist track boundary cut audio
All four cases failed with no program audio was mixed (test_group_hide_transition_audio.c:216 first).
Likely cause
Both tests play audio in real time and assert on what reached the mix. When the process is starved (another Docker build on the host), a ~130 ms timestamp jump past TS_SMOOTHING_THRESHOLD or a late audio thread drops audio, and the assertion fails. That is the test environment, not the code under test.
Options
make the tests drive the audio clock instead of wall time, or
widen what they accept for a timestamp jump (and assert the jump was logged), or
run them in a lane that is not shared with parallel builds.
Running them under ctest --repeat until-fail:N while another build loads the machine should reproduce it.
Two more one-off failures in a loaded local Tier 2 run (OFF build, 2026-10-15 ~22:50Z, unrelated change #918):
test_async_delay_audio (test_async_delay_audio.c:466), another real-time audio test.
test_v4l2_busy_signalforce_camera_closes_the_holder_and_opens_the_camera (test_v4l2_busy_signal.c:296, 10185 != 0): the forced holder process had not been reaped yet when the test checked. Not audio, but the same pattern: a wall-clock race that loses under load.
Another one, video this time: test_deinterlace_2x_field_order (discard_2x_bottom_first_60i_into_50, yadif_2x_top_first_unbuffered_60i_into_60; test_deinterlace_2x_field_order.c:370) failed once in a loaded local Tier 2 run (OFF build, 2026-10-16 ~01:50Z, unrelated change 965): 12 fields skipped, up to 2 in a row (allowed 1). Same wall-clock pattern: real-time frame cadence loses under machine load.
Another one: test_vlc_playlist_gap failed (28.6 s, ***Failed) in Tier 2 Linux (ON) on the merge-queue run for #936 (job 116819150784, 2026-10-16 05:13 UTC). #936 is a text-freetype2 refactor and touches nothing in vlc-video. It knocked #936 out of the queue; I re-queued it.
Add test_media_restart_stale_frame to the list: failed once in a heavily loaded local Tier 2 run (OFF, 2026-10-16 ~05:50Z, unrelated change 997), together with test_async_delay_audio and test_deinterlace_2x_field_order. Separately, test_v4l2_busy_signal (also logged here earlier) was a fixture race rather than load sensitivity; #953 / 997 fixes it.
These flakes are now costing merges: #936 (text-freetype2, no media code) was removed from the merge queue twice for failed_checks. The 05:19Z removal was test_vlc_playlist_gapstop_at_track_end_is_prompt (test_vlc_playlist_gap.c:476) in merge-group run 38850208674, Rust Tier 2 Linux (ON). The queue's Tier 2 runs every real-time test against a busy runner, so each of these is a coin flip on every merge.
test_vlc_playlist_gapstop_at_track_end_is_prompt is not load sensitivity: it is a real race in vlc-video (an async libVLC Opening event resets drain_abort after Stop signals it), filed and claimed as 1011. The other cases here (audio/video cadence under load) are still open.
test_media_restart_stale_frameclear_still_clears is a crash, not a timing flake: obs_shutdown() segfaults after obs_reset_video() failed ("Unable to open X display"). Filed as 1032.
A common cause for part of this list: in loaded local Tier 2 runs (2026-10-16 ~07:30Z and ~08:10Z), single cases fail setup with info: Unable to open X display while the neighbouring cases in the same binary pass, e.g. test_v4l2_busy_signalforce_camera_opens_while_the_holder_is_still_exiting ("Could not run test: Test setup failed") and test_media_restart_stale_frame (which then crashes in obs_shutdown, #1032). So the Xvfb server from xvfb-run -a intermittently isn't reachable while ctest runs tests in parallel (-j4). That's an environment fault, not load sensitivity in the tested code; worth fixing in the harness (one Xvfb per test, or retry on display-open failure) alongside #982. Separately, test_vlc_playlist_gapstop_at_track_end_is_prompt was a real vlc-video race: #1011, fixed in #1037.
Correction to my earlier comment: "Unable to open X display" is not evidence that Xvfb was unreachable. It is printed by obs_nix_x11_log_info() (libobs/obs-nix-x11.c:39) whenever obs_get_nix_platform_display() is NULL, i.e. whenever nothing called obs_set_nix_platform_display(), which only the frontend does. Every cmocka test that calls obs_startup logs it, including passing ones (the next test_v4l2_busy_signal case logged it and then registered X11 hotkey keycodes fine). So the "Xvfb dropping out under parallel ctest" explanation I gave is wrong and should be disregarded; the cause of these failures is still unknown. For the list here: test_v4l2_busy_signal's "Test setup failed" case and test_media_restart_stale_frame's segfault both still need a real cause.
Another hit, 2026-10-16 19:40 UTC, in the merge queue: test_vlc_playlist_gap / playlist_of_three_plays_every_file_to_the_end failed in Rust Tier 2 Linux (ON 2/3) of run 38956680388 (merge group for #1111, a linux-pipewire change unrelated to VLC). Log: Audio timestamp for 'vlc' exceeded TS_SMOOTHING_THRESHOLD, diff=127413000 ns, then file 1: the last 1.056 s are missing from the mix. The OFF lane passed. This kicked #1111 out of the queue; it's re-queued.
Another one: test_media_seek_keyframe / playing_seek_between_keyframes failed at test_media_seek_keyframe.c:629 in Rust Tier 2 macos-26 on the merge-queue run for #1292 (run 39055672620). #1292 only touches plugins/frontend-tools, so this is unrelated to it. Real-time media playback under queue load, like the others here.
Another observed instance: master run 39078519174, OFF job 117532901565, failed test_group_hide_transition_audio (9.14 s). Three subcases (hide_child_with_transitions, hide_group_with_child_transitions, hide_group_during_child_show_transition) reported no program audio was mixed, failing line 216 after the fixed one-second mix observation window. 293 CTest registrations; one executable failed.
OBS startup succeeded, created the main canvas and initialized llvmpipe OpenGL. This is separate from #1355's locally reproduced XOpenDisplay/startup failure. The informational Unable to open X display message precedes successful startup here too. No new Xvfb attribution, skip, assertion change or blind rerun was made. Raw log is retained under ignored build_ci_audit/master-e89-off-failure.log during the #1286 audit.
Additional live failure in master run 39084293679, revision d27d69f585eec876b3d90a18b74ca69194422cba, OFF job 117549418999: unchanged test_group_hide_transition_audio failed all four cases at line 216 with no program audio was mixed. Suite registered 295 tests, one failing executable. Startup-failure diagnostics passed; the run failed this OFF job. No retry, skip or assertion relaxation.
This recurs after a different full manual run (39082733551) passed both modes. A green full run therefore does not establish absence of this intermittent integrity failure. This is separate from #1355's verified Xvfb startup fix and #1283's recording-shutdown state race. Logs retained locally at build_ci_audit/master-1373-off-failure-117549418999.log while continuing #1286 cache validation. No cache-layer root cause is claimed.
Claiming: investigate only the test_group_hide_transition_audio scheduling portion while completing #1286. Compare the unchanged executable in isolated versus concurrent execution on the existing local Linux build, preserve all C assertions, and record skips/environment failures separately. If resource isolation is supported by reproduction, add focused scheduling regression coverage and a narrow CTest change; do not blanket-serialize media tests or close the unrelated VLC/seek cases.
Part 1 landed: #3000 (squash-merged) re-anchors program_level()'s wait on the mix's own clock. The window keeps its original anchors, so it still measures the same 400 ms of program audio; what changed is that it waits for the mix to reach the window's end (bounded at 5 s) instead of sleeping 1000 ms.
Evidence in the PR, from the Tier 2 harness on a host at load average 34: 1.00 on --repeat until-fail:100 for this test with ENABLE_RUST_LIBOBS=OFF and ON, and the unmodified 623-test suite green in both modes. The first attempt on that branch opened the window on the first arriving frame and the 100-repeat run caught a finished show transition inside it (0.89 of the tone) at 14/100, so the anchors stayed as they are.
Still mine, next in this order per the decision: part 2 test_async_delay_audiolisten(), part 3 test_vlc_playlist_gap (deterministic media clock).
Claiming: test_async_delay_audio determinism (part 2 of #838)
Part 1 (test_group_hide_transition_audio, #3000) landed; this is the second of the three PRs the Decided: on this issue names.
Re-anchor listen() (test/cmocka/test_async_delay_audio.c:427-441) on the mix's own clock: it records the first program-audio packet's data->timestamp after listening starts and stops after LISTEN_MS of mixed audio has gone past, bounded by a deadline, instead of sleeping LISTEN_MS of wall time.
Every assertion and threshold stays (samples >= 90% of RATE * LISTEN_MS, the tone-break and click-delay checks); on a starved machine the measurement waits instead of recording a shorter window.
Test-timing only, no product code. RED is the unmodified test under taskset plus a parallel build, GREEN is --repeat until-fail:100 on both ENABLE_RUST_LIBOBS=ON and OFF, in the warm Tier 2 volume.
Part 3 (test_vlc_playlist_gap, deterministic media clock) follows in its own PR.
Worker tick (obs-studio4): part 2 measured, and the decided approach does not match this test's failure. Reporting before writing code, and releasing the claim.
What I ran
Warm Tier 2 volume, ctest -R ^test_async_delay_audio$ under xvfb-run, on this checkout's branch off 08749f47c (master). I built both variants in the same volume, so only the test source varies.
Stressor
Variant
Runs
Failures
--cpus=2, 3 busy loops
unmodified master
3
0
--cpus=1, 4 busy loops
unmodified master
3
1 (tone_continuous_with_3s_video_delay)
--cpus=1, 3 busy loops, test at nice -n 19
unmodified master
3
1 (same case)
--cpus=0.15, no hogs
unmodified master
2
0
--cpus=1, 3 busy loops
re-anchored listen() (below)
3
1 (same case)
Failing output, both times identical in shape:
video delay 3000 ms: 192512 samples heard, 120321 loud, tone broken 1 times, mix skipped 0 times
video delay 3000 ms: the tone was broken up 1 times in 4000 ms
Why this is not the part-1 class
192512 samples over RATE = 48000 is 4.01 s of mix, i.e. the whole LISTEN_MS window arrived; the samples < 90% assertion never fired in any of the 11 local runs. So the wall-clock window is not what truncates this test.
The case that fails is the upstream-13830 assertion, not the "did not run" one: exactly one discontinuity inside the tone, with a full-length window.
I implemented the part-1 shape anyway to test the hypothesis: listen() records the first program packet's data->timestamp and waits for LISTEN_MS of mix time (bounded +5 s) instead of sleeping wall time, and the raw-audio callback primes the sine check again when the mix jumps more than one packet. Under the same stressor it still fails on the same case, and mix skipped 0 times says there is no timestamp skip to attribute the discontinuity to. The mix genuinely contains one gap.
So a test-side clock change does not remove this flake, and shipping it as "part 2" would be a PR that claims a fix the evidence does not support.
What the choice is
Treat the one-off gap as the environment — accept at most one isolated tone discontinuity per 4 s window (the original #13830 bug broke the tone repeatedly, so a bound of 1 keeps the guard), and report the count in the failure text. This is the Decided: "test-side determinism" intent applied to the assertion that actually fires.
I do not think a worker should pick either unilaterally: option 1 widens an assertion that exists to catch a real upstream bug, and the Decided: above only covers the wall-clock window.
Releasing: test_async_delay_audio determinism (part 2 of #838) — no branch pushed, nothing left in flight from this tick. Part 3 (test_vlc_playlist_gap, deterministic media clock) is untouched and can be taken independently if you would rather defer this one.
Claiming: test_vlc_playlist_gap determinism (part 3 of #838)
Part 1 (#3000) and the part-2 measurement are done; this is the last of the three PRs the Decided: names. Taking it as an investigation first, because part 2 showed the decided shape can miss the actual failure:
Reproduce test_vlc_playlist_gap in this checkout's warm Tier 2 volume under CPU starvation (--cpus=1 plus hogs, ctest --repeat until-fail:N), and record which of the two mechanisms fires: libVLC genuinely delivering fewer decoded frames (the Decided: comment's expectation) or the test mis-attributing samples when data->timestamp jumps past TS_SMOOTHING_THRESHOLD.
Only then write the test-side change: drive file position from the mix's own sample sequence if the samples are all there, and keep every assertion and threshold. That is the deterministic media clock the decision asks for, and it is test-timing only, with ENABLE_RUST_LIBOBS=OFF as the evidence.
RED is the unmodified test failing under the stressor; GREEN is --repeat until-fail:100 under the same stressor on both ENABLE_RUST_LIBOBS=ON and OFF. If the reproduction shows libVLC really dropping audio, I will report that and release instead of widening a tolerance, the same way part 2 was handled.
Worker tick (obs-studio4): part 3 measured, and the deterministic-clock approach has no lever in this test. Reporting before writing code, and releasing the claim.
What I ran
Warm Tier 2 volume for this checkout (obs-rust-linux-validate-build-e829c156d95d), current binary from 2026-10-30, host load 10-13 from the other workers, ctest -R '^test_vlc_playlist_gap$' --repeat until-fail:N -V under xvfb-run, container CPU capped with --cpus:
Container cap
Repeats
Failures
Timestamp jumps logged
--cpus=1
8
0
9
--cpus=0.25
4
0
several
(The 8-repeat run: 244 s total, 30.55 s per repeat, every repeat [ PASSED ] 8 test(s).)
So no RED, even at a quarter of a core, which is harsher than the 2-vCPU arm64 condition in the issue.
Why the decided shape does not fit this test
test_vlc_playlist_gap's measurement is already clock-free: analyze() (test/cmocka/test_vlc_playlist_gap.c:185-235) classifies each 1024-frame mix block by its peak level, derives the position inside the file from that level, and builds the per-file stats from the sequence of blocks, only using cap->mix_frames as an index. data->timestamp is never read. That is why the 129 ms TS_SMOOTHING_THRESHOLD jumps happen in almost every run here and the case still passes - the log lines are not what the assertions see.
The failure in the issue (file 2: the last 1.886 s are missing from the mix) is therefore blocks that never arrived, not blocks attributed to the wrong file. Making the media clock deterministic from the test side cannot invent audio libVLC did not deliver, and the only way to stop libVLC being real-time is to stop playing through libVLC, which would stop testing the product path.
The loss mechanism the Decided: comment names ("a starved media thread genuinely delivers fewer frames") is consistent with everything above; my runs did not reach it, so I have no RED to gate a change against, and per the policy in the same comment I am not widening TOLERANCE without one.
Suggested next step for the maintainer
(b) as written in the 2026-10-28 escalation: RUN_SERIAL (or a lane that is not shared with parallel builds) for this real-time binary - the cost is wall time on the gate, but it is the only lever that touches the actual mechanism.
Or accept the flake rate and reopen on the next occurrence; 11/11 under load is not evidence that 2026-10-15 cannot recur, only that it is rare enough that a stress loop does not find it here.
If a RED is wanted first, the condition to reproduce still looks like I/O + CPU (libVLC demux/decode on a starved filesystem), which neither of my runs applied; a --device-read-bps cap on the container would be the next thing to try.
No branch pushed. Releasing: test_vlc_playlist_gap determinism (part 3 of #838).
Activity
zackees commentedon Oct 15, 2026
Two more one-off failures in a loaded local Tier 2 run (OFF build, 2026-10-15 ~22:50Z, unrelated change #918):
test_async_delay_audio(test_async_delay_audio.c:466), another real-time audio test.test_v4l2_busy_signalforce_camera_closes_the_holder_and_opens_the_camera(test_v4l2_busy_signal.c:296,10185 != 0): the forced holder process had not been reaped yet when the test checked. Not audio, but the same pattern: a wall-clock race that loses under load.zackees commentedon Oct 16, 2026
Another one, video this time:
test_deinterlace_2x_field_order(discard_2x_bottom_first_60i_into_50,yadif_2x_top_first_unbuffered_60i_into_60;test_deinterlace_2x_field_order.c:370) failed once in a loaded local Tier 2 run (OFF build, 2026-10-16 ~01:50Z, unrelated change 965):12 fields skipped, up to 2 in a row (allowed 1). Same wall-clock pattern: real-time frame cadence loses under machine load.zackees commentedon Oct 16, 2026
Another one:
test_vlc_playlist_gapfailed (28.6 s, ***Failed) in Tier 2 Linux (ON) on the merge-queue run for #936 (job 116819150784, 2026-10-16 05:13 UTC). #936 is a text-freetype2 refactor and touches nothing in vlc-video. It knocked #936 out of the queue; I re-queued it.zackees commentedon Oct 16, 2026
Add
test_media_restart_stale_frameto the list: failed once in a heavily loaded local Tier 2 run (OFF, 2026-10-16 ~05:50Z, unrelated change 997), together withtest_async_delay_audioandtest_deinterlace_2x_field_order. Separately,test_v4l2_busy_signal(also logged here earlier) was a fixture race rather than load sensitivity; #953 / 997 fixes it.zackees commentedon Oct 16, 2026
These flakes are now costing merges: #936 (text-freetype2, no media code) was removed from the merge queue twice for
failed_checks. The 05:19Z removal wastest_vlc_playlist_gapstop_at_track_end_is_prompt(test_vlc_playlist_gap.c:476) in merge-group run 38850208674, Rust Tier 2 Linux (ON). The queue's Tier 2 runs every real-time test against a busy runner, so each of these is a coin flip on every merge.zackees commentedon Oct 16, 2026
test_vlc_playlist_gapstop_at_track_end_is_promptis not load sensitivity: it is a real race in vlc-video (an async libVLC Opening event resetsdrain_abortafter Stop signals it), filed and claimed as 1011. The other cases here (audio/video cadence under load) are still open.zackees commentedon Oct 16, 2026
test_media_restart_stale_frameclear_still_clearsis a crash, not a timing flake:obs_shutdown()segfaults afterobs_reset_video()failed ("Unable to open X display"). Filed as 1032.zackees commentedon Oct 16, 2026
A common cause for part of this list: in loaded local Tier 2 runs (2026-10-16 ~07:30Z and ~08:10Z), single cases fail setup with
info: Unable to open X displaywhile the neighbouring cases in the same binary pass, e.g.test_v4l2_busy_signalforce_camera_opens_while_the_holder_is_still_exiting("Could not run test: Test setup failed") andtest_media_restart_stale_frame(which then crashes inobs_shutdown, #1032). So the Xvfb server fromxvfb-run -aintermittently isn't reachable while ctest runs tests in parallel (-j4). That's an environment fault, not load sensitivity in the tested code; worth fixing in the harness (one Xvfb per test, or retry on display-open failure) alongside #982. Separately,test_vlc_playlist_gapstop_at_track_end_is_promptwas a real vlc-video race: #1011, fixed in #1037.zackees commentedon Oct 16, 2026
Correction to my earlier comment: "Unable to open X display" is not evidence that Xvfb was unreachable. It is printed by
obs_nix_x11_log_info()(libobs/obs-nix-x11.c:39) wheneverobs_get_nix_platform_display()is NULL, i.e. whenever nothing calledobs_set_nix_platform_display(), which only the frontend does. Every cmocka test that callsobs_startuplogs it, including passing ones (the nexttest_v4l2_busy_signalcase logged it and then registered X11 hotkey keycodes fine). So the "Xvfb dropping out under parallel ctest" explanation I gave is wrong and should be disregarded; the cause of these failures is still unknown. For the list here:test_v4l2_busy_signal's "Test setup failed" case andtest_media_restart_stale_frame's segfault both still need a real cause.zackees commentedon Oct 16, 2026
Another hit, 2026-10-16 19:40 UTC, in the merge queue:
test_vlc_playlist_gap/playlist_of_three_plays_every_file_to_the_endfailed in Rust Tier 2 Linux (ON 2/3) of run 38956680388 (merge group for #1111, a linux-pipewire change unrelated to VLC). Log:Audio timestamp for 'vlc' exceeded TS_SMOOTHING_THRESHOLD, diff=127413000 ns, thenfile 1: the last 1.056 s are missing from the mix. The OFF lane passed. This kicked #1111 out of the queue; it's re-queued.zackees commentedon Oct 17, 2026
Another one:
test_media_seek_keyframe/playing_seek_between_keyframesfailed attest_media_seek_keyframe.c:629in Rust Tier 2 macos-26 on the merge-queue run for #1292 (run 39055672620). #1292 only touchesplugins/frontend-tools, so this is unrelated to it. Real-time media playback under queue load, like the others here.test: ingest refresh wait bounds compare against the timeout, not 150…
zackees commentedon Oct 17, 2026
Another observed instance: master run 39078519174, OFF job 117532901565, failed
test_group_hide_transition_audio(9.14 s). Three subcases (hide_child_with_transitions,hide_group_with_child_transitions,hide_group_during_child_show_transition) reportedno program audio was mixed, failing line 216 after the fixed one-second mix observation window. 293 CTest registrations; one executable failed.OBS startup succeeded, created the main canvas and initialized llvmpipe OpenGL. This is separate from #1355's locally reproduced XOpenDisplay/startup failure. The informational
Unable to open X displaymessage precedes successful startup here too. No new Xvfb attribution, skip, assertion change or blind rerun was made. Raw log is retained under ignoredbuild_ci_audit/master-e89-off-failure.logduring the #1286 audit.zackees commentedon Oct 18, 2026
Additional live failure in master run 39084293679, revision
d27d69f585eec876b3d90a18b74ca69194422cba, OFF job 117549418999: unchangedtest_group_hide_transition_audiofailed all four cases at line 216 with no program audio was mixed. Suite registered 295 tests, one failing executable. Startup-failure diagnostics passed; the run failed this OFF job. No retry, skip or assertion relaxation.This recurs after a different full manual run (39082733551) passed both modes. A green full run therefore does not establish absence of this intermittent integrity failure. This is separate from #1355's verified Xvfb startup fix and #1283's recording-shutdown state race. Logs retained locally at build_ci_audit/master-1373-off-failure-117549418999.log while continuing #1286 cache validation. No cache-layer root cause is claimed.
zackees commentedon Oct 18, 2026
Claiming: investigate only the test_group_hide_transition_audio scheduling portion while completing #1286. Compare the unchanged executable in isolated versus concurrent execution on the existing local Linux build, preserve all C assertions, and record skips/environment failures separately. If resource isolation is supported by reproduction, add focused scheduling regression coverage and a narrow CTest change; do not blanket-serialize media tests or close the unrelated VLC/seek cases.
31 remaining items
test(cmocka): anchor the group-hide program meter to the mix's clock (#…
test(cmocka): wait for the mix to reach the window's end, not a fixed…
zackees commentedon Oct 30, 2026
Part 1 landed: #3000 (squash-merged) re-anchors
program_level()'s wait on the mix's own clock. The window keeps its original anchors, so it still measures the same 400 ms of program audio; what changed is that it waits for the mix to reach the window's end (bounded at 5 s) instead of sleeping 1000 ms.Evidence in the PR, from the Tier 2 harness on a host at load average 34:
1.00on--repeat until-fail:100for this test withENABLE_RUST_LIBOBS=OFFandON, and the unmodified 623-test suite green in both modes. The first attempt on that branch opened the window on the first arriving frame and the 100-repeat run caught a finished show transition inside it (0.89 of the tone) at 14/100, so the anchors stayed as they are.Still mine, next in this order per the decision: part 2
test_async_delay_audiolisten(), part 3test_vlc_playlist_gap(deterministic media clock).zackees commentedon Oct 30, 2026
Claiming: test_async_delay_audio determinism (part 2 of #838)
Part 1 (
test_group_hide_transition_audio, #3000) landed; this is the second of the three PRs theDecided:on this issue names.listen()(test/cmocka/test_async_delay_audio.c:427-441) on the mix's own clock: it records the first program-audio packet'sdata->timestampafter listening starts and stops afterLISTEN_MSof mixed audio has gone past, bounded by a deadline, instead of sleepingLISTEN_MSof wall time.samples >= 90%ofRATE * LISTEN_MS, the tone-break and click-delay checks); on a starved machine the measurement waits instead of recording a shorter window.tasksetplus a parallel build, GREEN is--repeat until-fail:100on bothENABLE_RUST_LIBOBS=ONandOFF, in the warm Tier 2 volume.test_vlc_playlist_gap, deterministic media clock) follows in its own PR.zackees commentedon Oct 30, 2026
Worker tick (
obs-studio4): part 2 measured, and the decided approach does not match this test's failure. Reporting before writing code, and releasing the claim.What I ran
Warm Tier 2 volume,
ctest -R ^test_async_delay_audio$underxvfb-run, on this checkout's branch off08749f47c(master). I built both variants in the same volume, so only the test source varies.--cpus=2, 3 busy loops--cpus=1, 4 busy loopstone_continuous_with_3s_video_delay)--cpus=1, 3 busy loops, test atnice -n 19--cpus=0.15, no hogs--cpus=1, 3 busy loopslisten()(below)Failing output, both times identical in shape:
Why this is not the part-1 class
192512samples overRATE = 48000is 4.01 s of mix, i.e. the wholeLISTEN_MSwindow arrived; thesamples < 90%assertion never fired in any of the 11 local runs. So the wall-clock window is not what truncates this test.listen()records the first program packet'sdata->timestampand waits forLISTEN_MSof mix time (bounded +5 s) instead of sleeping wall time, and the raw-audio callback primes the sine check again when the mix jumps more than one packet. Under the same stressor it still fails on the same case, andmix skipped 0 timessays there is no timestamp skip to attribute the discontinuity to. The mix genuinely contains one gap.So a test-side clock change does not remove this flake, and shipping it as "part 2" would be a PR that claims a fix the evidence does not support.
What the choice is
Decided:"test-side determinism" intent applied to the assertion that actually fires.I do not think a worker should pick either unilaterally: option 1 widens an assertion that exists to catch a real upstream bug, and the
Decided:above only covers the wall-clock window.Releasing: test_async_delay_audio determinism (part 2 of #838)— no branch pushed, nothing left in flight from this tick. Part 3 (test_vlc_playlist_gap, deterministic media clock) is untouched and can be taken independently if you would rather defer this one.zackees commentedon Oct 30, 2026
Releasing: test_async_delay_audio determinism (part 2 of #838)
No branch pushed; see the measurement above. Happy to take it back once the assertion-vs-product choice is made.
test(cmocka): anchor the group-hide program meter to the mix's clock (o…
zackees commentedon Oct 31, 2026
Claiming: test_vlc_playlist_gap determinism (part 3 of #838)
Part 1 (#3000) and the part-2 measurement are done; this is the last of the three PRs the
Decided:names. Taking it as an investigation first, because part 2 showed the decided shape can miss the actual failure:test_vlc_playlist_gapin this checkout's warm Tier 2 volume under CPU starvation (--cpus=1plus hogs,ctest --repeat until-fail:N), and record which of the two mechanisms fires: libVLC genuinely delivering fewer decoded frames (theDecided:comment's expectation) or the test mis-attributing samples whendata->timestampjumps pastTS_SMOOTHING_THRESHOLD.ENABLE_RUST_LIBOBS=OFFas the evidence.--repeat until-fail:100under the same stressor on bothENABLE_RUST_LIBOBS=ONandOFF. If the reproduction shows libVLC really dropping audio, I will report that and release instead of widening a tolerance, the same way part 2 was handled.No branch pushed yet; first PR this session.
zackees commentedon Oct 31, 2026
Worker tick (
obs-studio4): part 3 measured, and the deterministic-clock approach has no lever in this test. Reporting before writing code, and releasing the claim.What I ran
Warm Tier 2 volume for this checkout (
obs-rust-linux-validate-build-e829c156d95d), current binary from 2026-10-30, host load 10-13 from the other workers,ctest -R '^test_vlc_playlist_gap$' --repeat until-fail:N -Vunderxvfb-run, container CPU capped with--cpus:--cpus=1--cpus=0.25(The 8-repeat run: 244 s total, 30.55 s per repeat, every repeat
[ PASSED ] 8 test(s).)So no RED, even at a quarter of a core, which is harsher than the 2-vCPU arm64 condition in the issue.
Why the decided shape does not fit this test
test_vlc_playlist_gap's measurement is already clock-free:analyze()(test/cmocka/test_vlc_playlist_gap.c:185-235) classifies each 1024-frame mix block by its peak level, derives the position inside the file from that level, and builds the per-file stats from the sequence of blocks, only usingcap->mix_framesas an index.data->timestampis never read. That is why the 129 msTS_SMOOTHING_THRESHOLDjumps happen in almost every run here and the case still passes - the log lines are not what the assertions see.The failure in the issue (
file 2: the last 1.886 s are missing from the mix) is therefore blocks that never arrived, not blocks attributed to the wrong file. Making the media clock deterministic from the test side cannot invent audio libVLC did not deliver, and the only way to stop libVLC being real-time is to stop playing through libVLC, which would stop testing the product path.The loss mechanism the
Decided:comment names ("a starved media thread genuinely delivers fewer frames") is consistent with everything above; my runs did not reach it, so I have no RED to gate a change against, and per the policy in the same comment I am not wideningTOLERANCEwithout one.Suggested next step for the maintainer
RUN_SERIAL(or a lane that is not shared with parallel builds) for this real-time binary - the cost is wall time on the gate, but it is the only lever that touches the actual mechanism.--device-read-bpscap on the container would be the next thing to try.No branch pushed. Releasing: test_vlc_playlist_gap determinism (part 3 of #838).