Repository navigation
Description
Found while investigating #2432 (the arm64 packaged-startup SIGSEGV). This is a standalone defect; it is not claimed to be the cause of that crash.
The bug
plugins/obs-websocket/src/websocketserver/WebSocketServer.cpp guards _sessions with _sessionMutex everywhere except one place, and then waits on that unguarded state without a bound.
_sessions is a std::map (WebSocketServer.h:99), _sessionMutex is its lock (WebSocketServer.h:98). Locked uses: GetWebSocketSessions (:235), onOpen insert (:277-278), onClose erase (:348-357), Stop's close loop (:185-200).
Stop()'s wait loop is not locked (:205-206):
std::unique_lock<std::mutex> lock(_sessionMutex);
for (auto const &[hdl, session] : _sessions) { ... }
lock.unlock();
_threadPool.waitForDone();
while (_sessions.size() > 0) // :205 — no lock
std::this_thread::sleep_for(std::chrono::milliseconds(10));
_serverThread.join(); // :208The IO thread (ServerRunner, :104-117) calls onClose, which erases from the same map under the mutex, while the main thread reads size() with no lock. Concurrent size() against std::map::erase (node rebalancing) is a data race and undefined behaviour.
It is also unbounded: Stop() returns only after _sessions is empty, so a session whose close handshake never completes keeps obs_module_unload (src/obs-websocket.cpp:134-163) from returning. In the packaged-startup smoke test that shows up as the harness's 30 s shutdown budget expiring (test/packaged-startup.py:601) rather than as a crash.
Why it matters
Stop() is on the process-shutdown path (obs_module_unload), so a race or a stuck wait here fails the Native macOS/Windows DMG startup and Ubuntu/Windows app lanes at the very end of an otherwise green run. #2432's arm64 -11 log stops immediately after [obs_module_unload] Shutting down..., which is this function, before any of its three completion lines — a sample of the x86_64 hang in that same lane shows the other teardown race (rtmp-services' update join, #2672, already fixed).
Proposed fix
In obs-rust/obs-websocket (its own PR, then a submodule pointer bump here — never commit the pointer separately from the bump PR):
- Test the predicate under the mutex, releasing it between polls, and prefer a
condition_variable:onClosenotifies after erasing,Stop()waits on!_sessions.empty(). - Bound the wait (a few seconds) and log a warning naming the sessions that did not close, so unload cannot hang.
- Keep
_serverThread.join()after the wait; do not hold_sessionMutexacrossjoin().
Tests
- A unit test is impossible against the real class without a client, so the RED is a Tier 2/OBS-level one: two clients, one of them never answering the close frame, and
deinitmust return within the bound with a warning instead of hanging. Today it spins forever (asampleof the hang shows the busy-wait loop). - TSan would flag the unsynchronized
size():sanitizers.yaml -f tools=tsanon a build with a client that disconnects duringStop().
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsDone
Activity
zackees commentedon Oct 28, 2026
/ready
Triaged against the definition of Ready (WORKFLOW.md §Definition of Ready):
deinitreturns within the bound and logs a warning; plus a TSan run (sanitizers.yaml -f tools=tsan) for the unsynchronizedsize()._sessionsis only read while holding_sessionMutex(or via acondition_variablepredicate), (b)Stop()cannot wait unbounded: it bounds the wait and logs which sessions did not close, (c)_serverThread.join()still happens after the wait and is never called while holding_sessionMutex, (d) the busy-waitwhile (_sessions.size() > 0)is gone.obs-rust/obs-websocketas its own PR, then a submodule pointer bump here in the same PR — no blocker to link.P2(a teardown-path race that fails the packaged DMG/app lanes at the very end of an otherwise green run, not a red master) andsize:S(one function, one submodule bump).Scope note for whoever claims it: the arm64 SIGSEGV in #2432 is not claimed to be this bug; this is a standalone defect found while investigating it.
zackees commentedon Oct 28, 2026
Claiming: fix WebSocketServer::Stop's unguarded, unbounded
_sessionswaitPlan, two PRs (the submodule owns the fix; this repo owns the pointer bump):
obs-rust/obs-websocket: RED first — a test that drives the realWebSocketServerwith a client that never answers the close frame and assertsStop()returns within the bound instead of spinning (plus a TSan-visible predicate read under the lock). GREEN:condition_variablenotified fromonCloseafter the erase, polled under_sessionMutex, bounded wait with a warning that names the sessions still open,_serverThread.join()kept after the wait and never under the lock.obs-rust/obs-studio: bump theplugins/obs-websocketpointer to that commit, and record the teardown behaviour through the packaged-startup harness (test/packaged-startup.py) or state plainly what could not be run here.Neither PR advances
master; both go through the queue on their own repo's terms.zackees commentedon Oct 28, 2026
Status: the submodule fix is up for review — obs-rust/obs-websocket#22 (
fix/websocket-server-stop-drain, headf1d1417).What it does, matching this issue's proposed fix:
onClose()notifies astd::condition_variableimmediately after it erases from_sessions(and after releasing_sessionMutex).Stop()replaces the unlockedwhile (_sessions.size() > 0) sleep(10ms)poll withwait_foron that variable, predicate evaluated under_sessionMutex, bounded at 5 s.LOG_WARNING ... %zu session(s) still open after 5000 ms, continuing shutdown: <remote addresses>so an abandoned shutdown is visible in the log instead of being a silent hang, and the module still unloads._serverThread.join()stays after the wait; the mutex is not held across it.Verified so far:
clang-format --dry-run --Werrorclean, andclang++ -fsyntax-only -Wall -std=c++17clean against the real headers (websocketpp 0.8.2 / asio 1.36 / Qt 6.11 / OBSlibobs+frontend-api). Not verified at runtime: that repo has no test target and no build CI, and the racy path needs a live client, so the RED/GREEN has to come from the obs-studio side (Tier 2 + packaged startup load and shut this plugin down, and the new warning distinguishes "drained slowly" from "hung").Next, once #22 is merged: the pointer bump PR here —
plugins/obs-websocketto that merge commit plus aci/test_*.pypinning this shape (drain predicate read under the mutex, the wait is bounded,join()outside the lock), RED against the current pointer. I am not committing the pointer on its own (PR CI rejects a pointer that does not descend from master's).Separate defect noticed while reading
onClose():SessionPtr session = _sessions[hdl];usesoperator[], so a close for a handle that was never opened (e.g.onOpen()returned early because the config was unavailable) inserts a nullSessionPtrand the next line dereferences it. Not folded into #22; I will file it as its own issue.zackees commentedon Oct 28, 2026
Follow-up landed on the obs-studio side: #2722 bumps
plugins/obs-websocket30d6f511→3253206(merge of obs-rust/obs-websocket#22) and pins the shape offline.RED
ca8abb5f3(test only):uv run --no-project -m unittest ci/test_websocket_submodule.py→FAILED (failures=3, errors=1)— the pointer predates #22, andSessionDrainShapeTestfails on the unlocked
while (_sessions.size() > 0)poll, the missing_sessionsDrained.wait_for(/kSessionDrainTimeout, and the missing "still open after"warning.
GREEN
819d3f5cd:Ran 5 tests ... OK (skipped=1);ruff checkclean;./lintok (34.5 s).The one skip is the existing bounded-parser ancestry check (shallow submodule); the new
pointer check no longer skips merely because the checkout is shallow — it only skips when the
fix commit object is absent too.
Honest gap, per the issue's test plan: no runtime RED/GREEN here. That path needs a live
client that ignores the close frame, and obs-rust/obs-websocket has no test target. What the
packaged-startup and Tier 2 lanes now give us is a distinguishing signal instead of a hang:
[WebSocketServer::Stop] <n> session(s) still open after 5000 ms, continuing shutdown: <addresses>.TSan was not run either.
docs/features.mdcarries the ledger line ("pending merge"). I have not closed this issue:the runtime/Tier 2 evidence and the TSan run are still open, and #2722 is in the merge queue
(handed off to the coordinator).
zackees commentedon Oct 28, 2026
The parent-repo pointer this issue needs is #2723:
plugins/obs-websocket3253206->777071b(submodule master, i.e. #22 plus #23'sonClose()null-session fix for #2720).#2722stopped at3253206; the submodule moved one commit further after that.uv run --no-project -m unittest ci/test_websocket_submodule.pypasses at777071b(the bounded-parser, bounded-drain andSessionDrainShapeTestpins), so #22's fix is present as well. Auto-merge is on. Once #2723 is on master, this issue's code change is complete and #2432 can close on the next daily/verify run if the-11does not return.zackees commentedon Oct 28, 2026
Closed: the fix is on
master.plugins/obs-websocketnow points at777071b5db3dc3cf0fd62f3cea5a2a3ac7992af2(git ls-tree origin/master plugins/obs-websocket), which is obs-rust/obs-websocketmaster(compare/777071b...master→ identical).3253206):Stop()reads the drain predicate under_sessionMutexon_sessionsDrainedand bounds the wait atkSessionDrainTimeout(5 s), logging the sessions it gave up on. The unlockedwhile (_sessions.size() > 0)poll is gone.3253206(libobs: Preserve extensions and UTF-8 boundaries in generated filenames #22), and fix(obs-websocket): Point the submodule at the onClose null-session fix (#2720) #2723 carried the one-commit follow-up to777071b(libobs/util: Keep UTF-8 valid when truncating formatted filenames #23, theonClose()null-session fix for bug: obs-websocket: onClose() inserts a null session with _sessions[hdl] and dereferences it #2720).ci/test_websocket_submodule.py—test_pointer_keeps_the_bounded_session_drainplusSessionDrainShapeTest(predicate under mutex, bounded timeout + log, erase-before-notify) — passes at this pointer in PR CI; the ledger line is indocs/features.md.No compiled lane covers this submodule (Tier 2 configures
ENABLE_WEBSOCKET=OFF), so the remaining evidence is the release-gate app build; the crash symptom this was found from is tracked separately in #2432.fix(obs-websocket): Bump the submodule to the bounded session drain (o…
fix(obs-websocket): Point the submodule at the onClose null-session f…