You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
An exit watchdog: OBS ends its process when a plugin hangs after shutdown
#4051
OBS can finish its own shutdown and still not quit. In #3739, the log ends with Number of memory leaks, and the process stays alive with no window. The hang was in a static destructor of a third-party plugin: at exit(), a library in the plugin joined a thread that never stopped. OBS had no code left on the stack.
OBS cannot stop one plugin: a plugin is a shared library in the OBS process, not a process of its own. libobs also never unloads modules (os_dlclose is commented out in free_module), so the destructors of all plugins run at exit(), after main() returns.
For the user, the result is a process that never ends: it can block the next start, a restart, or a system shutdown. For us, it is a long hunt: #3739 needed a test machine, the plugin set of the reporter and a debugger to find the plugin.
Idea
A watchdog for the last step of the process:
In frontend/obs-main.cpp.hpp, after obs_main_host_shutdown() and the Number of memory leaks line, OBS starts a detached thread with a timeout of some seconds.
If the process still runs when the timeout ends, the thread writes one line ("exit did not finish in N s: a static destructor or atexit handler is blocked"), flushes it, and calls _exit().
A later step can name the module: the watchdog finds the shared library where the main thread waits (for example with backtrace() and dladdr from a signal to the main thread) and writes its name in the same line.
There is a precedent in this tree: obs-browser's obs_browser_finish_shutdown already calls std::_Exit(EXIT_SUCCESS) at the same place when CEF startup was abandoned (#3846, obs-browser#19).
What it does not do
It does not make a plugin's own shutdown code safe. A hang in obs_module_unload or in a source's destroy callback happens before this point and needs a different bound.
It does not run plugins in their own processes. That would break the C plugin ABI that this fork keeps.
Questions
Timeout: a fixed value (for example 5 s), or a setting?
Exit code after the watchdog fires: 0, or a code that shows a forced exit to scripts and service managers?
The log: at this point the frontend has removed its log handler (base_set_log_handler(nullptr, nullptr)). Should the log file stay open until the watchdog is done, or is a line on stderr enough?
Sanitizer builds: an early _exit() skips LeakSanitizer's report at exit. Should the watchdog be off when the build has a sanitizer, or off by an environment variable?
Windows and macOS: the same idea with TerminateProcess / _exit, or Linux first?
Naming the module (step 3): is a signal handler in the main thread acceptable for this, or is the timeout line alone enough?
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Problem
OBS can finish its own shutdown and still not quit. In #3739, the log ends with
Number of memory leaks, and the process stays alive with no window. The hang was in a static destructor of a third-party plugin: atexit(), a library in the plugin joined a thread that never stopped. OBS had no code left on the stack.OBS cannot stop one plugin: a plugin is a shared library in the OBS process, not a process of its own. libobs also never unloads modules (
os_dlcloseis commented out infree_module), so the destructors of all plugins run atexit(), aftermain()returns.For the user, the result is a process that never ends: it can block the next start, a restart, or a system shutdown. For us, it is a long hunt: #3739 needed a test machine, the plugin set of the reporter and a debugger to find the plugin.
Idea
A watchdog for the last step of the process:
frontend/obs-main.cpp.hpp, afterobs_main_host_shutdown()and theNumber of memory leaksline, OBS starts a detached thread with a timeout of some seconds._exit().backtrace()anddladdrfrom a signal to the main thread) and writes its name in the same line.There is a precedent in this tree: obs-browser's
obs_browser_finish_shutdownalready callsstd::_Exit(EXIT_SUCCESS)at the same place when CEF startup was abandoned (#3846, obs-browser#19).What it does not do
obs_module_unloador in a source's destroy callback happens before this point and needs a different bound.Questions
base_set_log_handler(nullptr, nullptr)). Should the log file stay open until the watchdog is done, or is a line on stderr enough?_exit()skips LeakSanitizer's report at exit. Should the watchdog be off when the build has a sanitizer, or off by an environment variable?TerminateProcess/_exit, or Linux first?All reactions