ThreadSanitizer reports thread leak in multiprocessing.Manager accepter thread

[ashm-dev]

Bug report

Bug description:

How to Reproduce

CC=clang CXX=clang++ ./configure --with-thread-sanitizer --with-pydebug --with-lto=full && make -j8

TSAN_OPTIONS=handle_segv=0 ./python -X dev -m test test_multiprocessing_forkserver.test_manager -j8

ThreadSanitizer Output

WARNING: ThreadSanitizer: thread leak (pid=191832)
  Thread T13 'Thread-13 (hand' (tid=192190, finished) created by thread T1 at:
    #0 pthread_create <null> (python+0xf821e) (BuildId: 8cf5a2f67fcf002f5a851af0abefbcffdc70b4d7)
    #1 do_start_joinable_thread /home/shamil/oss/cpython/main/Python/thread_pthread.h:281:14 (python+0x694fe1)
    #2 PyThread_start_joinable_thread /home/shamil/oss/cpython/main/Python/thread_pthread.h:323:9 (python+0x7b3fb1)
    #3 ThreadHandle_start /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:474:9 (python+0x7b3fb1)
    #4 do_start_new_thread /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:1920:9 (python+0x7b3fb1)
    #5 thread_PyThread_start_joinable_thread /home/shamil/oss/cpython/main/./Modules/_threadmodule.c:2043:14 (python+0x7b1df8)
    [... rest of stack trace ...]

SUMMARY: ThreadSanitizer: thread leak

Reproduces: Every run

Observations

1. Current TSAN suppressions (Tools/tsan/suppressions.txt):

# This file contains suppressions for the default (with GIL) build.
# reference: https://github.com/google/sanitizers/wiki/ThreadSanitizerSuppressions

# https://gist.github.com/mpage/daaf32b39180c1989572957b943eb665
thread:pthread_create

This suppression masks all thread leaks starting from pthread_create.

2. Code analysis of Lib/multiprocessing/managers.py:

The Server.accepter thread runs an infinite loop:

def accepter(self):
    while True:  # Line ~197
        try:
            c = self.listener.accept()
        except OSError:
            continue
        t = threading.Thread(target=self.handle_request, args=(c,))
        t.daemon = True
        t.start()

The Server.serve_forever method sets self.stop_event:

def serve_forever(self):
    self.stop_event = threading.Event()
    # ...
    accepter = threading.Thread(target=self.accepter)
    accepter.daemon = True
    accepter.start()
    try:
        while not self.stop_event.is_set():  # Line ~189
            self.stop_event.wait(1)

The Server.shutdown method sets the stop event:

def shutdown(self, c):
    # ...
    finally:
        self.stop_event.set()  # Line ~235

3. Handler threads (created in accepter) check stop_event:

def serve_client(self, conn):
    while not self.stop_event.is_set():  # Line ~259
        # handle requests

Analysis

The accepter thread uses while True and does not check self.stop_event, unlike:

• The main loop in serve_forever (line ~189)
• The handler threads in serve_client (line ~259)

When shutdown() is called and stop_event is set, the accepter thread continues running because it never checks the event.

The test teardown (Lib/test/_test_multiprocessing.py, ManagerMixin.tearDownClass) calls:

cls.manager.shutdown()
cls.manager.join()

ThreadSanitizer detects that the daemon accepter thread was created but not properly joined/terminated.

Additional Context

• The broad thread:pthread_create suppression in Tools/tsan/suppressions.txt currently masks this issue
• Similar test cleanup code exists in BaseMixin.tearDownClass which warns about dangling threads
• The thread name in the TSAN output 'Thread-13 (hand' corresponds to handler threads created by the accepter

CPython versions tested on:

3.15

Operating systems tested on:

Linux

Linked PRs

• gh-140267: Fixed thread leak in multiprocessing.Manager accepter thread #141734

[prithviraj-chaudhuri]

Taking a look at this issue. Can this be assigned to me?

[gpshead]

multiprocessing.managers uses daemon threads for the server accept loop and the request handlers which are by definition likely to wind up as a "thread leak" as the design using daemon threads to make blocking calls rely on not cleaning up their threads but instead just ending the entire process.

I don't think there is a real issue here? The manager code could be rewritten to use non-blocking IO but it'd be more complicated and not solve a real problem.

Rather than blindly chasing TSAN errors, always consider their context. The AI generated analysis that you opened the issue with was only partially accurate but missed the bigger picture.

We'd consider it more of a real problem to leak a thread from within a library that could be inside an individual users process, but the multiprocessing manager process is stand alone server process code, not for use as a library; it does not impact anything else.

[gpshead]

see also #141734 (comment) - if someone wants to take on rewriting the manager process to not use threads at all, that could be interesting. it's not fixing a bug, but might wind up being more maintainable code.