Skip to content

gh-140267: Fixed thread leak in multiprocessing.Manager accepter thread #141734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
prithviraj-chaudhuri wants to merge 9 commits into from
Closed

gh-140267: Fixed thread leak in multiprocessing.Manager accepter thread #141734

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
cb8e422
Waiting for the accepter thread to finish when the serve_forever meth…
prithviraj-chaudhuri Nov 19, 2025
4862fb8
Updated test_mymanager_context_prestarted to expect 0 or -signal.SIGT…
prithviraj-chaudhuri Nov 19, 2025
83014b5
📜🤖 Added by blurb_it.
blurb-it[bot] Nov 19, 2025
d7df812
Added timeout loop
prithviraj-chaudhuri Nov 22, 2025
a224c6f
Added timeout and joining daemon thread
prithviraj-chaudhuri Nov 23, 2025
7a64e68
Changed timeout to 0.5
prithviraj-chaudhuri Nov 26, 2025
22fb4b6
Closing handler threads before accepter and after socket close
prithviraj-chaudhuri Nov 27, 2025
49cdc22
Closing listener
prithviraj-chaudhuri Nov 28, 2025
6db7ec3
≈≈Merge branch 'main' into fix-issue-140267
prithviraj-chaudhuri Nov 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Lib/multiprocessing/managers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,7 @@ def serve_forever(self):
util.debug('resetting stdout, stderr')
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
accepter.join()
sys.exit(0)
Copy link
Copy Markdown
Member

@gpshead gpshead Nov 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fundamentally, the accepter and handle_request threads were all being set .daemon = True as they were never intended to be controlled and cleaned up. they'll all die as soon as the server process exits. attempting to join all of the threads is perhaps ideal but only works if they can be guaranteed not to be blocked on anything. Handler threads do blocking IO to a socket connection which is not guaranteed to have been closed because it could come from an uncooperative peer. without rewriting them to do non-blocking IO and check a stop_event themselves, joining them can't be reliable and could lead to anything making a connection to the socket being able to block cleanup.

This is likely why you see test failures for leftover env files with pymp- prefixed tmp directories or Windows \\pipe paths still laying around such as Warning -- test.test_concurrent_futures.test_process_pool leaked temporary files (1): pymp-cdk1jdmh with this change. Those are created (by Server for its self.listener via the Listener use of default_family and call to arbitrary_address(family)) to hold the socket that the manager process uses. If the manager process is blocked instead of exiting cleanly it may still exist.


def accepter(self):
Expand Down
2 changes: 1 addition & 1 deletion Lib/test/_test_multiprocessing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3417,7 +3417,7 @@ def test_mymanager_context_prestarted(self):
manager.start()
with manager:
self.common(manager)
self.assertEqual(manager._process.exitcode, 0)
self.assertIn(manager._process.exitcode, (0, -signal.SIGTERM))

def common(self, manager):
foo = manager.Foo()
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2025-11-19-05-43-58.gh-issue-140267.DTFG1E.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fixed thread leak in multiprocessing.Manager accepter thread by adding accepter.join() when before the serve_forever() method exits.
Fixed test_mymanager_context_prestarted test to expect either an exit code 0 or -signal.SIGTERM
Loading