Potential thread unsafety in test_free_threading.test_monitoring

[emmatyping]

Bug report

Bug description:

It seems that there is some kind of race with test_free_threading.test_monitoring, specifically the sys.settrace tests:

cpython/Lib/test/test_free_threading/test_monitoring.py

Lines 143 to 166 in fba5dde

	@threading_helper.requires_working_threading()
	class SetTraceMultiThreaded(InstrumentationMultiThreadedMixin, TestCase):
	"""Uses sys.settrace and repeatedly toggles instrumentation on and off"""
	def setUp(self):
	self.set = False
	self.called = False
	def after_test(self):
	self.assertTrue(self.called)
	def tearDown(self):
	sys.settrace(None)
	def trace_func(self, frame, event, arg):
	self.called = True
	return self.trace_func
	def during_threads(self):
	if self.set:
	sys.settrace(self.trace_func)
	else:
	sys.settrace(None)
	self.set = not self.set

.

This was initially spotted in a buildbot failure here: https://buildbot.python.org/#/builders/1610/builds/1590. There have been a few behaviors noted:

• refleaks causing a test failure
• assertion failures
• presumed deadlocks or other contention causing the tests to run extremely slowly (timing out after 45 minutes)
• tests pass without issue 😬

The variety of failure modes makes me think there is some kind of thread safety issue lurking behind the failures...

After some local debugging (the remote debugger rocks!), I was able to reproduce the extremely long test run of test_freethreading, and narrow down the majority of the time being in the sys.settrace tests. My command line was:

./python -E  -m test --timeout=2700 -R 3:3 -u-cpu -j10 test_free_threading

On my x86 laptop on Ubuntu 24.04/WSL.

Reading over the test, it isn't clear to me what is meant to be tested. sys.settrace only affects the current thread according to the docs (indeed, the C code sets the trace function to the current threadstate), so currently it is only checking that the main thread executes a frame of some sort.

I didn't have time to find the root cause of the weird behavior above, so I'm filing this bug to note down my findings.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

[colesbury]

Thanks for the bug report.

Reading over the test, it isn't clear to me what is meant to be tested. sys.settrace only affects the current thread according to the docs

sys.settrace is implemented on top of PEP 669 monitoring, which is global for the interpreter. The monitoring callbacks are then filtered so that it only affects the relevant thread, but there are still all the thread-safety issues of enabling/disabling monitoring globally, which is what the test tries to exercise, and apparently we still have a thread-safety issues :(

[kumaraditya303]

We have similar older reports of the same tests randomly failing as well like #129441 #128806 and #120158

[encukou]

Another Buildbot failure: https://buildbot.python.org/all/#/builders/1610/builds/1852