Skip to content

gh-128485: ensure that dlmalloc initializes itself at import time in ctypes #131633

Merged
kumaraditya303 merged 1 commit intopython:mainfrom
kumaraditya303:ctypes-alloc
Mar 24, 2025
Merged

gh-128485: ensure that dlmalloc initializes itself at import time in ctypes #131633
kumaraditya303 merged 1 commit intopython:mainfrom
kumaraditya303:ctypes-alloc

Conversation

@kumaraditya303
Copy link
Copy Markdown
Contributor

@kumaraditya303 kumaraditya303 commented Mar 23, 2025 •
edited by bedevere-app Bot
Loading

This fixes the following data race in initialization of the dlmalloc allocator by allocating a small memory block and immediately freeing it at module initialization to force initialization of dlmalloc allocator.

Details 
WARNING: ThreadSanitizer: data race (pid=23354)
  Atomic read of size 1 at 0x7f3a0080d588 by thread T3:
    #0 pthread_mutex_lock <null> (python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 dlmalloc /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:4157:8 (libffi.so.8+0x592b) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 ffi_closure_alloc /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/closures.c:998:9 (libffi.so.8+0x592b)
    #3 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #4 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #8 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #10 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #11 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #13 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #14 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #16 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #18 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Previous write of size 1 at 0x7f3a0080d588 by thread T20 (mutexes: write M0):
    #0 pthread_mutex_init <null> (python+0xe1a03) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 init_mparams /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:2557:7 (libffi.so.8+0x33e8) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #3 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #7 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #10 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #13 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #14 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #15 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #16 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Location is global '??' at 0x7f3a00802000 (libffi.so.8+0xb588)

  Mutex M0 (0x7f3a0080d1e0) created at:
    #0 pthread_mutex_lock <null> (python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 init_mparams /build/libffi-oEu4Vn/libffi-3.4.6/build/../src/dlmalloc.c:2553:5 (libffi.so.8+0x3370) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
    #2 PyCFuncPtr_new /home/realkumaraditya/cpython/./Modules/_ctypes/_ctypes.c:4075:13 (_ctypes.cpython-314t-x86_64-linux-gnu.so+0x19a54) (BuildId: c8bafb886e22ac3df767d01ecd0cd7ce8c1f0769)
    #3 type_call /home/realkumaraditya/cpython/Objects/typeobject.c:2245:11 (python+0x2fe152) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #5 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #7 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:1374:35 (python+0x3faf44) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f64d0) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f64d0)
    #10 _PyFunction_Vectorcall /home/realkumaraditya/cpython/Objects/call.c (python+0x1f0cef) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:169:11 (python+0x1f54df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 method_vectorcall /home/realkumaraditya/cpython/Objects/classobject.c:72:20 (python+0x1f54df)
    #13 _PyVectorcall_Call /home/realkumaraditya/cpython/Objects/call.c:273:16 (python+0x1f097f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #14 _PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:348:16 (python+0x1f097f)
    #15 PyObject_Call /home/realkumaraditya/cpython/Objects/call.c:373:12 (python+0x1f09e5) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #16 thread_run /home/realkumaraditya/cpython/./Modules/_threadmodule.c:353:21 (python+0x5996a2) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 pythread_wrapper /home/realkumaraditya/cpython/Python/thread_pthread.h:242:5 (python+0x4f3f87) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Thread T3 'ThreadPoolExecu' (tid=23358, running) created by main thread at:
    #0 pthread_create <null> (python+0xe01df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f2e18) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:313:9 (python+0x4f2c3a) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:438:9 (python+0x599237) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1823:9 (python+0x599237)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1946:14 (python+0x598001) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:551:18 (python+0x29a437) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3011:35 (python+0x4011a6) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f600f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f600f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:836:21 (python+0x3f600f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d13b1) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d13b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4cc940) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4cc940)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4cc098) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:396:15 (python+0x50fa9f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:415:15 (python+0x50fa9f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:681:21 (python+0x50eecb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:762:5 (python+0x50eecb)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:792:12 (python+0x50f338) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:816:12 (python+0x50f3bb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15e7eb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

  Thread T20 'ThreadPoolExecu' (tid=23375, running) created by main thread at:
    #0 pthread_create <null> (python+0xe01df) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #1 do_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:289:14 (python+0x4f2e18) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #2 PyThread_start_joinable_thread /home/realkumaraditya/cpython/Python/thread_pthread.h:313:9 (python+0x4f2c3a) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #3 ThreadHandle_start /home/realkumaraditya/cpython/./Modules/_threadmodule.c:438:9 (python+0x599237) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #4 do_start_new_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1823:9 (python+0x599237)
    #5 thread_PyThread_start_joinable_thread /home/realkumaraditya/cpython/./Modules/_threadmodule.c:1946:14 (python+0x598001) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #6 cfunction_call /home/realkumaraditya/cpython/Objects/methodobject.c:551:18 (python+0x29a437) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #7 _PyObject_MakeTpCall /home/realkumaraditya/cpython/Objects/call.c:242:18 (python+0x1efb28) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #8 _PyObject_VectorcallTstate /home/realkumaraditya/cpython/./Include/internal/pycore_call.h:167:16 (python+0x1f0748) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #9 PyObject_Vectorcall /home/realkumaraditya/cpython/Objects/call.c:327:12 (python+0x1f0748)
    #10 _PyEval_EvalFrameDefault /home/realkumaraditya/cpython/Python/generated_cases.c.h:3011:35 (python+0x4011a6) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #11 _PyEval_EvalFrame /home/realkumaraditya/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x3f600f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #12 _PyEval_Vector /home/realkumaraditya/cpython/Python/ceval.c:1908:12 (python+0x3f600f)
    #13 PyEval_EvalCode /home/realkumaraditya/cpython/Python/ceval.c:836:21 (python+0x3f600f)
    #14 run_eval_code_obj /home/realkumaraditya/cpython/Python/pythonrun.c:1365:12 (python+0x4d13b1) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #15 run_mod /home/realkumaraditya/cpython/Python/pythonrun.c:1436:19 (python+0x4d13b1)
    #16 pyrun_file /home/realkumaraditya/cpython/Python/pythonrun.c:1293:15 (python+0x4cc940) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #17 _PyRun_SimpleFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:521:13 (python+0x4cc940)
    #18 _PyRun_AnyFileObject /home/realkumaraditya/cpython/Python/pythonrun.c:81:15 (python+0x4cc098) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #19 pymain_run_file_obj /home/realkumaraditya/cpython/Modules/main.c:396:15 (python+0x50fa9f) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #20 pymain_run_file /home/realkumaraditya/cpython/Modules/main.c:415:15 (python+0x50fa9f)
    #21 pymain_run_python /home/realkumaraditya/cpython/Modules/main.c:681:21 (python+0x50eecb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #22 Py_RunMain /home/realkumaraditya/cpython/Modules/main.c:762:5 (python+0x50eecb)
    #23 pymain_main /home/realkumaraditya/cpython/Modules/main.c:792:12 (python+0x50f338) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #24 Py_BytesMain /home/realkumaraditya/cpython/Modules/main.c:816:12 (python+0x50f3bb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)
    #25 main /home/realkumaraditya/cpython/./Programs/python.c:15:12 (python+0x15e7eb) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3)

SUMMARY: ThreadSanitizer: data race (/home/realkumaraditya/cpython/python+0xe1d0e) (BuildId: a3c850bcdf819f5092d2cbb0f20c6441a59e91e3) in pthread_mutex_lock

I am skipping news items for each small fix to avoid many news items, I'll that in the end once majority of fixes are in place.

@bedevere-app bedevere-app Bot mentioned this pull request Mar 23, 2025
Closed
@kumaraditya303 kumaraditya303 added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 23, 2025
@bedevere-bot
Copy link
Copy Markdown

bedevere-bot commented Mar 23, 2025

🤖 New build scheduled with the buildbot fleet by @kumaraditya303 for commit fd00239 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131633%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Mar 23, 2025
@kumaraditya303 kumaraditya303 merged commit 04d4aac into python:main Mar 24, 2025
125 of 126 checks passed
@kumaraditya303 kumaraditya303 deleted the ctypes-alloc branch March 24, 2025 12:44
@kumaraditya303 kumaraditya303 mentioned this pull request Mar 24, 2025
Closed
diegorusso pushed a commit to diegorusso/cpython that referenced this pull request Apr 1, 2025
@kumaraditya303 @diegorusso
pythongh-128485: ensure that dlmalloc initializes itself at import ti…
391f7c1 
…me in ctypes (python#131633)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

skip news topic-ctypes topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kumaraditya303 @bedevere-bot