Debug PyThread_acquire_lock deadlock

I have a multi-threaded application running in a production environment that freezes at random with the state of FUTEX_WAIT_PRIVATEall threads, and gdb shows that all threads are trying to make a lock call PyThread_acquire_lock. This is a really massive application with tens of thousands of lines of code, and I cannot figure out which line this error is. Is there any way to debug this problem? I can fix the threading.Lock call and write to the file all the locks were received / released in the application, and then read this file in case of an error again, but I think there are other python functions that call PyThread_acquire_lock. So how can I debug the problem? Maybe it's possible to “subscribe” to this call to the C function from Python and record all these calls?

+4
source share
1 answer

You are one step away from the answer: attach gdbto a confusing process and use the Python gdb extensions to verify that the lines are at a dead end.

For gdb --version> = 7:

sudo apt install python2.7-dbg python3-dbg
sudo gdb /usr/bin/python[3] <pid_of_deadlocked_process>
(gdb) thread apply all py-list
(gdb) thread 2
(gdb) py-up
(gdb) py-print <lock_object>

Links: https://docs.python.org/devguide/gdb.html , https://wiki.python.org/moin/DebuggingWithGdb

+4
source

Source: https://habr.com/ru/post/1669425/


All Articles