The documentation for supporting CPython threads is quite inconsistent and sparse.
In general, it seems that everyone agrees that multi-threaded C applications that embed Python should always acquire a GIL before invoking the Python interpreter. This is usually done using:
PyGILState_STATE s = PyGILState_Ensure(); /* do stuff with Python */ PyGILState_Release(s);
The docs pretty much cite this very simply: https://docs.python.org/2/c-api/init.html#non-python-created-threads
However, in practice, getting a multi-threaded C program that embeds Python in real work is another story. It seems that there are many quirks and surprises, even if you carefully follow the documents.
For example, it seems that behind the scenes Python distinguishes between the "main thread" (which, I think, is the thread that calls Py_Initialize ) and other threads. In particular, any attempt to get GIL and run Python code in the "main" thread sequentially failed, when I try to do this (at least with Python 3.x), the program is interrupted by the message Fatal Python error: drop_gil: GIL is not locked which is stupid because of course the GIL is blocked!
Example:
int main() { Py_Initialize(); PyEval_InitThreads(); PyEval_ReleaseLock(); assert(PyEval_ThreadsInitialized()); PyGILState_STATE s = PyGILState_Ensure(); const char* command = "x = 5\nfor i in range(0,10): print(x*i)"; PyRun_SimpleString(command); PyGILState_Release(s); Py_Finalize(); return 0; }
This simple program breaks with "GIL is not a blocked error," although I explicitly blocked it. However, if I create another thread and try to get the GIL in this thread, everything works.
So, CPython has a (undocumented) concept of a "main thread" that is somehow different from the secondary threads generated by C.
Question : Is this documented anywhere? Has anyone had any experience that would shed some light on the fact that it is the rules that are intended for the acquisition of GIL, and if it is assumed that the "main" topic and child thread have any relation to this?
PS: In addition, I noted that PyEval_ReleaseLock is an outdated API call , but I have not seen an alternative that actually works. If you do not call PyEval_ReleaseLock after calling PyEval_InitThreads , your program will freeze immediately. Nonetheless,