What is the cost of issuing a GIL?

Question

Suppose I have a C extension function that does something that is completely independent of the Python interpreter. Is there any reason not to release the GIL?

For example, is there a reason not to write such code (besides issues such as readability and avoiding micro-optimization - which is important, but not very important for my question)?

Py_BEGIN_ALLOW_THREADS a = 1 + 1; Py_END_ALLOW_THREADS 

Clearly, this is trivial code where performance probably won't matter much. But is there any reason for the performance not to release the GIL here? Or should the GIL be released only for a more intensive processor?

Answer 1

GIL is a regular mutex. The cost of locking or unlocking an unidentified mutex is extremely low, not much more than the cost of changing a global variable. However, if you lock and unlock the disputed mutex very often, the cost of the mutex can become significant.

So this is usually not a good idea:

 Py_BEGIN_ALLOW_THREADS a = 1 + 1; Py_END_ALLOW_THREADS 

What happens here, you unlock the mutex, which you are trying to lock again immediately afterwards. If this is a gap between two large pieces of code, then it gives the other thread the ability to run. But if you have no problems with detailing the flows, just hold the lock.

So this is a good idea in this context:

 very_long_computation_requires_gil(); Py_BEGIN_ALLOW_THREADS; a = a + i; Py_END_ALLOW_THREADS; very_long_computation_also_requires_gil(); 

In fact, it is impossible to make an educated guess without knowing the context, and it is often still difficult without testing.

Answer 2

If you have a C extension function that does something completely independent of the Python interpreter, then releasing GIL is usually a good idea. The only downside is waiting for the GIL to return. In Python 3.2, you need to wait at least 1/20 seconds.

Answer 3

Is there any reason not to release the GIL?

If the C extension causes non-intercept code, then you may have problems if multiple Python threads access the extension at the same time. Thus, you can not release GIL in such extensions to protect against this event (of course, you could create your own mutex at the Python or C level to achieve this without affecting other threads).

Or should the GIL be released only for a more intensive processor?

Another main reason for releasing the GIL is to call a C extension that blocks (for example, a lock read on a socket) to enable other threads. This is exactly what happens when the Python interpreter itself performs a blocking operation on a thread.

Answer 4

Experts are still setting up and testing GIL.

These are new ideas about an old problem: inside-look-at-gil-removal-patch

You can also try Stackless Python (without GIL) or PyPy (Python with Just-In-Time compiler).