Is there such a thing as streaming processing in python?

Question

I wrote a beautiful multi-threaded script, and when I ran it, it did worse with 25 threads than with a direct call to the thread handler.

Then I discovered a global interpreter lock. I want to ask, before I drop python for this script and rewrite the thing in something else, is there a way to make the actual multithreading work in python?

Answer 1

Another approach is to discard threads and instead use the Multiprocessing module (Python 2.6+), which bypasses the GIL, and has an API that is at least similar to the API in the stream module.

Answer 2

Now this is an interesting question - I don't think he escaped the GIL in direct CPython.

Stackless Python should improve concurrency performance compared to CPython using microthreads , but I don't think it eludes GIL.

Also, according to the GIL page on python.org , Jython and IronPython do not have a GIL.

Answer 3

, .

CPU ( IO-) , , , GIL , concurrency. , concurrency, , , Python.

, , .

Answer 4

CPython ( Python C) , . ( , , , , !).

You might want to be interested in IronPython (.NET) or JPython (Java / JWM). I have not used them, but I believe that at least one of them supports threads that are native to this runtime.

Answer 5

You can try the multiprocessing module , if applicable to your problem.