Threading = True with Flask

Question

I use a jar, and some of my route handlers start computing, which can take several minutes. Using the flag development server, I can use app.run (threaded = True), and my server will continue to respond to other requests until it performs these multi-minute calculations.

Now I started using Flask-SocketIO, and I'm not sure how to do it. I understand that I can explicitly create a separate thread in python every time I start one of these calculations. Is this the only way to do this? Or is there something equivalent to threaded = True for a flask socket. (Or rather, I'm just completely embarrassed.)

Thanks for any help.

Answer 1

The idea behind thread mode in Flask / Werkzeug is for the development server to be able to handle multiple requests at once. In default mode, the server can process one request at a time, if the client sends the request when the server is already processing the previous request, then the second request should wait for the completion of this first request. In streaming mode, Werkzeug creates a stream for each incoming request, so that multiple requests are processed simultaneously. Obviously, you use streaming mode to have requests that take a lot of time, and the server responds to other requests.

Note that this approach is difficult to scale correctly when you exit the development web server and the production web server. For a production server, you must select a fixed number of workers, and this gives you the maximum number of concurrent requests that you can have.

An alternative approach is to use a coroutine-based server such as gevent, which is fully supported by Flask. For gevent, there is one workflow, but it has several light (or "green") threads that together allow each other to start. The key to ensuring that everything works on this model is to ensure that these green threads do not abuse the processor time they receive, since only one can work at a time. When this is done correctly, the server can scale much better than with the multi-user approach described above, and you can easily have hundreds / thousands of clients handled this way.

So now you want to use Flask-SocketIO, and this extension requires the use of gevent. In case the reason for this requirement is not clear, unlike HTTP requests, SocketIO uses the WebSocket protocol, which requires long-term connections. Using gevent and green threads allows you to have a potentially large number of constantly connected clients, which could not be done with several employees.

The problem is your long computation, which is not server friendly like gevent. To make it work, you need to ensure the frequency of use of your calculation function, so that other threads can run and not starve. For example, if your calculation function has a loop inside, you can do something like this:

def my_long_calculation(): while some_condition: # do some work here # let other threads run gevent.sleep() 

The sleep() function will basically stop your thread and switch to any other threads that require a processor. In the end, control will be returned to your function, and at that moment it will proceed to the next iteration. You need to make sure that the sleep calls are not too spaced (as this will make the rest of the application irrelevant) or not too close (as this may slow down your calculation).

So, in order to answer your question, while you are doing your long calculation correctly, you do not need to do anything special to handle simultaneous requests, as this is the usual mode of gevent.

If for some reason the profitability approach is not possible, you may need to think about offloading tasks related to processor intensity with another process. Perhaps use Celery to do this as a quest line.

Sorry for the long answer. Hope this helps!