Asynchronous Celery Replacement

Question

We use celery for background tasks in our Django project. Unfortunately, we have many blocking sockets in tasks that can be installed over time. Thus, celery becomes fully loaded and not responding.

Gevent can help me with sockets, but Celery has only experimental support for gevent (and, as I found in practice, this does not work well). So I decided to switch to another system of task queues.

I can choose between two different ways:

• Write my own task system. This is the least preferred choice because it takes a lot of time.
• Find a good and proven replacement for celery that will work after the monkey patch.

Is there any analogue of celery that guarantees me the fulfillment of my tasks even after a sudden release?

Answer 1

Zeromq may be suitable for your use.

See https://serverfault.com/questions/80679/how-to-pick-between-rabbitmq-and-zeromq-or-something-else

However, you will need to write your own message library to save messages.

Answer 2

Have you tried using Celery + eventlet? He works well in our project.