Are the assigned tasks confirmed in celery?

I have the following setup:

  • RabbitMQ broker + Django
  • Celery with CELERYD_PREFETCH_MULTIPLIER = 32 (I have a lot of small tasks so pre-fetching them makes a lot of sense in terms of performance)
  • CELERY_ACKS_LATE = False (tasks are not idempotent)

I use celery in a docker container, so when I restore dockers, celery doesn't gracefully close. This is normal if the tasks are not confirmed, as the broker will send them back as soon as the workers reappear in the new docker container, but otherwise they will be lost.

In queries with predefined settings for the color control panel, the flower has received status.

I carefully read the official documentation and related question , and I intuitively feel that the predefined tasks in my setup are confirmed. This is true?

+5
source share
1 answer

With CELERY_ACKS_LATE=False task will be confirmed as soon as the worker starts its execution: before execution .

So, if the worker begins to complete the task, and you kill her, the task will be lost. Other predefined tasks will not be lost because they are still not confirmed.

+2
source

Source: https://habr.com/ru/post/1264150/


All Articles