Celery does not work with RabbitMQ with the option "--autoreload"

Question

• I do not know if there is an error or configuration problem.
• Purpose: debugging / changing Celery tasks without manually restarting the worker every time I make a small change.

So, I configured Celery + RabbitMQ. Everything works.

The --autoreload option has been --autoreload for the celery -A proj worker --loglevel=debug , and registration stops at this:

 [2014-09-11 19:22:00,447: DEBUG/MainProcess] | Worker: Hub.register Autoreloader... 

Without him:

 [2014-09-11 19:37:34,316: DEBUG/MainProcess] | Worker: Hub.register Pool... [2014-09-11 19:37:34,317: DEBUG/MainProcess] basic.qos: prefetch_count->16 [2014-09-11 19:37:36,275: DEBUG/MainProcess] pidbox received method enable_events() [reply_to:None ticket:None] [2014-09-11 19:37:36,275: INFO/MainProcess] Events of group {task} enabled by remote. 

Everything else works great. RabbitMQ receives a message after

 celery call tasks.update 

There are compounds from celery. It just doesn’t mean that celery has to start the mission.

Sounds like a connection problem, but I don't know what it is. If you can help, write.

Answer 1

In celery there is an error on the celery / employee / autoreload.py line 67

This bug fix is ​​fixed: https://github.com/pashinin/celery/commit/92b52db6eeeb75494700ffe807ecd4c1fe6b0643

you can fix the library by changing line 67 of autoreload.py with

 for chunk in iter(lambda: f.read(2 ** 20), ''): 

to

 for chunk in iter(lambda: f.read(2 ** 20), b''): 

After this change, you may encounter another problem: if you already have one task created, it will NOT be recreated, and this task will not be updated after the module is rebooted. Reloaded tasks become active only after they are completed again.

Celery developers don't seem to want to fix this problem early on. Until then, you have two options:

• An updated task must be completed before a new one is received.

• Restart celery worker after task update