How can I determine if I work in a celery workplace?

Question

Is there a way to determine, programmatically, that the current module being imported / executed is executed in the context of a celery worker?

We decided to set the environment variable before starting the Celery worker and check this environment variable in the code, but I wonder if there is a better way?

Answer 1

Just,

import sys IN_CELERY_WORKER_PROCESS = sys.argv and sys.argv[0].endswith('celery')\ and 'worker' in sys.argv if IN_CELERY_WORKER_PROCESS: print ('Im in Celery worker') 

http://percentl.com/blog/django-how-can-i-detect-whether-im-running-celery-worker/

Answer 2

Depending on your usage scenario, you may discover it by setting whether the request identifier is set:

 @app.task(bind=True) def foo(self): print self.request.id 

If you call the above as foo.delay() , then the task will be sent to the employee, and self.request.id will be set to a unique number. If you call it as foo() , then it will be executed in your current process, and self.request.id will be None .

Answer 3

You can use the current_worker_task property from the Celery application instance class. The docs are here.

Having defined the following task:

  # whatever_app/tasks.py celery_app = Celery(app) @celery_app.task def test_task(): if celery_app.current_worker_task: return 'running in a celery worker' return 'just running' 

You can run the following in a python shell:

 In [1]: from whatever_app.tasks import test_task In [2]: test_task() Out[2]: 'just running' In [3]: r = test_task.delay() In [4]: r.result Out[4]: u'running in a celery worker' 

Note. Obviously, for test_task.delay() to succeed, you need to have at least one celery worker who works and is configured to load tasks from whatever_app.tasks .

Answer 4

Starting with celery version 4.2, you can also do this by setting the worker_ready flag of the worker_ready signal

in celery.py :

 from celery.signals import worker_ready app = Celery(...) app.running = False @worker_ready.connect def set_running(*args, **kwargs): app.running = True 

Now you can check in your task using the global instance of the application to see if you are working. This can be very useful to determine which registrar to use.

Answer 5

Adding an environment variable is a good way to check if a module works with celery. In the process of submitting an application, we can set an environment variable to note that it does not work in the context of a celery worker.

But the best way might be to use some celery signals, which can help you find out if the module works in the working or target sender. For example, a worker-process-init signal is sent to each child executing process (in pre-sale mode), and the handler can be used to set some global variable indicating that it is a work process.

Answer 6

It is good practice to start workers with names to simplify management (stop / kill / restart). You can use -n to indicate worker.

 celery worker -l info -A test -n foo 

Now, in the script, you can use app.control.inspect to see if this one works.

 In [22]: import test In [23]: i = test.app.control.inspect(['foo']) In [24]: i.app.control.ping() Out[24]: [{' celery@foo ': {'ok': 'pong'}}] 

You can read about it in celery desktop docs