Celery Clips Log Messages

Question

My celery log sequentially truncates (not so) long error messages, for example:

[2012-04-08 04:53:10,084: INFO/MainProcess] Task mainapp.tasks.async_submitter[2df2fe93-156b-4944-9ecf-c55ba53e8aaa] succeeded in 0.190640926361s: 'An error occurred during the submission of... 

Needless to say, this eliminates half of the registration goal. How can i stop this?

I am using celery with django-celery (via django-supervisor with supervisor) on linux.

Answer 1

Unfortunately, celery will clip messages by default yes.

In versions 3.1.7 to 3.1.9 , you can probably set this limit by fixing the global module:

 import celery.worker.job celery.worker.job.RESULT_MAXLEN = 1048576 # 1 Mib 

Answer 2

The task returned by design should be used for further processing in your code, and that is why the developer took the right to cut the log, also because the return can be a pretty big result of the development and can make unreadable log information output. Of course, I could be wrong, but since I work with celery, I always considered this logic to be correct and never bothered me. In your case, I believe that it makes sense to register a message immediately before returning using logger.info (this will not be truncated), and if the message you are returning is not really needed for further processing, but only for logging purposes no need to return any value.