Django TimeLimitExceeded Error

Question

Today I got an email with Celery email, can someone explain this and maybe how can I fix the timeout problem? It would be very helpful, thanks.

PS My message seems to be sent despite this error, right?

Error:

Task Request to Process with id 65123935-b190-4718-9ed0-fb863359f27f raised exception: 'TimeLimitExceeded(300.0,)' Task was called with args: (<Batch: Batch object>,) kwargs: {}. The contents of the full traceback was: Traceback (most recent call last): File "/usr/local/lib/python2.7/dist-packages/billiard/pool.py", line 496, in on_hard_timeout raise TimeLimitExceeded(job._timeout) TimeLimitExceeded: TimeLimitExceeded(300.0,) -- Just to let you know, py-celery at w1.ip-10-32-53-113. 

Task:

 class ProcessRequests(Task): name = "Request to Process" max_retries = 1 default_retry_delay = 3 def run(self, batch): # Only run this task on non-scheduled tasks if batch.status != "Scheduled": q = Contact.objects.filter(contact_owner=batch.user) if batch.group == None: q = q.filter(id=batch.contact_id) else: q = q.filter(group=batch.group) for e in q: msg = Message.objects.create( recipient_number=e.mobile, content=batch.content, sender=e.contact_owner, billee=batch.user, sender_name=batch.sender_name ) gateway = Gateway.objects.get(pk=2) msg.send(gateway) 

Answer 1

You get a TimeLimitExceeded exception because your task took more than 300 seconds. 300 seconds is the default time that a task can complete.

As Avichal Badaya replied, you can customize celery to ensure longer tasks are completed.

You mentioned that your message is still sent, but this can happen because your tasks are written for the party. Thus, either you try to send several messages, or you try to send only one message, and some how to confuse the packet.

Can you show us what you call this task?

Answer 2

Can you try changing celeryd settings. the file should basically be /etc/init.d/celeryd.

 CELERYD_OPTS="--time-limit==3600 -E --loglevel=DEBUG" 

Answer 3

First you need to know what you do when you receive a message about your function / method of the task and its time complexity. If your task takes longer than the default 300 seconds to complete, then the main celery process will first send a SoftTimeLimitExceed exception, which you can catch in your task if we wrap it in a try / except block. This exception is valid for the cleaning process, if any. Immediately after this exception, the celery will send you another TimeLimitExceed exception and it will kill your workflow.

If you turned on the late ACK setting in celery settings, then the same message will be delivered to another employee for processing (this is dangerous, as this will also lead to a failure).

So, if you get this exception, it means that it takes more than 300 seconds to complete your task, so set the maximum time for the worker to adjust the celery according to your requirements.

Hello,

Haridas N.

Answer 4

You are sending messages to mobile phones with msg.send(gateway) , right?

Inside this method, you are connecting to a remote web service that actually sends a message, right?

Then your message could be sent, but your connection to the remote side was not closed, despite the fact that the webservice should have closed it, after sending the message and responding to your request.

The last thing you did not do before creating the socket used to connect to the webservice:

 import socket socket.setdefaulttimeout(seconds) # seconds argument is of type float 

Thus, your task hangs on the socket for an infinite time, waiting for the web service to send a response and / or close the connection. In fact, it would wait forever, if not interrupted by celery timeout .

The default timeout value for the None socket is as specified in the documentation http://docs.python.org/2/library/socket.html#socket.setdefaulttimeout

Hope this will be helpful.