Django 1.6 and Celery 3.0 Memory Leaks

Question

After upgrading Django to 1.6, my celery worker ate RAM. It seems that the memory allocated for workers is not freed and grows after each task.

Related Settings:

# DB: DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql_psycopg2', 'NAME': 'somedb', 'USER': '', 'PASSWORD': '', 'HOST': 'localhost', 'PORT': '', } } # CELERY SETTINGS: CELERY_RESULT_BACKEND = 'redis://' BROKER_URL = 'redis://' 

Similar versions of the package:

 Django==1.6 celery==3.0.24 django-celery==3.0.23 billiard==2.7.3.34 kombu==2.5.16 redis==2.7.6 

Serves both in my local env (with DEBUG=False ), which starts the worker manually, and in an intermediate environment where celery works with Upstart.

---

Update:

• Tried setting autocommit=False without success.
• Maybe this is not related to updating the version of Django, but with some settings or third-party packages that I had to update to make the transition to 1.6.

Answer 1

It turns out that the memory leak is not directly caused by the Django or Celery update.

After many searches, I found it was surprising that a celery worker memory leak was due to the fact that I updated the django-debug-toolbar from 0.9.4 to 0.11.0 (which is necessary for Django 1.6).

I still don’t know what exactly caused this problem, or why this happens only in celery workflows, and not in application servers (Gunicorn).

Removing django-debug-toolbar from installed applications and middleware solves the problem. At least temporarily.

Answer 2

It seems that the change from django-debug-toolbar 0.9.4 to 0.11.0 really caused a memory leak caused by the LoggingPanel, which stores an infinite number of messages. If you had a process using the logging subsystem, you are probably facing this problem. You can also remove the LoggingPanel from the default panel list to work around this issue.

Apparently, in 0.9.4 panels were lazily initialized only when middleware was accessed. This changed to 0.11.0 : the panels are initialized immediately after import, and the LoggingPanel module intercepts all the logs, regardless of whether DEBUG installed.

I sent a fix for this error.