Celery Memory Management

Question

I have 3 celery nodes that work from 6 to 8 workers, and my tasks tend to use a lot of memory. It seems that when tasks end, they do not free up memory. Is there a way to either make the workflow die and restart after completion or improve garbage collection from celery?

Answer 1

For these workers, you can set max-tasks-per-child-setting to one.