Short version
Memory leak and R14 errors on Heroku with Rails + Unicorn in the absence of requests. No problem locally, or with Webrick, or when the server is under load.
Long version
I have a Rails 4 application running on Heroku. When there is no load on the server, the memory consumption reported by Heroku increases by about 0.5 MB per minute. After a few hours, R14 errors begin to appear in the logs, and after a while the process does not respond, and Heroku kills him. I also see an increase in memory usage in New Relic.
02:03:00 heroku[web.1]: source=web.1 sample#load_avg_1m=0.00 sample#load_avg_5m=0.00 sample#load_avg_15m=0.00 02:03:00 heroku[web.1]: source=web.1 sample#memory_total=533.19MB sample#memory_rss=511.88MB sample#memory_cache=0.00MB sample#memory_swap=21.30MB sample#memory_pgpgin=1034783pages sample#memory_pgpgout=903740pages 02:03:00 heroku[web.1]: Process running mem=533M(104.1%)
Environment: Ruby 2.1.4 + Rails 4.1.7 + Unicorn + Heroku with 1 web dynamo
What I tried so far
The default number of Unicorn workers decreased from 3 to 2. It didn’t help, it just made the memory leak a little slower.
Disabled background workers (removed from Procfile). There is no effect. Nothing has changed.
Mounted Killer Unicorn Workers. This did not help, because no requests were made, so the worker killer never kicks (is there a worker worker killer?).
Used by ApacheBench for stress testing the server. Memory usage was constant, around 190 MB / process according to New Relic. There was no increase. It seems that a memory leak only occurs when there is no server load.
Unicorn removed and used by default Webrick. Memory usage has become permanent, around 208 MB / instance according to New Relic!
I tried to reproduce this problem on the local computer by starting the server in production using RAILS_ENV=production foreman start . In this case, there was no memory leak.
Based on point 5, I think the memory leak is related to Unicorn, but somehow only happens on Heroku. How to determine where the problem is?
Thanks! Any help is appreciated.
source share