I have 4 lambda functions that will be called simultaneously (via SNS), the frequency of the SNS event is 5 minutes. Each function processes a large amount of data and images (~ 300 MB), so I store them in the /tmp folder (500 MB limit).
At the beginning of the function, I wrote code in the /tmp cleanup folder to make sure it didn’t go out of memory (since I knew that AWS lambda sometimes uses the previous container to improve performance).
I check it manually (I create a message and publish SNS up to 4 lambda functions), it worked fine.
But when it starts automatically (called every 5 minutes), the result is not my expectation. The first execution is fine, but the next time after 1 out of 4 or even 4 lambda functions, they throw an error related to "from memory": "There is no space on the device", it is impossible to load lib, ...
The previous one, I use nodejs (4.3), it worked fine in both cases.
But for some reason I have to switch to python, the main stream and mounting of the created data are the same. But this failed with an automatic start.
I think the problem came from the cache of the previous container (reusable container), I checked /tmp after cleaning ( ls -alh /tmp ) there are no files there, but when checking the storage ( df /tmp ) it shows that it is used is 77% .
Any suggestion to make a clean /tmp folder or work with a solution is very valuable. Thanks!
Edited: The code I use to clear the /tmp folder is:
from subprocess import call ... call('rm -rf /tmp/*', shell=True)