App Engine instance frequency is constantly growing

Question

I expect the memory usage of my application kernel instances (Python) will be relatively flat after the first launch period. Each request to my application is short-lived, and it seems that memory access should be released soon after that to use one request.

However, in practice this is not so. The following is a snapshot of the instance memory usage provided by the console. My application has relatively low traffic, so I usually only have one instance running. During the two-day period of the schedule, the graph of memory usage is constantly increasing. (Two blips where two instances were briefly launched.)

I regularly get memory errors, so I would like to prevent this constant increase in memory usage.

During snapshot:

• Memcache uses less than 1 MB
• Target lines are empty
• Low traffic (0.2 counts per second)

I would expect instance memory usage to fall in these circumstances, but this does not happen.

Since I use Python with its automatic garbage collection, I do not see how I could do this.

Is this the expected behavior of the application engine and is there anything I can do to fix it?

Answer 1

I found another answer that explains part of what is happening here. I will give a summary based on this answer:

• When using NDB, objects are stored in a cache context, and a cache context is part of your memory usage.

• From the documentation, you can expect that the memory will be released at the end of the HTTP request.

• In practice, memory is not freed after the HTTP request has completed. Apparently, context caching is reused, and the cache is cleared before its next use, which can take a lot of time.

In my situation, I add _use_cache=False to most entities to prevent them from being stored in the context cache. Due to the way my application works, I do not need contextual caches for these objects, and this reduces memory usage.

The above is only a partial solution!

Even when caching is disabled for most of my entities, my memory usage is still growing! Below is a snapshot of a 2.5-day period when memory is constantly increasing from 36 MB to 81 MB. This occurs on weekends on July 4th with low traffic.