Process memory usage behavior after running a free

Question

I have an application for which I am analyzing memory usage. I download the application with some data, and the application is such that it caches (saves some hash tables and other data structure in the form, lets say some records) information from this data with pumping. For each record stored in memory, the application allocates memory using malloc / calloc. After some time, about 80% of these records time out and the application frees up the memory that it allocated for these records. To save the memory usage check in the application, in the background I ran a script to grab the output from above, as well as “free -m” and plotted to see the memory usage in the system as follows. The graph shows the trend observed in the values ​​printed by "free -m" under the line "- / + buffers / cache" with the used and free columns. I expected the graph to start increasing at the beginning and then decreasing as the application frees up memory. But this is not a real scenario. Inputs at the top are also not reduced. Can someone help me understand the memory dynamics for an application, as shown on Linux.

Answer 1

If you are sure that you do not have a memory leak, this may be due to the fact that your allocator keeps the freed memory. There may be several reasons for this:

• Your allocator may expect that freed memory will be needed again in the future. This is an attempt to reduce the overhead during memory allocation by eliminating the need for a system call to satisfy the next allocation request.

• You may be suffering from memory fragmentation. This means that the freed memory is located on pages that still contain objects that were not freed, so these pages cannot be returned to the system. This means that your program takes up more memory than the memory occupied by used objects.

To make sure you suffer last, not first, try to select an object about half the total size of what was released when 80% was released. If your program grows by this value, it means that free memory is fragmented.

Answer 2

malloc is a library call. If he does not have memory in his own memory pool, he will ask the kernel to associate more memory with the process.

malloc for small buffers will use memory from its own pool, and free will return memory to this pool without releasing it to the system if the pool does not become too large.

malloc for large buffers is implemented in a completely different way, it will call the kernel and therefore will be slower, but when such a buffer is freed, it will be immediately freed.

http://linux.die.net/man/3/malloc

Read the notes there.