Why does the compiler delay std :: list deallocation?

Question

I have the following code to check for free memory using the std :: list container:

#include <iostream> #include <list> #include <string> #include <boost/bind.hpp> /* count of element to put into container */ static const unsigned long SIZE = 50000000; /* element use for test */ class Element { public: Element() : mId(0) {} Element( long id ) : mId(id) {} virtual ~Element() { } inline long getId() const { return this->mId; } inline bool operator<( const Element & rightOperand ) const { return this->mId < rightOperand.mId; } inline bool isEven() const { return 0 == ( this->mId & 1 ); } private: long mId; }; typedef std::list< Element > Elements; int main( int argc, char * argv[] ) { std::string dummy; { Elements elements; std::cout << "Inserting "<< SIZE << " elements in container" << std::endl; std::cout << "Please wait..." << std::endl; /* inserting elements */ for( long i=0; i<SIZE; ++i ) { elements.push_back( i ); } std::cout << "Size is " << elements.size() << std::endl; std::getline( std::cin, dummy); // waiting user press enter /* remove even elements */ elements.remove_if( boost::bind( & Element::isEven, _1 ) ); std::cout << "Size is " << elements.size() << std::endl; std::getline( std::cin, dummy); } std::getline( std::cin, dummy); return 0; } 

Running this code gives me the following memory profile:

It seems that gcc postpones the release in my test program too, in the end it has no choice and freeing memory before returning to the command line.

Why is liberation happening so late?

I tried with a vector to test another container, and truncated dependency tricks work and free up freed memory when I expect it.

gcc 4.5.0, linux 2.6.34

Answer 1

Most operating systems (including Linux) allow processors to allocate fairly large chunks of memory, rather than very small ones; even if possible, it is likely more expensive to make many small distributions than several large ones. Typically, a C ++ library will receive large chunks from the operating system and use its own heap manager to allocate small chunks of them to the program. Large chunks usually do not return to the operating system once they have been partitioned this way; they will remain dedicated to the process and will be reused for future appropriations.

list allocates memory in small chunks (one per node), and therefore usually the allocated memory will not be released until the program exits. vector can get its memory as one big allocation directly from the operating system, in which case it will be freed when it is freed.

Answer 2

What exactly does your chart show? The std::list destructor frees all memory so that it can be reused elsewhere in the program, but freeing does not necessarily return memory to systems where it can be used by other processes. Historically, the least, on Unix, when memory was allocated for a process, this remains with this process until the process is complete. Newer algorithms can actually return memory to the OS, but even things like fragmentation can prevent it from doing this: if you allocate, then release a really big block, it can be returned, but if you allocate a lot of small blocks (this is, what std::list does), the runtime actually allocates large blocks from the OS that it sends; such large blocks cannot be returned until all small blocks in them have been freed and are likely to be returned even then.

Answer 3

It depends on how you measure memory usage. If it measures the memory of a process in use, this is what you can really expect.

It is often enough for a program to request memory and assign it from a control environment (such as an operating system), but when the memory is freed, it is not necessarily deleted from the process. It can be returned to the free pool in the process.

This was how distribution was used to work in the old days. A call to brk or sbrk would increase the heap size, providing more memory for the process. This memory will be added to the arena from which malloc calls were satisfied.

But free will return memory to the arena, and not back to the operating system.

I suppose something similar happens in this case.

Answer 4

Your memory profile is actually a need for the process address space (the sum of mmap -ed pages, for example, given by /proc/self/statm or /proc/self/maps from the point of view of the process itself).

But when the C or C ++ memory deallocation function (previously allocated with malloc or new , which use mmap to get memory from the Linux kernel) with free or delete , it is not (using munmap because it would be too slow or impractical [fragmentation problems] - but just saved as reusable for the future malloc or new .

Thus, the release occurs at the request free , but the memory is not returned to the system, but is saved for later reuse.

If you really want to get the memory back, write your own allocator (above mmap and munmap ), but this is usually not worth the effort.

Perhaps using a Boehm GC might help (very useful so as not to worry about free -ing or delete -ing) if you explicitly call GC_gcollect() (but I'm not sure about that), but you really shouldn't worry about that.

And your question is not technically related to gcc (this would be the same with another C ++ compiler). It is associated with malloc and new (i.e. with standard C and C ++ libraries) under Linux.