Reduce the size of std :: vector to fit its actual data to save memory usage? does vec.swap () not work in MSVC?

Question

In fact, I have millions of vector objects in my program. By default, for each vector, the system assigns more space than it actually needs, since these vectors are read-only after loading is complete.

So, I want to reduce my ability to save memory. a typical way is to use the vector.swap () method as described in this question :

std::vector<T> tmp(v); // copy elements into a temporary vector v.swap(tmp); // swap internal vector data 

I tried this code, but found that the .swap() operation does not reduce the actual memory cost. (I looked at the size of the personal working set in the task manager to get the process memory usage)

Why is this? UPDATE " I use VS 2008, and I added a few lines to display the current capacity at runtime. As can be seen from the code below and its output, vec.capacity was reduced to 10 after swapping, but this is not reflected in the task manager.

Or maybe, as @Adam Rosenfield said, this change did not return the freed up OS space? Or should I not use the task manager to find its memory? I hope this swap () operation can have the same effect as delete pointer_to_a_big_vec , which will free memory directly.

My test sample is here:

 void vecMemoryTest() { vector<int> vec; //make it big! for(int i = 0; i <= 100000; i++){vec.push_back(i);} for(int i = 0; i <= 100000; i++){vec.push_back(i);} cout << "Before .resize() : " << vec.capacity() << endl; //OK now I only need the first 10 elements vec.resize(10); cout << "After .resize() : " << vec.capacity() << endl; //So try to free other spaces vector<int> tmp(vec.begin(), vec.end()); vec.swap(tmp); // now should free wasted capacity // But in fact it doesn't cout << "After .swap() : " << vec.capacity() << endl; } 

and output:

Before .resize (): 207382 After .resize (): 207382 After .swap (): 10

Answer 1

Check the capacity() this vector. You will most likely find that the vector has actually reduced memory usage. What you see is perhaps an implementation of malloc() that does not free memory back to the OS.

Answer 2

The working set counter only considers the accepted virtual address space. It does not take into account how individual users of this address space use it.

A slightly more informative test would be to run vecMemoryTest() several times and keep track of the working set. If after the first start it does not grow, you know that you are freeing memory correctly. If that happens, it could mean anything.

Even better, check the vecMemoryTest() change to highlight a huge vector the same size again and then destroy it. If the working set remains (basically) the same, you know that you have correctly compressed the original vector .

Answer 3

I did not consider the implementation, but it is possible that the copy constructor for vector also copies capacity.

Use the iterator form to create a new temporary:

 vector<int> tmp(vec.begin(), vec.end()); vec.swap(tmp); 

Answer 4

C ++ 11 provides std::vector::shrink_to_fit , which you can try. This function requests that the capacity of the vector be reduced to fit the size of the data it stores. However , the implementation is not tied to this request, so you will need to check how it behaves.

If you want to avoid "block" redundancy, you will need to study the specialized implementation of the standard library; for example, those used to develop games can provide the necessary functionality.

It is important to remember that when you request memory with new , the operating system often gives you more memory than you requested. This may affect what you see.

Answer 5

Are you working with Visual Studio 2010? If so, it supports the new C ++ 0x method for the vector, shrink_to_fit . See the MSDN link .