A container for representing several memory blocks as one continuous

Question

Is there any “standard” container (STL, boost) that can represent several memory blocks as one continuous? I need to work with some data with the following conditions:

• The total data size is not known in advance (web response)
• The memory is allocated in chunks (with some external allocation function that I do not control)
• Memory deallocation is not controlled by me, so reallocation is relatively expensive

So, after getting all the data, I have a list of pieces of memory. And I need to apply some STL algorithms (search, copy, etc.) to the data in general. There is a decision to write a container for storing information about these pieces + move an iterator that is able to "jump" from one fragment to another.

But the problem seems pretty general, so I hope there is some known answer that I am missing. Thanks in advance.

Answer 1

You said that memory is provided to you. It seems you do not want to copy it. No problem, the STL philosophy is quite flexible. You really don't need a container; they are just for memory management and that have already been taken care of.

You need an iterator. There is no standard; You will have to write it yourself. There are too many minor changes to provide a standard solution for this. But don't worry, it's pretty easy. You get the necessary typedefs if you inherit from std::iterator<value_type> , so you only need to write operator* (direct) and operator++ / operator-- / operator+ / operator- (which understand the pieces).

Answer 2

I needed something similar, and after I looked around, I wrote my own, based on the operator overload []. For the corresponding Q & As, see the following streams;

Can I use the [] operator in C ++ to create virtual arrays

Good C ++ class class for efficiently managing large datasets in a fast and efficient way?

Answer 3

So, after getting all the data, I have a list of pieces of memory. And I need to apply some STL algorithms (search, copy, etc.) to the data in general. There is a decision to write a container to store information about these pieces + move an iterator that is able to "jump" from one fragment to another.

It sounds like you need an iterator that easily moves all your pieces. std::deque<> provides a similar iterator, since it also allocates memory in chunks.

If you really need it to be in one contiguous block of memory. In this case, all the pieces will need to be copied into one continuous piece of memory.