When is GetBuffer () on a MemoryStream ever useful?

Question

I knew that GetBuffer() on a MemoryStream in C # /. NET should be used with care, because, as the documents describe here , there may be unused bytes at the end, so you should definitely look at only the first bytes of MemoryStream.Length in the buffer.

But then I came across a case yesterday when the bytes at the beginning of the buffer were junk! Indeed, if you use a tool such as a reflector and look at ToArray() , you can see this:

 public virtual byte[] ToArray() { byte[] dst = new byte[this._length - this._origin]; Buffer.InternalBlockCopy(this._buffer, this._origin, dst, 0, this._length - this._origin); return dst; } 

To do anything with the buffer returned by GetBuffer() , you really need to know _origin. The only problem is that _origin is private and there is no way to get to it ...

So my question is: what is the use of GetBuffer() on MemoryStream() without any GetBuffer() knowledge of how MemoryStream was created (which is what _origin sets)?

(It is this constructor and this constructor alone that sets the origin - if you want the MemoryStream around the byte array to start with a specific index in the byte array:

 public MemoryStream(byte[] buffer, int index, int count, bool writable, bool publiclyVisible) 

)

Answer 1

If you really want to access the internal _origin Value, you can use the call MemoryStream.Seek (0, SeekOrigin.Begin). The return value will be exactly the value of _origin.

Answer 2

The answer is in the MSDN GetBuffer () document, you may have missed it.

When creating a MemoryStream without providing a byte array ( byte[] ):

it creates an expandable capacity initialized to zero.

In other words, a MemoryStream will reference the byte[] with the correct size if a Write call is made to the stream.

Thus, with GetBuffer() you can directly access the base array and read it.

This one can be useful when you are in a situation where you get a stream without knowing its size. If the received stream is usually very large, calling GetBuffer() much faster than calling ToArray() , which copies the data under the hood, see below.

To get only the data in the buffer, use the ToArray method; however, ToArray creates a copy of the data in memory.

I wonder at what point you could call GetBuffer () to get the source data at the beginning, it could be between two Write calls, where the data from the first would be garbage collected, but I would not if that happens.

Answer 3

ToArray () is an alternative to GetBuffer (). However, ToArray () makes a copy of the object in memory. If the bytes are greater than 80,000, the object will be placed in a bunch of large objects (LOH). So far, nothing unusual. However, the GC does not handle LOH and the objects in it very well (memory is not freed, as you expect). Because of this, an OutOfMemoryException may occur. The solution is to either call GC.Collect () to collect these objects, or use GetBuffer () and create several smaller objects (less than 80,000 bytes) - they will not fit the LOH and the memory will be freed, as expected by GC.

There is a third (best) option that should only use threads, for example. read all the bytes from the MemoryStream and write them directly to HttpResponse.OutputStream (again using a byte array and 80,000 bytes as a buffer). However, this is not always possible (as it was in my case).

As a summary, you can say that when a copy of an object in memory is not needed, you will have to avoid ToArray (), and in those cases GetBuffer () may come in handy, but it may not be the best solution.

Answer 4

.NET 4.6 has a new API, bool MemoryStream.TryGetBuffer(out ArraySegment<byte> buffer) , which is similar in spirit to .GetBuffer() . This method will return an ArraySegment , which includes _origin information, if possible.

See this question for details on when .TryGetBuffer() will return true and populate the out parameter with useful information.

Answer 5

This can be useful if you use a low-level API that accepts an ArraySegment , such as Socket.Send . Instead of calling ToArray , which will create another copy of the array, you can create a segment:

 var segment=new ArraySegment<byte>(stream.GetBuffer(), 0, stream.Position); 

and then pass this to the Send method. For big data, this will avoid allocating a new array and copying into it, which can be expensive.

Answer 6

GetBuffer() always assumes that you know the structure of the data supplied to the string (and its use). If you want to get data from a stream, you should always use one of the provided methods (for example, ToArray() ).

It may be something similar, but only the case that I could think of now is a fixed structure or a virtual file system sitting in a stream. For example, at your current position, you read the offset for the file sitting inside the stream. Then you create a new stream object based on this stream buffer, but with a different _origin . This saves you from copying all the data for a new object, which can save you a lot of memory. This will save you from transferring the original buffer as a link with you, because you can always restore it again.

Answer 7

The most important point from the GetBuffer MSDN documentation , except that it does not create a copy of the data, is that it returns an array with unused bytes:

Note that the buffer contains allocated bytes, which may be unused. For example, if the string "test" is written to a MemoryStream, the length of the buffer returned from GetBuffer is 256, not 4, and 252 bytes are not used. To get only the data in the buffer, use the ToArray method; however, ToArray creates a copy of the data in memory.

So, if you really want to avoid creating a copy due to memory limitations, you should be careful not to send the entire array from GetBuffer for wiring or dump it into a file or attachment, as this buffer grows by 2 permissions with every fill and almost always have a lot of unused bytes at the end.