Can one huge array of continuous bytes be made as an array of two-dimensional bytes?

Question

I have a very large two-dimensional byte array in memory,

byte MyBA = new byte[int.MaxValue][10];

Is there any way (possibly unsafe) that I can fool C # into thinking that this is one huge array of contiguous bytes? I want to do this so that I pass it to MemoryStream, and then a BinaryReader.

MyReader = new BinaryReader(MemoryStream(*MyBA)) //Syntax obviously made-up here

Answer 1

I do not believe that .NET gives this, but it is quite simple to implement your own implementation System.IO.Stream, which easily switches the auxiliary array. Here are the basic (unverified) basics:

public class MultiArrayMemoryStream: System.IO.Stream
{
    byte[][] _arrays;
    long _position;
    int _arrayNumber;
    int _posInArray;

    public MultiArrayMemoryStream(byte[][] arrays){
        _arrays = arrays;
        _position = 0;
        _arrayNumber = 0;
        _posInArray = 0;
    }

    public override int Read(byte[] buffer, int offset, int count){
        int read = 0;
        while(read<count){
            if(_arrayNumber>=_arrays.Length){
                return read;
            }
            if(count-read <= _arrays[_arrayNumber].Length - _posInArray){
                Buffer.BlockCopy(_arrays[_arrayNumber], _posInArray, buffer, offset+read, count-read);
                _posInArray+=count-read;
                            _position+=count-read;
                read=count;
            }else{
                Buffer.BlockCopy(_arrays[_arrayNumber], _posInArray, buffer, offset+read, _arrays[_arrayNumber].Length - _posInArray);
                read+=_arrays[_arrayNumber].Length - _posInArray;
                            _position+=_arrays[_arrayNumber].Length - _posInArray;
                _arrayNumber++;
                _posInArray=0;
            }
        }
        return count;
    }

    public override long Length{
        get {
            long res = 0;
            for(int i=0;i<_arrays.Length;i++){
                res+=_arrays[i].Length;
            }
            return res;
        }
    }

    public override long Position{
        get { return _position; }
        set { throw new NotSupportedException(); }
    }

    public override bool CanRead{
        get { return true; }
    }

    public override bool CanSeek{
        get { return false; }
    }

    public override bool CanWrite{
        get { return false; }
    }

    public override void Flush(){
    }

    public override void Seek(long offset, SeekOrigin origin){
        throw new NotSupportedException();
    }

    public override void SetLength(long value){
        throw new NotSupportedException();
    }

    public override void Write(byte[] buffer, int offset, int count){
        throw new NotSupportedException();
    }       
}

2 ^ 31 UnmanagedMemoryStream, System.IO.Stream ( , ). - ():

var fileStream = new FileStream("data", 
  FileMode.Open, 
  FileAccess.Read, 
  FileShare.Read, 
  16 * 1024, 
  FileOptions.SequentialScan);
long length = fileStream.Length;
IntPtr buffer = Marshal.AllocHGlobal(new IntPtr(length));
var memoryStream = new UnmanagedMemoryStream((byte*) buffer.ToPointer(), length, length, FileAccess.ReadWrite);
fileStream.CopyTo(memoryStream);
memoryStream.Seek(0, SeekOrigin.Begin);
// work with the UnmanagedMemoryStream
Marshal.FreeHGlobal(buffer);

Answer 2

. .

, .

Answer 3

memoryStream, , Write

EDIT: MemoryStream - , , , . , , , . . , .

Answer 4

, 2D-, .

[int.MaxValue] [10] [int.MaxValue * 10]. [4,5] int.MaxValue * (4-1) + (5-1). ( (i-1) * + (j-1).

, .

Answer 5

, , , . , .

, , , . , byte[int.MaxValue], ?

Answer 6

Framework 4.0, MemoryMappedFile. Windows. , / , .

If you are not using Framework 4.0, you can still use this option, but you need to either write your own or find exsiting wrapper. I expect that there will be a lot on the "Project Code".