Read 10MB from file first with C #

Question

I need to get the first 10 megabyte file and calculate md5 from this, how can I achieve this? I can not find a sample sample to read. I have something like this:

FileStream file = new FileStream(fileName, FileMode.Open); MD5 md5 = new MD5CryptoServiceProvider(); byte[] retVal = md5.ComputeHash(file); file.Close(); StringBuilder sb = new StringBuilder(); for (int i = 0; i < retVal.Length; i++) { sb.Append(retVal[i].ToString("x2")); } var md5Final = sb.ToString(); 

But it reads the whole file.

Answer 1

You can read the file in chunks and pass it to MD5CryptoServiceProvider in chunks using TransformBlock . Thus, you do not need to consume 10 MB of memory for the buffer. Example:

 long read = 0; int r = -1; const long bytesToRead = 10 * 1024 * 1024; const int bufferSize = 10*1024; byte[] buffer = new byte[bufferSize]; MD5 md5 = new MD5CryptoServiceProvider(); using(var stream = new FileStream(fileName, FileMode.Open, FileAccess.Read) ) { while(read <= bytesToRead && r != 0) { read += (r = stream.Read(buffer, 0, bufferSize)); md5.TransformBlock(buffer, 0, r, null, 0); } } md5.TransformFinalBlock(buffer, 0,0); string md5Final = String.Join("", md5.Hash.Select(x => x.ToString("x2"))); 

Answer 2

To read part of a file with small pieces, you can try:

 public byte[] ReadPart(Stream stream) { byte[] buffer = new byte[1024]; int read=0; int chunk; while ( (chunk = stream.Read(buffer, read, buffer.Length-read)) > 0) { read += chunk; if (read == buffer.Length && read < 10*1024*1024) // 10MB { int nextByte = stream.ReadByte(); if (nextByte==-1) { return buffer; } byte[] newBuffer = new byte[buffer.Length*2]; Array.Copy(buffer, newBuffer, buffer.Length); newBuffer[read] = (byte)nextByte; buffer = newBuffer; read++; } } // Buffer is now too big. Shrink it. byte[] ret = new byte[read]; Array.Copy(buffer, ret, read); return ret; } 

and then calculate MD5 in this array ...

Answer 3

  var bytes = new byte[10000000]; // Or 10*1024*1024 according to your def of a MB int realLength; using (var file = new FileStream(filename, FileMode.Open)) { realLength = file.Read(bytes, 0, bytes.Length); // The file may be shorter than expected. } var md5 = new MD5CryptoServiceProvider(); byte[] hash = md5.ComputeHash(bytes, 0, realLength); 

I have to admit that allocating 10 MB might not be the most elegant solution. I wrote it with 10 KB instead of 10 MB. YMMV.

Answer 4

You can read the first 10Mb in bytes [], and then provide a ComputeHash instance of a MemoryStream supported by byte [].

Answer 5

You can implement the Stream class, which limits the length of the underlying Stream object ( FileStream in your case):

 public class LengthLimitedStream : Stream { private Stream baseStream; private long maxLength; public LengthLimitedStream(Stream stream, long maxLength) { if (stream.Position > maxLength) { throw new IOException(); } this.baseStream = stream; this.maxLength = maxLength; } public override bool CanRead { get { return this.baseStream.CanRead; } } public override long Length { get { return Math.Min(this.maxLength, this.baseStream.Length); } } public override long Position { get { return this.baseStream.Position; } set { if (value > maxLength) { throw new IOException(); } this.baseStream.Position = value; } } public override int Read(byte[] buffer, int offset, int count) { if (this.Position + offset + count > this.maxLength) { count = (int)(this.maxLength - (this.Position + offset)); } return this.baseStream.Read(buffer, offset, count); } // Lots of stuff omitted you may want to implement but not important in this case ... } 

(add parameter checking and such things, but you get the idea)

Thus, you can still use the ComputeHash(Stream) method, and it looks clean and simple (in my opinion).

Answer 6

How about converting the file to an array of bytes so that you can iterate over the first 10 MB. To get an array of bytes =>

 private byte [] StreamFile(string filename) { FileStream fs = new FileStream(filename, FileMode.Open,FileAccess.Read); // Create a byte array of file stream length byte[] fileData = new byte[fs.Length]; //Read block of bytes from stream into the byte array fs.Read(fileData,0,System.Convert.ToInt32(fs.Length)); //Close the File Stream fs.Close(); return fileData; //return the byte data } 

After that, I just go through fileData and take the first 10 MB and do MD5