Partial Stream MD5 Hash Calculation

Question

I have a large data set (~ 1 GB) stored in a user file format, the last 16 bytes of which are the MD5 hash of all previous bytes of the file.

I want to test MD5 of this file for embedded MD5 using HashAlgorithm.ComputeHash (Stream), however this will calculate the hash of the whole file, INCLUDING the hash in the last 16 bytes, which obviously does not work.

How can I calculate the MD5 hash from a PART stream? I know that I can read the stream into an array and pass it to HashAlgorithm.ComputeHash (Bytes), however, the overhead of duplicating this 1 GB of data in memory is forbidden.

Answer 1

Taken from here, where you can also get other ways to do this.

Create a partial file stream class, read the size you need and make it a hash.

class PartialFileStream : FileStream { public PartialFileStream(string path, FileMode mode, long startPosition, long endPosition): base(path, mode) { base.Seek(startPosition, SeekOrigin.Begin); ReadTillPosition = endPosition; } public long ReadTillPosition { get; set; } public override int Read(byte[] array, int offset, int count) { if (base.Position >= this.ReadTillPosition) return 0; if (base.Position + count > this.ReadTillPosition) count = (int)(this.ReadTillPosition - base.Position); return base.Read(array, offset, count); } } 

Answer 2

You can use the FileStream.Seek option to search for a specific stream position and read from there.

Answer 3

I found that I needed this a second time in 6 months, so I am publishing my solution for a partial input stream.

 class PartialStream: Stream { public Stream Source { get; } public long Offset { get; } public override long Length { get; } private long End => Offset + Length; public override bool CanRead => true; public override bool CanSeek => false; public override bool CanWrite => false; public override long Position { get => Source.Position - Offset; set => throw new NotSupportedException(); } public PartialStream(Stream source, long length) { Offset = source.Position; Length = length; } public PartialStream(Stream source, long offset, long length, bool seekToOffset = true) { if (seekToOffset) source.Seek(offset, SeekOrigin.Begin); Offset = offset; Length = length; } public override int Read(byte[] array, int offset, int count) { if (Source.Position >= End) return 0; if (Source.Position + count > End) count = (int)(End - Source.Position); return Source.Read(array, offset, count); } public override void Flush() => throw new NotSupportedException(); public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); public override void SetLength(long value) => throw new NotSupportedException(); public override void Write(byte[] buffer, int offset, int count) => throw new NotSupportedException(); }