Reading large volumes of bmp data directly from a file

Question

I am working on an application that needs to deal with large BMP files that are often too large to fit in memory. As part of the software that we develop when creating the project, my application converts the data from the BMP file to another format, which makes it easy to extract into partitions.

Currently, since the file is often too large to install in memory, the program reads a section of byte data directly from the file, processes it, and moves to the next section. The code for reading the file is similar to the following (simplified for clarity):

FileStream fs = File.OpenRead(fileName); fs.Seek(sectionStart, SeekOrigin.Begin); currentSectionAsBytes = new byte[sectionSize]; fs.Read(currentSectionAsBytes, 0, currentSectionAsBytes.Length); 

This still works, since the files we work with all have a width that is divisible by 4, and therefore the bmp files do not have a complement.

Recently, we have been working with images of various sizes, which are filled at the end of each row of data, as a result of which the received data is erroneous.

My question is if anyone knows of the best ways to get BMP data directly from a file. As stated earlier, I cannot read Bitmap from a file due to the large file size. For now, my best idea is to get the size of the fill and manually delete it after receiving currentSectionAsBytes. It seems too cumbersome and complicated. There must be a better solution.

Answer 1

Manual deletion from an array is a bad idea when you run out of memory (as this will lead to the creation of another array). I think the right approach to your problem is to add two additional parameters to your processing method, so its syntax will look like this:

 void ProcessSection(byte[] data, int offset, int count); 

And then repeat data not from 0 to data.Length , but from offset to count . This is the usual way to work with arrays in .Net (for example, look at the fs.Read() method), much more can be done. And yes, you will have to calculate the offset / quantity according to your bitmap properties.

Ofc, alternatively, you can consider offset when you call the fs.Seek method (for example, when you calculate sectionStart ), but I think the first apporach is more universal.