Byte compression array

Question

My problem: I want to save an array of bytes in a compressed file, and then I want to read it with good performance.

So, I create an array of bytes, then move on to the ZLIB algorithm, and then save it in a file. To my surprise, the algorithm does not work very well. Probably because the array is a random pattern. Using this approach, it will be easy to read. Just copy the stream to memory, unzip them and copy them to an array of bytes.

But I need to compress the file. Should I use an algorithm like RLE to compress an array of bytes? I think I can store an array of bytes as a string, and then compress it. But I think that I will have poor performance when reading data.

Sorry for my bad english.

thank

Answer 1

It is unclear whether you mean "random sampling" in a statistical sense (a sample of real data randomly selected) or simply randomly generated data. The first is to squeeze the fine.

However, truly random data cannot be compressed (good). In fact, compressibility is one measure of randomness .

Answer 2

If the data is random, he will have high entropy . There is no algorithm that could compress such data with great success. That is, since this is random, you may be lucky in a particular case, but it is usually futile to try to squeeze it.

Answer 3

Are you sure you really need to compress random data? Better compression can be achieved if you know what the data source is and some other specifications that you can use to select and optimize compression algorithms. First, how big are your arrays? If these are symbol values, the difference is in the frequency of each letter. If this array is sorted or it has unique values, it can also be used to optimize space. So what is the source of your data?