A deflate compression stream where pre-compressed data can be inserted. Is there a .NET lib?

Question

I use Deflate and GZip compression for web content..NET Framework DeflateStream works very well (it does not compress as well as SharpZipLib, but it is much faster). Unfortunately, it (and all other libraries I know) misses the function of recording pre-compressed data, such as stream.WritePrecompressed (byte [] buffer).

Using this function, you could insert pre-compressed blocks into a stream. This can reduce CPU utilization to compress this part and increase the overall throughput of the web server.

Is there any managed lib capable of doing this? Or is there a good starting point outside of ComponentAce's ZLIB.NET to do this?

Answer 1

Another approach is to flush the deflater stream (and possibly close it) to ensure that all buffered compressed data is written to the output stream and then just write your pre-compressed data to the original output stream and then reopen the deflater stream on top of your output flow again.

Answer 2

IIRC # ZipLib allows you to set the compression level, did you try to reset the stream and reset the level to 0, and then sent the already compressed data before increasing the compression level again?

If you are only looking at this for performance reasons, this might be an acceptable solution.

Answer 3

Yes, you can insert pre-compressed blocks into a zlib stream. Start with the zpipe.c example in the zlib source. Only where you want to insert the pre-compressed block, replace Z_NO_FLUSH with Z_FULL_FLUSH (otherwise, do not use Z_FULL_FLUSH, because the compression ratio will suffer.)

Now the compressed output is byte aligned and the last deflation block is closed. A full flash means that the next block after the pre-compressed block cannot contain any backlinks.

Add a pre-compressed block to the output stream (e.g. memcpy). Move strm.next_out to the next empty byte. Keep deflating where you left off.

flush = feof(source) ? Z_FINISH : Z_NO_FLUSH; ret = deflate(&strm, flush);