Protobuf Checksum (crc)

Question

I am going to store some large objects in a database (BLOB). And protobuf, as I see it, is one of the best candidates for BLOB serialization / deserialization. Despite the fact that it has a binary format, it is still easy to read and change its contents (strings, integers, etc.). So I need some kind of data verification, whenever its original BLOB or modified (by a hacker? Too smart user?).

One possibility would be to have a selected field in the table, call it crc, calculate the BLOB checksum, and put it there. But it would be much better (in many scenarios) when crc is part of the BLOB itself.

I can add extra bytes to the end of the protobuf stream, but I will have to delete them (or the deserializer will throw an "invalid blablabla field" exception).

I can put the protobuf stream in a shell, but again, the overhead of a U / W.

Is there an easy and cheap way to add something to the end of a protobuf stream to avoid the need for extra operations during deserialization? In XML, I could add a comment. I don’t think there is a comment in protobuf, but how to put a CRC that will look like 1 or 2 bytes?

Answer 1

Protobuf . , , . 1 2 CRC, , , "varint" ( , "varint" - 7- 8- , , , 7, 14 21 CRC), :

• , 3 , varint encoded
• CRC, varint

! , , , .

, , . , :

• 4 protobuf, "n"
• "n" protobuf
• 2 CRC, "n"

, , . , "varint" , , . , CRC, .

Answer 2

Crc . , Seek ( ).

:

// serialize
using (var file = File.Create("test.bin"))
using (var mem = new MemoryStream())
{
    Serializer.Serialize(mem, obj); // serialize obj into memory first
    // ... calculate crc
    file.Write(new byte[] { crc }, 0, 1);
    mem.WriteTo(file);
}

// deserialize
using (var file = File.OpenRead("test.bin"))
{
    var crc = file.ReadByte();
    // ... calculate and check crc
    file.Seek(1, SeekOrigin.Begin);
    Serializer.Deserialize<ObjType>(file);
}