Can a protocol protocol parse a debug string message?

Question

I would like to use the protocol buffer in my program to read data from a file. I would also like to be able to edit the data file with any text editor, for starters (later I will write a data editor and switch to the full binary file).

Is there a way to parse a human readable format? (debug string provided by protobuf itself or another format).

Answer 1

There is also a text format, but support for this is implementation specific. For example, I do not support it at all in protobuf-net. But yes: this is defined and discussed (for example) here: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.text_format.html

Personally, I would rather use a binary and write an interface around the model.

Answer 2

If you don't mind using command line tools, the Piqi project includes piqi convert to convert four formats: binary protocol buffers, JSON, XML, and Piq . The Piq format is specially designed for viewing and editing data in a text editor.

Answer 3

Are you sure you want to use ProtoBuf? You can use Json first, and then switch to Bson or MessagePack as a binary format.

The Json / Bson combination has the advantage that you can use the same library for them (Json.net). I believe Bson is a little bigger than ProtoBuf.

Or you can use Json / MessagePack. Technically, MessagePack is a better binary format than Bson / ProtoBuf IMO. But toolkit support is worse, and you will need a separate library for Json and MessagePack. It supports all Json and much more (in particular, it can use both dictionary and whole keys in dictionaries).

A quick comparison of MsgPack and ProtoBuf:

• The resulting data size, if similar constructions are used, seems comparable.
• Encoding / decoding performance is highly dependent on implementation, but I expect it to have a similar value.
• MsgPack describes itself more., In ProtoBuf you don’t even see if something is subordinate or blob.
• MsgPack supports non-integer keys in a dictionary. One thing that allows this is to save properties by name when you don't care about size and switch to integers where there are big profits.
• MsgPack stores the counter of elements instead of the size for arrays / dictionaries. This has the advantage that you don’t have to go back and fit in the size all the time, which makes it easier to write to the serializer and possibly gives a higher write speed. On the other hand, you cannot easily skip an element because you do not know its size.
• MsgPack naturally supports a superset of Json, so you can easily migrate from Json.
• Protocol support, documentation, and popularity are much better with ProtoBuf. In particular, ProtoBuf.net looks better than the C # code available for MsgPack.

Answer 4

There is no programming language specified in the question, and my answer concerns only Java.

In Java, Message instance toString returns a human-readable text format. Then the same format can be split into a Message instance on TextFormat.merge :

 String messageString = ... MyMessage.Builder builder = MyMessage.newBuilder(); TextFormat.merge(messageString, builder); MyMessage newMessage = builder.build(); 

( Variants of the merge method can also be read from the stream so as not to read the entire line of the message into memory.)