Protocol buffers error, multiple serialization to binary

Question

I get some weird behavior from the protobuf io binary file. I pre-process the text body into a protobuf intermediate file. My serialization class is as follows:

class pb_session_printer { public: pb_session_printer(std::string & filename) : out(filename.c_str(), std::fstream::out | std::fstream::trunc | std::fstream|binary) {} void print_batch(std::vector<session> & pb_sv) { boost::lock_guard<boost::mutex> lock(m); BOOST_FOREACH(session & s, pb_sv) { std::cout << out.tellg() << ":"; s.SerializeToOstream(&out); out.flush(); std::cout << s.session_id() << ":" << s.action_size() << std::endl; } exit(0); } std::fstream out; boost::mutex m; }; 

The output of the fragment is as follows:

 0:0:8 132:1:8 227:2:6 303:3:6 381:4:19 849:5:9 1028:6:2 1048:7:18 1333:8:28 2473:9:24 

The first field shows that serialization continues as usual.

When I run my boot program:

 int main() { std::fstream in_file("out_file", std::fstream::in | std::ios::binary); session s; std::cout << in_file.tellg() << std::endl; s.ParseFromIstream(&in_file); std::cout << in_file.tellg() << std::endl; std::cout << s.session_id() << std::endl; s.ParseFromIstream(&in_file); } 

I get:

 0 -1 111 libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "session" because it is missing required fields: session_id 

session_id: 111 is a record at the end of the stream, I obviously do not understand the semantics of the binary capabilities of the library. Please, help.

Answer 1

If you write several protobuffers in one file, you will need to write the protobuf + protobuffer size and read them separately (so without ParseFromIstream , as Cat Plus Plus mentioned). When you read in protobuffer, you can ParseFromArray it with ParseFromArray .

Your file will look like this (spaces are just for reading):

protobuf size protobuf protobuf protobuf size etc.

Answer 2

Message::ParseFromIstream documented to consume all input. Since you serialize a sequence of messages of the same type, you can simply create a new message with the repeated field of this type and work with it.