Using std :: endl in ostream makes my file binary

Question

I am working on a project that uses libzip. I work in C ++ 14 and I wrote a tiny wrapper around libzip to make my life easier.

I have a std::ostream object built around a custom class that inherits std::streambuf . This streambuf uses libzip functions to write to a file in the archive.

Everything works fine until I use std::endl . When I do this, the output file is read as binary by all my text readers (for writing lines only).

My text reader discovers that it is binary, because in the place where I used std::endl there is a NUL byte, any file with a NUL byte inside it is treated as binary.

So my question is: is this normal? Is there a way to use std::endl ?

My code (extracted so that it is not exactly the same).

source.hpp

 // my attributes std::unique_ptr<zip_source_t, std::function<void(zip_source_t*)>> _source; std::unique_ptr<std::ostream> _stream; std::unique_ptr<_ZipBuffer> _buffer; class _ZipBuffer : public std::streambuf { private: zip_source_t* _source; std::streamsize xsputn(char const* s, std::streamsize n) override; int overflow(int c) override; public: _ZipBuffer(zip_source_t* file); }; 

source.cpp

 // create the streambuf and send it to the ostream _buffer.reset(new _ZipBuffer(_source.get())); _stream.reset(new std::ostream(_buffer.get())); // the implementation of _ZipBuffer Zip::Source::_ZipBuffer::_ZipBuffer(zip_source_t* source) { _source = source; } std::streamsize Zip::Source::_ZipBuffer::xsputn(char const* s, std::streamsize n) { return zip_source_write(_source, s, n * sizeof(char)); } int Zip::Source::_ZipBuffer::overflow(int c) { return zip_source_write(_source, &c, sizeof(int)); } 

main.cpp

 Zip::Source src; src << "Some text and a number : " << 2.5 << std::endl; src << "another line !"; // zip is an object of class Zip that takes my source and write it in the archive zip.addFile("test.txt", src); 

If I remove std::endl in my main body, the text file will be recognized as a text file. If I add it, it will be recognized as a binary file.

The binary is a valid utf-8 output (except for the NUL byte):

 496c 2065 7374 2070 6f73 7369 626c 6520 6427 c3a9 6372 6972 6520 6465 7320 6e6f 6d62 7265 7320 c3a0 2076 6972 6775 6c65 203a 2032 2e35 0a00 0000 736f 6d65 7468 696e 6720 656c 7365 

Thanks!

Answer 1

You have implemented overflow() as follows:

 int Zip::Source::_ZipBuffer::overflow(int c) { return zip_source_write(_source, &c, sizeof(int)); } 

Your C ++ library apparently implements std::endl by calling overflow() with '\n' , passed as a parameter.

This perfectly matches the C ++ specification. Your overflow () implementation has an error.

The overflow() parameter is a single character passed as an int . Your implementation writes the entire binary int file to the output file, which is exactly what you see. Your sizeof(int) apparently 4, so you see 0x0a and three more empty bytes written to the output.

Answer 2

Ok, just found a problem ...

In the overload of std :: streambuf :: overflow, I wrote: I write an integer when I get a char. So, the leading 0 in my int was written to the file.

I had to drop my int to char and the problem disappeared :)