Building and sending binary data over a network

Question

I am creating a command line client for minecraft. The protocol has a full specification: http://mc.kev009.com/Protocol . To answer your question in advance, yes, I'm a bit of C ++ noob.

I have various problems implementing this protocol, each of which is critical.

• The protocol says that all types are large. I have no idea how I should check if my data is insignificant, and if so, how to convert to big-endian.
• The string data type is a bit strange. This is a modified UTF-8 string, preceded by a short string containing the length of the string. I don't know how I should pack this into a simple char [] array, or how to convert my simple strings to modified UTF-8.
• Even if I knew how to convert my data to big-endian and create modified UTF-8 strings, I still don't know how to pack them into a char [] array and send it as a package. All I did before is a simple HTTP network, which is simple ASCII.

The concepts, links, related function names and short snippets are appreciated!

EDIT

1 and 3 are now responding. 1 replied by user user470379. 3 answers this AWESOME theme which explains what I want to do very well: http://cboard.cprogramming.com/networking-device-communication/68196- send-non char * - data.html However, I'm not sure about the modified UTF -eight.

Answer 1

The traditional approach is to determine the structure of C ++ messages for each protocol message and implement serialization and deserialization functions. For example, an entry request may be presented as follows:

#include <string> #include <stdint.h> struct LoginRequest { int32_t protocol_version; std::string username; std::string password; int64_t map_seed; int8_t dimension; }; 

Now you need serialization functions. First, he needs serialization functions for integers and strings, as these are member types in LoginRequest .

Integer serialization functions must do the conversion in and out of the big-endian view. Since message members are copied to and from the buffer, byte ordering can be done when copying:

 #include <boost/detail/endian.hpp> #include <algorithm> #ifdef BOOST_LITTLE_ENDIAN inline void xcopy(void* dst, void const* src, size_t n) { char const* csrc = static_cast<char const*>(src); std::reverse_copy(csrc, csrc + n, static_cast<char*>(dst)); } #elif defined(BOOST_BIG_ENDIAN) inline void xcopy(void* dst, void const* src, size_t n) { char const* csrc = static_cast<char const*>(src); std::copy(csrc, csrc + n, static_cast<char*>(dst)); } #endif // serialize an integer in big-endian format // returns one past the last written byte, or >buf_end if would overflow template<class T> typename boost::enable_if<boost::is_integral<T>, char*>::type serialize(T val, char* buf_beg, char* buf_end) { char* p = buf_beg + sizeof(T); if(p <= buf_end) xcopy(buf_beg, &val, sizeof(T)); return p; } // deserialize an integer from big-endian format // returns one past the last written byte, or >buf_end if would underflow (incomplete message) template<class T> typename boost::enable_if<boost::is_integral<T>, char const*>::type deserialize(T& val, char const* buf_beg, char const* buf_end) { char const* p = buf_beg + sizeof(T); if(p <= buf_end) xcopy(&val, buf_beg, sizeof(T)); return p; } 

And for strings (handling modified UTF-8 is the same as asciiz strings ):

 // serialize a UTF-8 string // returns one past the last written byte, or >buf_end if would overflow char* serialize(std::string const& val, char* buf_beg, char* buf_end) { int16_t len = val.size(); buf_beg = serialize(len, buf_beg, buf_end); char* p = buf_beg + len; if(p <= buf_end) memcpy(buf_beg, val.data(), len); return p; } // deserialize a UTF-8 string // returns one past the last written byte, or >buf_end if would underflow (incomplete message) char const* deserialize(std::string& val, char const* buf_beg, char const* buf_end) { int16_t len; buf_beg = deserialize(len, buf_beg, buf_end); if(buf_beg > buf_end) return buf_beg; // incomplete message char const* p = buf_beg + len; if(p <= buf_end) val.assign(buf_beg, p); return p; } 

And a few helper functors:

 struct Serializer { template<class T> char* operator()(T const& val, char* buf_beg, char* buf_end) { return serialize(val, buf_beg, buf_end); } }; struct Deserializer { template<class T> char const* operator()(T& val, char const* buf_beg, char const* buf_end) { return deserialize(val, buf_beg, buf_end); } }; 

Now, using these primitive functions, we can easily serialize and deserialize the LoginRequest message:

 template<class Iterator, class Functor> Iterator do_io(LoginRequest& msg, Iterator buf_beg, Iterator buf_end, Functor f) { buf_beg = f(msg.protocol_version, buf_beg, buf_end); buf_beg = f(msg.username, buf_beg, buf_end); buf_beg = f(msg.password, buf_beg, buf_end); buf_beg = f(msg.map_seed, buf_beg, buf_end); buf_beg = f(msg.dimension, buf_beg, buf_end); return buf_beg; } char* serialize(LoginRequest const& msg, char* buf_beg, char* buf_end) { return do_io(const_cast<LoginRequest&>(msg), buf_beg, buf_end, Serializer()); } char const* deserialize(LoginRequest& msg, char const* buf_beg, char const* buf_end) { return do_io(msg, buf_beg, buf_end, Deserializer()); } 

Using the helper functors above and presenting I / O buffers as char ranges of iterators requires only one function template to serialize and deserialize the message.

And all together, use:

 int main() { char buf[0x100]; char* buf_beg = buf; char* buf_end = buf + sizeof buf; LoginRequest msg; char* msg_end_1 = serialize(msg, buf, buf_end); if(msg_end_1 > buf_end) ; // more buffer space required to serialize the message char const* msg_end_2 = deserialize(msg, buf_beg, buf_end); if(msg_end_2 > buf_end) ; // incomplete message, more data required } 

Answer 2

For # 1 you need to use ntohs and friends. Use *s (short) versions for 16-bit integers and *l (long) versions for 32-bit integers. hton* (the host for the network) converts the outgoing data to big-endian regardless of the platform of the platform you are on, and ntoh* (the network for the host) converts the incoming data back (again, regardless of the final platform)

Answer 3

from my head ...

 const char* s; // the string you want to send short len = strlen(s); // allocate a buffer with enough room for the length info and the string char* xfer = new char[ len + sizeof(short) ]; // copy the length info into the start of the buffer // note: you need to hanle endian-ness of the short here. memcpy(xfer, &len, sizeof(short)); // copy the string into the buffer strncpy(xfer + sizeof(short), s, len); // now xfer is the string you want to send across the wire. // it starts with a short to identify its length. // it is NOT null-terminated.