Sockets: high performance messaging

Question

I have doubts about using Berkeley Sockets under Ubuntu. As for performance and reliability, which option is better? To send a large number of messages, but with a small length, or to send a small number of messages, but with a large size? I do not know what is the basic design rule that I must follow.

Thanks everyone!

Answer 1

In terms of reliability, if you do not have special requirements, do not worry about a lot. If you talk about TCP, this will do better than you manage things until you come across some kind of edge case that really requires you to play with some buttons, in which case there would be a more specific question. Regarding packet size, with TCP, if you don't bypass the Nagel algorithm , you really have no control that you might think.

In UDP, it's probably best to use path MTU discovery , which TCP does for you automatically, but as a general rule, you use something perfectly in the 500 byte range. If you start to think too much, you will find that you are inventing parts of TCP.

Answer 2

When using TCP, one parameter must use the TCP_CORK option. See getsockopt page. Set TCP_CORK to the socket, write a packet of small messages, then delete the TCP_CORK parameter, and they will be transmitted in the minimum number of network packets. This can increase throughput by increasing latency.

Answer 3

Each network message has a 40-byte header, but large messages are harder to route and easier to lose. If you're talking about UDP, so the best message size is the Ethernet block, which is 1496 bytes long; if yiu uses TCP, leave it at the network level to handle how much you need to send.

Answer 4

Performance you can learn iperf . Do some experimentation and you will see it yourself. As for reliability, as I understand it, if you use TCP, a TCP connection ensures that the data will be delivered, of course, if the connection is not broken.

Answer 5

“Regarding performance and reliability, which option is best.”

On the lossy layer, performance and reliability are almost direct compromises against each other, and larger experts than us put years of work in search of sweet spots and methods that surpassed the direct compromise and improved at one time.

You have two main options:

1) Use stream sockets (TCP). Any "Messages" that your application knows about are defined at the application level, not in sockets. For example, you can consider an HTTP request as a message, and the response is different in the opposite direction. You should see that your job is to keep the output buffer as complete as possible, and the input buffer as empty as possible. Reliability is almost not related to the length of the message, and for a fixed size of data performance, it is mainly determined by the number of executed request-response rather than the number of individual entries in the socket. Obviously, if you send one byte at a time with TCP_NODELAY, you will lose performance, but this is quite extreme.

2) Use datagrams (UDP). Messages are socket level objects. Performance is potentially better than TCP, but you should invent your own system for reliability, and perhaps this will lead to performance degradation due to the need to transfer data. TCP has the same problem, but big minds, etc. The length of the datagram can be very inconvenient to interact with both performance and reliability, hence the MTU discovery mentioned by Duck. If you send a large packet and fragment it, then if any fragment goes astray, your message will not be received. There is size N, where if you send datagrams of N-size, they will not be fragmented, but if you send datagrams N + 1, they will be. Therefore, +1 doubles the number of failed messages. You do not know N until you know the network route (and perhaps not even then). Therefore, at the time of compilation it is impossible to say what sizes will give good performance: even if you evaluate it, it will be different for different users. If you want to optimize, there is no alternative to knowing your things.

UDP also works more than TCP if you need reliability that is built into TCP. Potentially, UDP has big gains, but it should probably be considered assembly language socket programming.

There also (3): use a protocol to increase UDP reliability, such as RUDP. This is not part of the Berkeley-style socket APIs, so you'll need a library to help.