Multiple sendto () using UDP socket

Question

I have network software that uses UDP to communicate with other instances of the same program. For various reasons, I have to use UDP here.

Recently, I had problems sending huge datasets over UDP, and I had to implement a fragmentation system to split my messages into small pieces of data. So far, it worked well, but now I am facing a problem when I have to send a lot of data blocks.

I have the following algorithm:

• Split a message into small pieces of data (about 1,500 bytes)
• Iterate over the list of data blocks and for each, send it using sendto()

However, when I send many data blocks, the receiver receives the first 6 messages. Sometimes he misses the sixth and gets the seventh. It depends.

In any case, sendto() always indicates success. This always happens when I test my software on the loopback interface (127.0.0.1), but never on the LAN.

If I add something like std::cout << "test" << std::endl; between sendto() , then every frame will be received.

I know that UDP allows packet loss and that my frames can be lossy for many reasons, and I believe this is due to the fact that I'm sending pieces of data.

What would be the right approach?

• Implementing some kind of acknowledgment mechanism (like TCP) seems redundant.
• Adding some arbitrary wait time between sendto() is ugly and probably slows down performance.
• Increase (if possible) the receiver's internal UDP buffer? I don’t even know if this is possible.
• Something else?

I really need your advice here.

Many thanks.

Additional information on request

The reason I should use UDP is because I have a few limitations:

• TCP does not work with NAT traversal (at least without a specific configuration).
• Some messages may be lost. Some others cannot.
• Message delivery order does not matter.

Answer 1

The implementation of the confirmation mechanism sounds exactly as you need. Thus, you can guarantee that no more than N packets are simultaneously "in flight", and you can retransmit packets that have not been confirmed for too long.

Answer 2

If you lose packets through the loopback interface after sending only 6 or 7 packets, then it sounds like your receive buffer is too small. You can increase the size of setsockopt with the SO_RCVBUF option. However, if you send 1500 bytes, then if this is really a problem, it means that the receive buffer is only about 9K (or, most likely, 8K, however this seems pretty small by default). I believe that on Windows, the default receive buffer is 16K.

Even if you increase the size of the receive buffer, you still have to solve the problems that others talked about. A few more things to consider are trying to dynamically determine the maximum packet size to avoid fragmentation. In addition, it makes sense to make the packet size and the number of packets sent between manually configured settings.

Answer 3

An unreliable UDP call is a simplification that puts TCP as a panacea for all network problems. In the same vein, the definition of TCP as reliable is again erroneous. While it is true that TCP has mechanisms for attempting to transmit data, many of the errors that cause the UDP packet to fail will also cause TCP to fail.

For example, a hardware network error will have the same effect on UDP and TCP packets. If the error persists, TCP will not pass as accurately as UDP. In fact, in this case, TCP has the disadvantage that it will try to complete the lost cause longer. Now, if you send data over the Internet, TCP has some advantages, since the route sent by the packet cannot be predetermined. However, UDP is quite adequate for sending data over a local network. If your packages do not reach their destination, this indicates a hardware problem that needs to be fixed. TCP will not help here.

Also, when choosing a protocol, you should also understand your data. If your data is transient, such as reading from a sensor, it makes sense to use UDP over TCP. If a packet is lost in this situation, it does not really matter, since another packet will soon occur. TCP, on the other hand, will shut down and try again. By the time data arrives, it will already be obsolete.

Actually TCP is designed for streaming data. In this situation, it is important that all data packets are reliable and orderly. UDP is for packet data and is for this type of data. UDP is absolutely acceptable, reliable, has less overhead and detects and repairs faster due to network failures.

Answer 4

You must implement confirmation and retransmission. Require, for example, acks for each N packets and store N packets in the retransmission buffer.

(maybe you can get some ideas from rudp or implement Il over UDP)

UDP is unreliable and also does not control the flow. In short, you lose packets from time to time - especially if you send data quickly - the kernel or any routers / switches between them will drop packets if there is not enough space - and there is little magic that you can use to not do this occur.

Answer 5

TCP exists to solve just such a problem. Why is TCP not an option? You will have to solve all the same problems and, ultimately, ultimately get the same solution, only without the benefit of a decade of research, development and debugging of the TCP stack.

If you really have to use UDP, the first question you want to ask is that you are ready to give up TCP guarantees? Are you happy to receive packages out of order? Is it possible to lose a certain percentage of packets? Can you handle the arrival of duplicate packages? The answers to these questions will hopefully lead to development.

Without knowing your specifics, it is impossible to answer your question simply: "Do it, and everything will be fine," except, of course, "Do TCP, and everything will be fine."

Answer 6

UDP transmits datagrams and is unreliable.

TCP transmits data streams and is reliable.

What you want seems to be based on a datagram, but reliable. So you need to create something on UDP or TCP to give you this. I am sure that UDP has full protocol specifications that provide just that. You only need to find and implement them.

Answer 7

In any case, sendto () always indicates success. This always happens when I test my software on the loopback interface (127.0.0.1), but never on the LAN.

If I add something like std :: cout <<"test" <std :: cps; between sendto (), then every frame is accepted.

It looks like your receiver's buffer is too small.

some tips:

• Increase receiver buffer. setockopt SO_RCVBUF
• Let the application decide if the retransmission will be a lost packet.
• Make sure that the payload matches the MTU, which means the payload + HEADER <= 1500 to avoid ip fragmentation.