Getting variable data size over TCP sockets

Question

I am having a slight problem with transmitting data over (TCP) sockets. A little background about what I'm doing:

I am sending data from side A to B. The transmitted data may be of variable length, assuming that the maximum size should be 1096 bytes.

A) send(clientFd, buffer, size, NULL) 

to B, since I don’t know what size to expect, I always try to get 1096 bytes:

 B) int receivedBytes = receive(fd, msgBuff, 1096, NULL) 

However, when I did this: I realized that A sends small pieces of data for about 80-90 bytes. After several outbursts of sending, B combined them together to get Bytes to be 1096. This obviously misrepresented data and hell broke.

To fix this, I broke my data into two parts: header and data.

 struct IpcMsg { long msgType; int devId; uint32_t senderId; uint16_t size; uint8_t value[IPC_VALUES_SIZE]; }; 

On the side:

 A) send(clientFd, buffer, size, NULL) 

on B, I first get the header and determine the size of the payload to receive: and then I get the rest of the payload.

 B) int receivedBytes = receive(fd, msgBuff, sizeof(IpcMsg) - sizeof( ((IpcMsg*)0)->value ), 0); int sizeToPoll = ((IpcMsg*)buffer)->size; printf("Size to poll: %d\n", sizeToPoll); if (sizeToPoll != 0) { bytesRead = recv(clientFd, buffer + receivedBytes, sizeToPoll, 0); } 

So, for every dispatch that has a payload, I get a call twice. This worked for me, but I was wondering if there is a better way to do this?

Answer 1

You are on the right lines with the idea of ​​sending a header that contains basic information about the following data, followed by the data itself. However, this does not always work:

 int receivedBytes = receive(fd, msgBuff, sizeof(IpcMsg) - sizeof( ((IpcMsg*)0)->value ), 0); int sizeToPoll = ((IpcMsg*)buffer)->size; 

The reason is that TCP can freely fragment and send your header into as many blocks as it sees fit, based on its own assessment of the underlying network conditions applied to what is called a congestion management strategy. On a local network, you almost always get your header in one packet, but try it all over the world via the Internet, and you can get a lot less bytes at a time.

The answer is not to directly call TCP to "receive" (usually recv ), but to divert it to a small utility function that takes the size that you really need to get, and a buffer for entering it. Go to receive and add packets until all data appears or an error occurs.

If you need to go asynchronously and serve several clients at the same time, then the same principle applies, but you need to examine the "select" call, which allows you to receive notifications of data arrival.

Answer 2

TCP / IP is a raw interface for sending data. This ensures that if the bytes are sent, that they are all there and are in the correct order, but does not give any guarantees regarding chunking and does not know anything about the data you are sending.

Therefore, when sending a “packet” over TCP / IP, which should be processed as such, you should know when you have a complete packet in one of the following ways:

• Fixed-size packages. In your case, 1096 bytes
• First, send / receive a well-known “header” that will tell you the size of the packet to be sent.
• Use some kind of "end of package" symbol.

In either of the first two, you know the number of bytes you expect to receive, so you need to buffer everything you get until you get the full message, and then process this.

If you get more than you expected, that is, it flows into the next packet, you break it, process the completed packet, and leave the rest buffered for further processing.

In the latter case, when you have the end of a packet symbol, which may be anywhere in your message, so that everything that follows it, you buffer for the next packet.