Data structure for storing serial port data in firmware

Question

I am sending data from a linux application via a serial port to an embedded device.

In the current implementation, a byte circular buffer is used in the firmware. (Nothing but an array with a read and write pointer) When bytes enter, it is written to a round buffer.

Now the PC application seems to send data too quickly for the firmware to work. Bytes are skipped, as a result of which the firmware will return WRONG_INPUT too much time.

I think baud rate (115200) is not a problem. A more efficient data structure on the firmware side may help. Any suggestions for choosing a data structure?

Answer 1

The best answer is a circular buffer. This is the easiest way to simulate hardware FIFO in pure software.

The real problem is likely to be either how you collect bytes from the UART to put in a buffer, or overflow this buffer.

At 115200 bauds with the usual 1 start bit, 1 stop bit and 8 data bits, you can see as many as 11520 bytes per second. This gives you an average of about 86.8 μs per byte to work with. On a PC, it will look like a lot of time, but in a small microprocessor there may not be so many general instructions or, in some cases, a lot of requests for I / O registration. If you overflow your buffer because bytes arrive on average faster than you can use them, then you will have errors.

Some general recommendations:

• Do not poll I / O.
• Use interrupt Rx Ready.
• Enable FIFO reception, if available.
• Clear FIFO completely in the interrupt handler.
• Make the ring buffer large enough.
• Consider flow control.

The size of your loopback buffer is sufficient to hold the complete message. If your protocol knows the message size limits, you can use higher levels of your protocol to control the flow and survive effortlessly, so that the XON / XOFF flow works correctly in all cases, cross, or RTS / CTS work as expected at both ends of the wire, which can be almost as hairy.

If you cannot make the ring buffer so large, you will need some flow control.

Answer 2

There is nothing better than a circular buffer.

You can use a slower baud rate or speed up the application in the firmware so that it can process data arriving at full speed.

If the PC output is in packets, this can help make the buffer large enough to process one packet.

The final option is to implement some form of flow control.

Answer 3

What do you mean by an embedded device? I think that most modern DSPs and processors can easily handle such a load. The problem is not the ring buffer, but the way you collect bytes from the serial port.

Does your UART have fifo hardware? If so, you should enable it. If you have an interrupt by byte, you can get into trouble quickly, especially if you work with the OS or with virtual memory, where the cost of IRQ may be higher.

If your host firmware is very simple (without multitasking) and you don’t have fifo hardware, polling may be a better solution than interrupting because then your processor only accepts UART data and you have no interrupt overhead.

Another problem may be related to the transfer protocol. For example, if you have a long data packet that you have for the checksum and you execute the entire checksum at the end of the packet, then all the processing time of the packet is at the end of this, and that is why you can skip the beginning of the next packet.

So the circular buffer is beautiful, and you need to improve it: - The way to interact with the equipment - Protocol (packet length, icon, etc.)

Answer 4

Before trying to solve the problem, you first need to establish what the problem is. Otherwise, you may lose time trying to fix something that is not really broken.

Without knowing more about your setup, it’s hard to give more specific advice. But you should investigate further to determine what exactly the hardware and software are currently doing when the bytes come in, and then what is the weak point where they went missing.

Answer 5

The round buffer with controllable IO interrupt will work with the smallest and slowest of the built-in targets.

First try it at the lowest bit rate and only then try at high speeds.

Answer 6

Using a circular buffer in combination with an IRQ is a great offer. If your processor generates an interrupt every time a byte is received, take that byte and save it in the buffer. How you decide to clear this buffer depends on whether the data stream or data packets are being processed. If you are processing a stream, just follow the background process to remove the bytes from the buffer and process them first. If you are processing packets, just keep serving the buffer until you get the full packet. I have used the package method successfully many times in the past. I would use some type of flow control to signal the PC if something went wrong like the full buffer, or if the processing time of the packet indicates the PC for a long time when it is ready for the next packet.

Answer 7

You can implement something like an IP datagram that contains the data length, identifier, and checksum.

Edit : Then you can hardcode a fixed length for packets, like 1024 bytes or something else that makes sense for the device. The PC side will then check whether the queue on the device is full every time it is written in the packet. The firmware side ran a checksum to check if all the data is valid and read to the length of the data.