Latent bursts when recording real-time data on Linux

Question

I have a real-time priority robot control code on a PREEMPT-RT fixed Linux OS on Beaglebone Black. All code is written in C and runs at a frequency of 500 Hz.

I noticed latency in the range of several hundred milliseconds each so often when running the code, and I tracked it down to the data logging function that I wrote. This delay makes my robot control fail because I have a lot, depending on the functionality in real time.

Below is the relevant piece of code. I will draw a lot of code for clarity, but I will edit this post if anything is needed.

FILE *file; int main(int argc, char** argv) { file = fopen(logname, "w"); while (1) { /* Control code stuff*/ logData(); time_msec = time_msec + controlLoopTime; } } void logData() { if (time_msec - logTimer_msec >= LOG_TIMER) { logTimer_msec = time_msec; if (!bLogCreated) { fprintf(file, "SensorData1 SensorData2 SensorDataN" ); bLogCreated = TRUE; } // log data to file fprintf(file, "%.2f %.2f\n", sensorData1, sensorData2, sensorDataN ); } } 

I need to write data from several variables (maybe 20-50) with a good speed, maybe 100-125 Hz. Data does not need to be recorded at a control speed (every 2 ms), but I reduced it to 12 ms, and I still see latent bursts every few minutes.

Delay can be a problem when calling fprintf . Is this a limitation of BeagleBone Black, my code, or just the nature of data logging?

A similar question was asked here, but it did not seem to touch upon my problem: Finding latency problems (stalls) in embedded Linux systems

Answer 1

Using fprintf is a huge time receiver, especially for registering R / T. Record in binary format and write a utility to print it later.

Instead:

 fprintf(file,"%.2f %.2f %.2f",data1,data2,data3); 

make:

 fwrite(&data1,sizeof(double),1,file); fwrite(&data2,sizeof(double),1,file); fwrite(&data3,sizeof(double),1,file); 

Even better:

 struct data { double data1; double data2; double data3; time_t event_time; ... }; struct data data; fwrite(&data,sizeof(struct data),1,file); 

---

If it is still too slow, add the structure to the ring queue and select a separate thread.

If writing to a disk cannot keep up with binary data [now], support the call queue and unload the queue after opening if you find a fatal error

---

Also, consider using mmap to access the file when recording. See My answer [with benchmarks] here: read in turn in the most efficient way * specific platform *