Verify that the contents of the file are actually written to disk, and not queued in the disk controller buffer

Question

I wrote a program that combines two small files into one larger file. At first I read the data from the input files, combining the data and writing the output to a temporary file. After that, I will rename the temp file to the desired file name (located in the same section on the disk). Here is the pseudo code:

FILE* fp_1 = fopen("file_1.dat", "r+b"); FILE* fp_2 = fopen("file_2.dat", "r+b"); FILE* fp_out = fopen("file_tmp.dat", "w+b"); // 1. Read data for the key in two files const char* data_1 = ...; const char* data_2 = ...; // 2. Merge data, store in an allocated buffer // 3. Write merged buffer to temp file fwrite(temp_buff, estimated_size, 1, fp_out); fflush(fp_out); fclose(fp_1); fclose(fp_2); fclose(fp_out); // Now rename temp file to desired file name if(std::rename("file_tmp.dat", "file_out.dat") == 0) { std::remove("file_1.dat"); std::remove("file_2.dat"); } 

I have repeatedly tested the program with two input files of 5 MB each. Once, I suddenly turned off the system by unplugging the power cable. After rebooting the system, I checked the data and found that the input files were deleted and file_out.dat was filled with all zeros. This led me to believe that the system went down right after deleting the two input files, and the output was still somewhere in the buffer of the disk controller. If so, is there a way to check if the data is actually written to disk?

Answer 1

Not in the general case. The disk may lie in the OS, claiming that the recording is finished when it is really just in the queue on the hard disk on board the RAM cache, which will be lost due to sudden power losses.

The best you can do is explicitly ask the OS to say “really really synchronize everything” after you have executed fflush , or a limited area with fsync or using something like sync or syncfs (the first synchronizes all file system, the latter limits the scope to a file system corresponding to a single file descriptor). You want to make the target fsync after the final fflush , but before rename and / or the wider sync / syncfs after rename , but before the remove calls, the data and file system tables are definitely updated before deleting the source files.

Of course, as I said, this is all the best; if the disk controller is on the OS, you can do nothing to write new firmware and drivers for the disk, which is probably too far away.