Reading a file when it is being written by another process

Question

I am trying to read binary data from a buffer file that is constantly being written by another process (which I cannot change). I use the following code to open a file:

fileH = CreateFileA((LPCSTR)filename, GENERIC_READ, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL); 

And it opens correctly, without errors. However, when I read data from a file, it seems that it blocks another process from writing to the file, since I lost the data.

The buffer is round, which means that the file size is fixed, and new data is constantly written over the old data in the buffer.

EDIT: Sometimes the most trivial solution works ...

I contacted the software company and told them about the error, and within a day they published a new version with a fix. Sorry, this may not work for everyone.

Answer 1

It's hard to say what your options are, not knowing how the writing process opens the file. Obviously, it does not open the file for exclusive access and keeps it open. Otherwise, you cannot read it at all.

The behavior you describe indicates that the writing process opens the file for exclusive access, writes to it, and then closes the file. If this happens, you will not be able to open your program and save it. This can cause the write process to crash whenever it tries to write.

If you cannot change the recording process, your options are limited and not very attractive. Most likely, you will have to force your program to open the file, read a small piece, close the file and wait a bit before reading it again. Even then, there is no guarantee that you will not open the file when the recording process tries to write. Which, I think, you have already discovered.

Did you know that the write process loses data when it cannot open the file, or if it just buffers the data and writes it the next time it really can open the file? If so, then my suggestion of going through the file a bit might work. Otherwise, you will lose data.

There is no open mode, which I know about, it is the equivalent of "Open the file for reading, but if someone wants to get exclusive access, then you must have it."

Another possibility would be for your program to rename the file whenever you want to read it, and then delete the renamed file after reading it. This assumes, of course, that the recording process will create a new file if necessary. Even then, a problem may arise if the recording process tries to write during renaming. I don’t think this will be a problem (renaming may be atomic with respect to the file system), but this is what you would need to investigate.

Answer 2

I would recommend looking at the source code for the excellent Far Manager . Its internal viewer can easily process files with several gigabytes, has no problems displaying files that are being written, and can update the changed contents of a file almost in real time. I never noticed any problems with blocking the displayed files.

The source code related to the question seems to be in the viewer.cpp file.

Interestingly, it does not use GENERIC_READ :

 ViewFile.Open(strFileName, FILE_READ_DATA, FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE, nullptr, OPEN_EXISTING); 

I suspect the fall of SYNCHRONIZE may be important here.

File change detection is located in Viewer::ProcessKey , KEY_IDLE case:

 // Smart file change check -- thanks Dzirt2005 // bool changed = ( ViewFindData.ftLastWriteTime.dwLowDateTime!=NewViewFindData.ftLastWriteTime.dwLowDateTime || ViewFindData.ftLastWriteTime.dwHighDateTime!=NewViewFindData.ftLastWriteTime.dwHighDateTime || ViewFindData.nFileSize != NewViewFindData.nFileSize ); if ( changed ) ViewFindData = NewViewFindData; else { if ( !ViewFile.GetSize(NewViewFindData.nFileSize) || FileSize == static_cast<__int64>(NewViewFindData.nFileSize) ) return TRUE; changed = FileSize > static_cast<__int64>(NewViewFindData.nFileSize); // true if file shrank } 

Reading a cached file is implemented in cache.cpp . But there is nothing destructive in the world, only some Seek() and Read() (which ultimately results in SetFilePointerEx and ReadFile API calls). OVERLAPPED is not used.