Check .mat file exists and is not damaged - Matlab

Question

I have 2 independent Matlab workers, with FIRST receiving / saving data and SECOND reading (and doing some calculations, etc.).

FIRST saves the data as a .mat file on the hard drive, and SECOND reads it from there. It takes ~ 20 seconds for SAVE use this data as .mat and 8millisec before DELETE . Before saving data, FIRST deletes the old file and then saves the newer version.

How can SECOND check if data exists and is not corrupt ? I can use exists , but that does not tell me if the data is corrupted or not. For example, if SECOND tries to read data accurately, when FIRST saves it, exists passes, but LOAD gives you an error message - Data Corrupt, etc.

Thanks.

Answer 1

You cannot, without some kind of synchronization mechanism, by the time SECOND finishes checking and starts reading the file, FIRST may have started writing it again. You need some kind of lock or mutex.

Two variants of the base Matlab.

If it is a local file system, you can use a separate lock file located next to the data file to control concurrent access to the data file. Use the Java NIO FileChannel and FileLock objects from inside Matlab to lock the first byte of the lock file and use it as a semaphore to control access to the data file, so the reader waits until the author finishes and vice versa. (If this is on a network file system, do not try to do this - file locking may seem to work, but it is usually not officially supported and in my experience is unreliable.)

Or you can simply put try / catch around your load() call and pause it for a few seconds and try again if you get a damaged file error. The .mat file format is such that you won’t get a partial read if the writer is still writing it; You will get this damaged file error. So you can use this as a lazy look for collision detection and snooze. This is what I usually do.

To reduce the contention window, consider having FIRST write a temporary file in the same directory, and then use renaming to move it to its final destination. Thus, the file will be unavailable during the quick operation of moving the file system, and not 20 seconds of data recording. If you have multiple authors, insert the PID and hostname in the temp file name to avoid conflicts.

Answer 2

Sounds like a classic resource sharing problem between two threads (RW)

In short, you should find a method of secure communication between employees. Check it out.

Also try typing

showdemo ('paralleldemo_communic_prof')

in matlab