When you mmap file, you basically exchange memory directly between your process and the kernel page cache - the same cache that stores the file data that was read from disk, or waits to be written to disk. A page in the page cache that is different from what is on the disk (since it was recorded) is called "dirty."
There is a kernel thread that scans dirty pages and writes them back to disk under the control of several parameters. The most important is dirty_expire_centisecs . If any of the pages for the file has been dirty longer than dirty_expire_centisecs , then all dirty pages for this file will be written. The default value is 3000 centimeters (30 seconds).
Another set of variables: dirty_writeback_centisecs , dirty_background_ratio and dirty_ratio . dirty_writeback_centisecs determines how often the kernel thread checks for dirty pages, and the default is 500 (5 seconds). If the percentage of dirty pages (as part of the memory available for caching) is less than dirty_background_ratio , then nothing happens; if it is larger than dirty_background_ratio , then the kernel will start writing some pages to disk. Finally, if the percentage of dirty pages exceeds dirty_ratio , then any processes trying to write will be blocked until the amount of dirty data decreases. This ensures that the amount of unwritten data cannot increase without limitation; ultimately, processes that produce data faster than a disc can write will have to slow down to match the speed of the disc.
The question of how mtime is updated is related to the question of how the kernel knows that the page is dirty first. In the case of mmap answer is that the kernel sets read-only display pages. This does not mean that you cannot write them, but it does mean that the first time you throw an exception in the memory management unit that is processed by the kernel. The exception handler does (at least) four things:
- Marks the page as dirty so that it will be recorded.
- Updates the mtime file.
- Marks the page as read-write, so the write may be successful.
- Goes to the instruction in your program that writes to the
mmap ed page, which runs this time.
Therefore, when you write data to a blank page, this leads to an mtime update, but also causes the page to become a write-record, so further recording does not throw an exception (or an mtime update) Note 1 . However, when a dirty page is deleted to disk, it becomes clean and read-only again, so any subsequent write to it causes another write to the destination disk, as well as another mtime update.
So now, with a few assumptions, we can start collecting puzzles.
First off, dirty_background_ratio and dirty_ratio probably don't come into play. If the pace of your recordings was fast enough to trigger background flushes, then most likely you will see the “wrong” behavior in all files.
Secondly, the difference between “irregular” files and “30 seconds” files is a page access pattern. I assume that "irregular" files are written in some kind of append-mode or round-buffer, so you start writing to a new page every few seconds. Each time you contaminate a previously untouched page, it starts the mtime update. But for files displaying a 30-second pattern, you only write one page (perhaps it is one page or less in length). In this case, mtime is updated on the first record, and then not again, until the file is dirty_expire_centisecs to disk, exceeding dirty_expire_centisecs , which is 30 seconds.
Note 1: This behavior is technically incorrect. This is unpredictable, but standards allow some degree of unpredictability. But they require that mtime should be at the end or after the last write to the file, and before or after msync (if any). In the case when a page is written several times over the interval before it turns red on disk, this is not what happens - mtime gets the label of the first record. This was discussed, but the patch that fixed it was not accepted. Therefore, when using mmap mtimes may be erroneous. dirty_expire_centisecs kind of restriction, but only partially, since other traffic on the disk can cause the flash to wait, expanding the recording window, to bypass mtime even further.