Preliminary receipt of files in the cache of the Windows page

Question

I have a program written in C that allows the user to scroll through the display of about a million small files. Each file must undergo some processing (read-only) before it is displayed to the user. I implemented a buffer that preprocesses files in a certain radius around the user's position, so if they work linearly through them, there is not much delay. For various reasons, I can only actually run my processing algorithm one file at a time (although I can open several files and I can read them), so my buffer is loaded sequentially. My processing algorithms are as optimized as they are going to get, but I ran into I / O problems. At first, my download process is slow, but when the files were available several times,it accelerates by about 5 times. Therefore, I strongly suspect that something that slows me down is waiting for the Windows cache page to pull my files into memory. I know very little about this. If I could ensure that my files were cached before they needed my processing algorithm, I would be in business.

My question is: is there a way to convince / cajole / trick / to intimidate Windows into loading my files into the page cache before I really get around to reading / processing them?

Answer 1

There is only one way to get a file into the file system cache: reading it. This is a problem with chicken and egg. First you can get an egg using an auxiliary stream that reads files. It should have had some skill about which file is likely to be next. And don't read too much.

Answer 2

On a POSIX system, you should use posix_fadvise :

POSIX_FADV_WILLNEED

        Indicates that the application is expecting access to the specified data in the near future.

Windows. fadvise/madvise ? - .