What is a good way to test using msync for latest Linux kernels?

Question

I use msync in my Linux 2.6 application to ensure consistency in the event of a failure. I need to thoroughly test my use of msync, but the implementation seems to clear all relevant pages for me. Is there a way to prevent the automatic removal of mmap'd pages to disk to detect erroneous use of msync on my part?

Answer 1

We apologize for @samold, "swappiness" has nothing to do with it. Swappiness only affects how the kernel trades when replacing dirty anonymous pages and crowding out the page cache when memory is low.

You need to play with Linux servers managing the pdflush task . To start, I would suggest:

sysctl -w vm.dirty_writeback_centisecs=360000 

By default, vm.dirty_writeback_centisecs is 3000, which means that the kernel will consider any dirty page older than 30 seconds "too old" and try to clear it to disk. Scrolling it up to 1 hour, you should avoid washing dirty pages to disk generally, at least during a short test. With the exception of...

 sysctl -w vm.dirty_background_ratio=80 

By default, vm.dirty_background_ratio is 10, as is 10 percent. This means that when more than 10 percent of the physical memory is occupied by dirty pages, the kernel will think that it needs to take care of something on the disk, even if it is younger than dirty_writeback_centisecs . Raise this key to 80 or 90, and the kernel should agree to transfer most of the RAM occupied by dirty pages. (I would not put this too high, although since I'm sure no one ever does this, and this can cause strange behavior.) Except ...

 sysctl -w vm.dirty_ratio=90 

By default, vm.dirty_ratio is 40, which means that when 40% of the RAM is dirty pages, processes trying to create more dirty pages are blocked until something is evicted. Always do this more than dirty_background_ratio . Hmm, think about it, put it in front of it, just to make sure it's always more.

What is it for my initial suggestions. Perhaps your kernel will start to paginate pages anyway; Linux VM is a mysterious animal and seems to need to be tuned in every release. Hope this will be the starting point.

See Documentation / sysctl / vm.txt in kernel sources for a complete list of tunable virtual machines. (Preferably refer to the documentation for the kernel version that you are actually using.)

Finally, use the / proc / PID / pagemap interface to see which pages are actually dirty at any time.

Answer 2

A few hunches:

You can play with the swappiness system using /proc/sys/vm/swappiness tunable:

  /proc/sys/vm/swappiness The value in this file controls how aggressively the kernel will swap memory pages. Higher values increase agressiveness, lower values descrease aggressiveness. The default value is 60. 

(Wow. proc(5) needs to be run through spell checking.)

If setting swappiness to 0 does not do the trick, there are more customizable controls; the file Documentation/laptops/laptop-mode.txt contains a good description of the behavior of the laptop_mode script:

 To increase the effectiveness of the laptop_mode strategy, the laptop_mode control script increases dirty_expire_centisecs and dirty_writeback_centisecs in /proc/sys/vm to about 10 minutes (by default), which means that pages that are dirtied are not forced to be written to disk as often. The control script also changes the dirty background ratio, so that background writeback of dirty pages is not done anymore. Combined with a higher commit value (also 10 minutes) for ext3 or ReiserFS filesystems (also done automatically by the control script), this results in concentration of disk activity in a small time interval which occurs only once every 10 minutes, or whenever the disk is forced to spin up by a cache miss. The disk can then be spun down in the periods of inactivity. 

You might want to take these numbers to the extreme; if you’re really interested in learning about the behavior of your application, it’s wise to set these values ​​high enough and see how long the sync(1) command takes to execute when all this is done. But this is a system-wide drag and drop - other applications may not be so happy.