Allocation of "temporary" memory (on Linux)

Question

I am trying to find any system functionality that would allow a process to allocate temporary memory, that is, memory that is considered discarded by the process and can be removed by the system when memory is required, but allowing it to benefit from available memory when possible. In other words, a process tells the system that it is ready to sacrifice a block of memory when the process is not using it. Releasing a block is also preferable for replacing it (more expensive or expensive, to replace it and then re-compile its contents).

Systems (like Linux) have things in the kernel like the F / S memory cache. I am looking for something like this, but accessible to user space.

I understand that there are ways to do this from a program, but it is really a great job for the kernel to handle this. To some extent, I ask the kernel:

• if you need to reduce my or another place of residence, first delete these temporary pages.
• If you are shooting these temporary pages, do not change them, just cancel them.

In particular, I'm interested in a solution that will work on Linux, but it would be interesting to know if there are any other O / S.

UPDATE

An example of how I expect this to work:

• display page (swap). There is no difference with what is available right now.
• tell the kernel that the page is "temporary" (due to the lack of a better name), which means that if this page leaves, I do not want it to be loaded.
• tell the kernel that I need a temporary back page. If the page was not displayed, since I designated it as “temporary”, I was told that this happened. If this does not happen, it starts behaving like a regular page.

Here are the problems associated with an existing MM:

To prevent pages from loading, I have to distribute them throughout. But then they can be pulled out at any time, without warning. Testing with mincore () does not guarantee that the page will still be at the time mincore () is complete. Using mlock () requires elevated privileges.

So, the closest I can do this is using mlock () and anonymous pages. Following the expectations that I set out earlier, it will be:

• display anonymous blocked page. (MAP_ANON | MAP_LOCKED | MAP_NORESERVE). Print a page using magic.
• to create the page "temporarily", unlock the page
• if necessary, block the page again. If there is magic, this is my data, otherwise it was lost, and I need to restore it.

However, I do not need the pages to be locked in RAM when I use them. In addition, MAP_NORESERVE is problematic if the memory is overloaded.

Answer 1

This is what the VmWare ESXi , known as the Virtual Machine Monitor (VMM) layer, implements. It is used in virtual machines and is a reclaim memory reclaim from virtual machine guests. Virtual machines that have more allocated memory than they actually use / require release / free it up to VMM so that it can assign it back to those who need it.

This Memory Reclamation technique is mentioned in this article: http://www.vmware.com/files/pdf/mem_mgmt_perf_vsphere5.pdf

In similar lines, something similar that you can implement in your kernel.

Answer 2

I am not sure to accurately understand your needs. Remember that processes work in virtual memory (their address space is virtual), that kernel deals with virtual ones for translating a physical address (using the MMU ) and paging . Therefore, a page error can occur at any time. The kernel will choose to enter or exit the page at arbitrary moments - and will choose which page to change (only the kernel will take care of RAM, and it can display any page of physical RAM as desired). Perhaps you want the kernel to tell you when the page is actually dropped. How will the kernel remove temporary memory from your process without notifying your process? The kernel can take away and then return some RAM ... (so you want to know when this back memory is fresh)

You can use mmap (2) with MAP_NORESERVE first, then again (in the same memory range) with MAP_FIXED|MAP_PRIVATE . See also mincore (2) and mlock (2)

You can also use madvise (2) with MADV_WONTNEED or MADV_WILLNEED etc.

Perhaps you want mmap some device, for example /dev/null , /dev/full , /dev/zero or (more likely) to write your own kernel module that provides a similar device.

GNU Hurd has an external pager mechanism . You still cannot get exactly that on Linux. (Perhaps consider mmap on some FUSE file.)

^{I don’t understand what you want when the kernel spits out its memory, and what you want when the kernel re-looks at such a page because your process is accessing it. Do you want a zero resolution page or SIGSEGV ?}