How to ensure malloc and mmap compatibility, i.e. Work with non overlapping memory areas?

Question

My main problem is that I need to enable several OS processes to exchange data through a large heap of shared memory, which maps to the same address ranges in all processes. (To make sure the pointer values ​​are really significant.)

Now I am having problems with the fact that part of the program / library uses standard malloc / free, and it seems to me that the base implementation does not take into account the mappings that I create using mmap. Or, another option is that I create mappings in regions that malloc has already planned to use.

Unfortunately, I cannot guarantee 100% identical malloc / free behavior in all processes before setting up mmap mappings.

This leads me to have the MAP_FIXED flag in mmap. The first process uses 0x0 as the base address to ensure that the display range is at least somehow reasonable, but doesn't seem to be shared with other processes. (The binary is also associated with -Wl, -no_pie.)

I tried to find out if I could query the system to find out which pages it plans to use for malloc by reading malloc_default_zone , but this API does not seem to offer what I need.

Is there a way to ensure that malloc does not use specific memory pages / address ranges?

(It should work on OSX. Linux tips that guide me in the right direction are also appreciated.)

Answer 1

I notice this in the mmap documentation :

If MAP_FIXED is specified, a successful mmap removes any previous mapping in the selected address range

However, malloc will not use a fixed card, so as long as you get to malloc you will be fine: you can check if the region is free by first trying to match it without MAP_FIXED , and if it succeeds with the same address (which it will do if the address is free), you can reassign using MAP_FIXED , knowing that you do not select the section of the address space that malloc has already taken

The only guaranteed way to guarantee that the same block of logical memory will be available in two processes is to have one plug from the other.

However, if you are compiling 64-bit pointers, you can simply select the (unusual) memory area and hope for the best, since the chance of a collision is tiny.

See also this question on valid address spaces.

Answer 2

The OpenSD implementation of malloc () uses mmap () to allocate memory. I suggest you look at how this works, and then write your own malloc () implementation of your own and tell your program and the libraries it uses to use your own malloc () implementation.

Here is the OpenBSD malloc ():

http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/lib/libc/stdlib/malloc.c?rev=1.140

RBA