Multiple shared memory link lists in Linux

Question

I use C and Linux as a platform. I want to share several structures in several processes. These structures have link list headers (these lists must also be separated) and pointers to each other. The memory required for this data can reach 1 MB. Since I cannot use pointers in shared memory, because they will be invalid for different processes.

There are two options: 1) either use offset values ​​instead of pointers. 2) otherwise use different shared memory and use shared memory identifiers (returned by shmget) instead of pointers.

As the amount of memory that should be shared is huge, which option is better? Can you suggest another option?

Thanks.

Answer 1

Use offset values.

Instead of pointers, use size_t offsets (in characters) from the beginning of the shared memory area. You will need to do this wherever you access or manipulate these lists.

Edited to add:

Using offsets this way compiles for very efficient code on most architectures, and you can use __sync..() built-in modules to access and modify them atomically. Remember to use the built-in for all accesses, including reading: otherwise, the value may be atomically changed during non-atomic reading (or vice versa), which will lead to data corruption.

If you know that your shared memory will never grow above 4 GB, then you can use uint32_t as an offset type, storing four bytes to a “pointer” on 64-bit architectures. If you align all targets with a 32-bit boundary, then you can quadruple it to 8 GB.

An extremely nice side effect of using uint32_t is that you can handle pointer pairs (two consecutive offsets) atomically, on all 64-bit and some 32-bit architectures. Assuming you also have everything in shared memory, also aligned to 32-bit boundaries and using offsets for each 32-bit block, you can get / set pointer pairs atomically using

 static inline void get_pair(void *const base, const uint32_t offset, uint32_t *const pair) { uint64_t *const ptr = (uint64_t *)(offset + (uint32_t *)base); uint64_t val; val = __sync_or_and_fetch(ptr, (uint64_t)0); memcpy(pair, &val, sizeof val); } static inline void switch_pair(void *const base, const uint32_t offset, const uint32_t *const new, uint32_t *const old) { uint64_t *const ptr = (uint64_t *)(offset + (uint32_t *)base); uint64_t oldval, newval; memcpy(newval, &new, sizeof newval); do { /* Note: this access does not need to be atomic, */ memcpy(oldval, ptr, sizeof oldval); /* because the next one verifies it. */ } while (!__sync_bool_compare_and_swap(ptr, oldval, newval)); if (old) memcpy(old, &oldval, sizeof oldval); } 

The __sync...() functions work in GCC and Intel CC at least. The new C11 standard also uses the __atomic..() C ++ 11 built-in modules, but it will take some time to implement the functions implemented in modern compilers.

If you write library code or code that you expect to support for several years, this will probably save you time finding both built-in types and adding comments to yourself (or someone who will support it when it is this time transition between built-in modules) to describe which atomic built-in you use, if they were already available.

Finally, remember that using shared memory like this means that you have to be as careful as if you had multiple threads accessing memory at the same time. Atomic operations will help, and there are some very smart tricks that you can do with lists if you can manipulate pairs of pointers atomically, but you still need to be very sharp about corner cases and possible race conditions.