Increase maximum virtual memory above 256 GB

Question

I am running a program that allocates 8 MB stacks using mmap . During testing, in order to find out how many stacks I can allocate (targeting 100,000), I see that the size of virtual memory is growing rapidly as expected, and the reserved size remains small (less than 1 GB). Then the program shuts down with Cannot allocate new fiber stack: Cannot allocate memory (Errno). Using gdbto save segfault, and then looking at htop, I found that this happens in approximately 256 GB of virtual memory.

I tried to use it prlimit --as=unlimited --rss=unlimited --memlock=unlimited --data=unlimitedat program startup, but it doesn't seem to matter.

Is there any way to increase this limit? Is it advisable to increase this limit? Is there a better way for a crystal to distribute stacks?

Answer 1

You may have reached the maximum /proc/sys/vm/max_map_count. This parameter sets the maximum size of mmaps your process can have. The default value is 65536. Therefore, most likely, this is not the size of the memory you want malloc, but the number of malloc calls that cause an error Cannot allocate memory.

You can increase the maximum value with:

sysctl -w vm.max_map_count=131070

See also NPTL blocks maximum flows at 65528?

Answer 2

I would check your swap file size. if you are running out of exchanges, all of these parameter changes will not help you until you fix it.

-h, , . , .