Linux kernel, iptables and vmalloc size

Question

We had problems with iptables on several of our Linux boxes, where it seems that the number of loaded shift rules causes a vmalloc error (the vmap distribution for size 3506176 failed: use vmalloc = to increase the size.) To appear in dmesg and any additional rules stop loading.

After much research, we increased the size of vmalloc from 128 MB to 512 MB and rebooted, and this temporarily fixed the problem. It seems that the 64-bit kernel does not have this problem (?). I checked my CentOS 6 block (64 bits) and it has VmallocTotal: 34,359,738,367 kB (!).

So my question is: can it solve the problem with the 32-bit PAE core? It would be much easier to change kernels than OS on multiple sites ...

Thanks Jak

Answer 1

The 32-bit PAE kernel will not solve this problem because the problem is related to fragmentation of vmalloc space allocation. On x86-64, the vmalloc space is very large (much larger than the physical size of the RAM), so you will not find yourself in a situation where it is fragmented enough for distribution failures to occur. However, in a 32-bit space, vmalloc is much smaller - a hundred MB. Moving to PAE does not make this virtual hosting space anymore.

The workaround for your problem, if you want to stay in the 32-bit version, is to change the kernel so that iptables allocates vmalloc from the preallocated space, thus avoiding fragmentation caused by other callers for vmalloc (although there is no guarantees that this will solve your problem perfectly, since it depends on how iptables allocates memory in relation to what you are doing with it, which is unknown in this area of ​​questions).

Answer 2

You can use a 64-bit kernel with 32-bit user space - this will give you the benefits of the huge kernel-changing vmalloc arena.