Limit Limit for Memcached

Question

Why is there a hard code restriction (.5 megagram after compression) in memcached ? Has anyone recompiled them? I know that I should not send such large pieces, but these extra heavy pieces happen to me from time to time and cause damage.

Answer 1

This question has been used in the official FAQ.

What are some limitations in memcached that I could apply? (Wayback Machine)

Quote:

The simple restrictions that you are likely to see with memcache are the key and element size restrictions. Keys are limited to 250 characters. The stored data cannot exceed 1 megabyte, as this is the largest typical plate size. "

The FAQ is now revised, and now there are two separate questions:

What is the maxiumum key length? (250 bytes)

The maximum key size is 250 characters. Please note that this value will be less if you use client "prefixes" or similar functions, since the prefix is ​​attached to the front of the original key. Shorter keys are generally better because they save memory and use less bandwidth.

Why are items limited to 1 megabyte?

Ahh, this is a popular question!

Short answer: due to how the memory allocation algorithm works.

Long answer: Memcached's storage engine (which will be pluggable / revised in the future ...), uses a memory overlap management approach. The memory is divided into pieces of plates of different sizes, starting from the minimum number and increasing on the factorial to the highest possible value.

Let's say that the minimum value is 400 bytes, and the maximum value is 1 megabyte, and the factorial is 1.20:

plate 1 - 400 bytes plate 2 - 480 bytes plate 3 - 576 bytes ... etc.

The larger the plate, the greater the gap between it and the previous plate. Thus, the higher the maximum value, the lower the memory memory efficiency. Memcached also needs to preallocate some memory for each slab that exists, so installing a smaller factorial with a larger maximum value will require even more overhead.

There is another reason why you would not want to do this ... If we are talking about a web page and you are trying to save / load the values, you are probably doing something wrong. With this size, it will take a noticeable amount of time to load and unpack the data structure in memory, and your site will most likely not work very well.

If you really want to store items larger than 1 MB, you can recompile memcached with the edited value slabs.c:POWER_BLOCK or use the inefficient malloc / free backend. Other suggestions include databases, MogileFS, etc.