Redis memory and processor spikes

Question

We use redis for some data in our application, and it is absolutely great. I noticed, however, random CPUs and memory spikes in the redis-server process.

This is the Giraffe toolbar from our production and intermediate environment. The stage is obviously much less busy, but the production is not very busy either normally or ...

This seems to correlate with background preservation, but not with all of them. Only a handful of them create this spike. Perhaps all this happens, and only until the resolution of the measurement (some of them just do not fall into our memory / processor control cycle). I'm not quite sure.

I'm still wondering if this is expected / normal. We are not seeing any problems, but I want to be safe. If we have more traffic / activity in our production, are we likely to see many more such spikes?

UPDATE :

redis log file during splash

 [18588] 05 May 11:42:51.004 * 10 changes in 300 seconds. Saving... [18588] 05 May 11:42:51.258 * Background saving started by pid 32712 [32712] 05 May 11:43:00.511 * DB saved on disk [32712] 05 May 11:43:00.549 * RDB: 1 MB of memory used by copy-on-write [18588] 05 May 11:43:00.629 * Background saving terminated with success 

Answer 1

From a further experiment and reading redis persistence , I think the following observations can be made:

• When using RDB (default settings) redis will be plugged every time the save operation is triggered, which (by default) is set at least every 15 minutes. When more write operations are performed on Redis, RDB writes occur as often as once every 60 seconds.
• Each fork will use the “copy-on-write” memory allocation, which means that while the memory will not actually double - it will appear on instruments like ps , htop , etc.>
• The plug itself can be quite processor intensive, especially on xen virtual hosts (this is what we are currently using).
• The write operation seems to completely overwrite the existing RDB file. It does not only record changes, but rather flushes the entire data set to disk.

So, on a modest virtual host with 4Gb RAM and a data set of about 750 MB (at the time I posted the question), this is starting to get pretty “expensive." We observed CPU / Memory spikes as well as increased IO even with a fairly moderate load / use of redis.

So, to answer my own question - this is similar to the "expected" behavior.

As for improving the situation, we decided to switch our configuration to using a combination of RDB and AOF. AOF (Append Only File) seems to only write changes to disk. You can (and should) configure the AOF file for overwriting (using the settings auto-aof-rewrite-percentage and auto-aof-rewrite-min-size ). Using RDB for snapshots is also recommended. However, in this configuration, you can probably do full dubs / snapshots less often and still maintain good performance and even better durability.

Answer 2

If I remember correctly, redis expands the process when it performs background storage, but only duplicates the memory that changes during the save. Thus, a hit in the CPU / memory will depend on how much of the data will be changed during storage. Therefore, sometimes it can be a huge surge and a much smaller surge in other cases (or nothing at all, depending on how your load looks).

Answer 3

The docs say, "Redis AOF works by phasing updating an existing state like MySQL or MongoDB, while RDB snapshots create everything from scratch again and again, which is conceptually more robust."

Source: http://redis.io/topics/persistence (AOF deficiencies)