Why is the local MongoDB database almost completely in memory?

Question

I have a replicated mongodb installation and I see a lot of page errors. So I started researching and discovered (via vmmap ) that almost the entire local database is in memory (that is, part of the working set). The only compilation of significance is, of course, oplog.rs , which is used for replication. Looking at running requests, those on oplog are data that is much closer to the tail than to the head of oplog. So why is it still remembered? Of course, it must be replaced due to a large number of errors.

I do not understand something? Am I reading vmmap information incorrectly? Or is something really wrong?

Please note that this is a test installation, and there are several other mongod instances on this equipment, so the total amount of memory used here does not mongod up to the total amount in the machine. In general, memory usage is ~ 100%.

Answer 1

Mongo delegates page control to the kernel — since it uses mapped files, it simply relies on the kernel to decide what to do on the page. Your local database gets under way every time you write, or get read from another. Oplog is a limited collection, so it will constantly change the fixed space in the data files (and therefore the fixed space in RAM), which should keep it touched and not very high in the list of priorities for uploading.

As for your many page errors, is it possible that this is simple caching? Mongo is not going to load its working set into memory when it has just begun, so it will take a little time to warm up and extract material from and to memory.

Remember to consider caches and buffers - your memory usage can be read at 100%, but the kernel expires cache and buffers before it exits from other things, so it may be that while your system is reporting 100 % usage, a significant part of this is caches and buffers, which will be flushed as needed, which means that the working set of mangoes should never be unloaded at all.

You may be able to test this by running a program designed to consume more and more memory ( like this one ) and see how the mongo behaves as soon as your system actually gets swap.