How MongoDB Magazine Works

Here is my view, and I'm not sure if this is correct or not:

A journaling journal is a retry journal. It records data file changes.

For example, I want to change the field value of a single record from 'a' to 'b', then mongodb will find how to change the dbfile file (including the entire namespace, data, index, etc.), then mongodb writes the changes to the log.

After that, mongodb makes all the real changes to the dbfile. If something goes wrong when mongoDB restarts, it will read the log (if it exists). Then it will modify the dbfile to make the dataset consistent.

So, the data for the change is not recorded in the log, but instead how to change the dbfile file.

I'm right? Where can I get more information about the magazine format?

+6
source share
1 answer

EDIT: My original link to Dwight’s 2011 MongoSF presentation is now dead, but there is a 2012 Ben Becker presentation with similar content.

Just in case, which will stop working at some point, I will give a brief description of how the journal works in the original MMAP storage engine, but it should be noted that with the advent of the storage plug-in (MongoDB 3.0 and later), now depends entirely on the storage engine (and possibly the parameters) that you use, so please check.

Return to the original memory engine log (MMAP). At a very rudimentary level, the journal contains a series of operations in the queue, and all operations are written to it as they arise - basically, only sequential write to disk is added.

After these operations have been applied and flushed to disk, they are no longer needed in the log and may be outdated. In this sense, the log acts mainly as a circular buffer for write operations.

Internally, operations in the journal are stored in “commit groups” —a logical group of write operations. Once the operation is in the full commit group, it can be considered synchronized with the disk as part of the log (and will satisfy j:true write, for example, a question). After an unclean shutdown, mongod will try to apply all full commit groups that were not previously mongod to disk, incomplete commit groups will be discarded.

The operations in the log are not what you see in the oplog , but rather they are a simpler set of files, offsets (essentially disk space) and data that should be written to this location. This allows you to efficiently reproduce data for a compact format for the magazine, but will make the content more like nonsense in most cases (unlike the aforementioned dictionary, which is mainly read as JSON documents). This basically answers one of the questions posed - he has no idea about the contents of the database file and the changes that need to be made to it, it's even simpler - he basically knows that you need to go to the X place of the disk and write the Y data , what he.

Recording, the sequential nature of the log means it fits well on a spinning disk, and the sequential access pattern is usually not consistent with the MMAP data access patterns (although these are not necessarily access patterns of other engines). Therefore, it is sometimes recommended that you place the journal on your disk or partition to reduce IO competition.

+6
source

Source: https://habr.com/ru/post/913311/


All Articles