Sort and restrict MongoDB queries by insert order in a multiprocessor environment

Question

I use MongoDB in a multiprocessor environment, and I was wondering how I can sort requests by insertion order and restrict it to documents inserted after a specific document. In one process I can use ObjectID, but otherwise two ObjectIds from different processes in the same second may have the wrong order.

Example:

ObjectId("5236dc5c 88ee6f 2075 bd0049") 

can be generated by process 2075 before

  ObjectId("5236dc5c 88ee6f 2071 f35fb8") 

process 2071. Note that the timestamp of both identifiers is (5236dc5c). This timestamp is indicated in seconds.

Answer 1

A unique automatic incremental sequence field is required to ensure insertion order. There are two recommended methods for this:

• Collection of counters
• Optimistic cycle

It should be noted that the documentation does not indicate: only Optimistic Loop really guarantees that the insertion order is equal to the sequence order. However, this is very likely for the Counters Collection approach, but the theory is not guaranteed by theory, since increment and insertion of a document are two separate operations.

Answer 2

Using ObjectIds or a date field to sort may not give you the results you are looking for. ObjectIds and dates in pasted documents are created on the client side, so if you work with connections from multiple computers, you will tidy up the inconsistencies if the time between your machines is not perfect.

Can you provide more details on what you are trying to do? There are several different ways to get the behavior you want from MongoDB, depending on why you need a list of documents inserted after a specific document.

If, for example, you try to use this ordered list of documents as a kind of queue, you can instead use findAndModify to get an unread document and atomically update the read field to ensure that you don't read it twice. Each call to findAndModify would find the very last document in the collection without a read field set to true, atomically set this field to true and return your document to the client for processing.

On the other hand, if your use case does require a list of documents in the inserted order, you can use the natural order of the inserted documents. In MongoDB, documents are written to disk in insertion order if changes to the size or deletion of the document do not require moving things. Using a limited collection to ensure you maintain a natural order, you can get a list of documents using this. Please note that there are several basic limitations associated with using private collections, which you can find in the documentation .