Compare mongo diff to two collections

Question

I have two mongo collections, one of which relates to production env, and the other to testing env. How to compare the difference between the two of them? I tried resetting them to bson and then switched to json. But I can't just do a simple diff, because the sorting may be different, and the json file is too big to sort.

Answer 1

Try the following in a shell, it will iterate over each item in the collection and try to match each document based on the identifier.

Let's say we have 2 collections of db.col1 and db.col2 :

 > db.col1.find() { "_id" : 1, "item" : 1 } { "_id" : 2, "item" : 2 } { "_id" : 3, "item" : 3 } { "_id" : 4, "item" : 4 } > db.col2.find() { "_id" : 1, "item" : 1 } { "_id" : 2, "item" : 2 } { "_id" : 3, "item" : 3 } { "_id" : 4, "item" : 4 } 

Then we can create a javascript function to compare the two collections.

 function compareCollection(col1, col2){ if(col1.count() !== col2.count()){ return false; } var same = true; var compared = col1.find().forEach(function(doc1){ var doc2 = col2.findOne({_id: doc1._id}); same = same && JSON.stringify(doc1)==JSON.stringify(doc2); }); return same; } 

Then the call looks like this:

 > compareCollection(db.col1, db.col2) true 

If we then have the 3rd db.col3 collection

 > db.col3.find() { "_id" : 1, "item" : 1 } 

And compare this

 > compareCollection(db.col1, db.col3) false 

we will get the expected result.

If we also have a fourth set that has the relevant documents, but different data db.col4

 > db.col4.find() { "_id" : 1, "item" : 10 } { "_id" : 2, "item" : 2 } { "_id" : 3, "item" : 3 } { "_id" : 4, "item" : 4 } 

This will also return false

 > compareCollection(db.col1, db.col4) false 

Answer 2

dbHash did the trick:

 use db_name db.runCommand('dbHash') 

It returns hash values ​​for each collection. Then you can compare them. This is pretty accurate.

Answer 3

If you need to compare only a subset of fields (for example, you do not need to compare identifiers), you can do this as follows. Export collections to csv, specifying fields for comparison ( source ):

 mongoexport -d <db_name> -c <col_name> --fields "field1,field2" --type=csv | sort > export.csv 

And then make a simple diff in the csv files. Note that the order of the columns in the csv file matches the --field option.

Pros:

• You can specify a subset of fields to compare.
• You can see the actual scatter of records.

Minuses:

• To compare complete records, you need to know all the possible fields.
• mongoexport can be slow for huge databases.

To get all fields in all documents in a collection, see this answer .

Answer 4

use Studio 3T to compare mongodb. You can compare collections, db, single records. you just need to download and connect mongo. here is the download link https://studio3t.com/