Mongo Distinct Query with full row object

Question

I'm primarily new to mongo, so I don’t know much, and I can’t just remove duplicate lines due to some dependencies.

I have the following data stored in mongo

{'id': 1, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'}, {'id': 2, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'}, {'id': 3, 'key': 'pehnvosjijipehnvosjijipehnvosjijipehnvosjijipehnvosjiji', 'name': 'some name', 'country': 'IN'}, {'id': 4, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'}, {'id': 5, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'} 

you can see that some of the lines are duplicated with a different identifier, as long as it is needed to solve this input problem, I have to solve it in the output.

I need the data as follows:

 {'id': 1, 'key': 'qscderftgbvqscderftgbvqscderftgbvqscderftgbvqscderftgbv', 'name': 'some name', 'country': 'US'}, {'id': 3, 'key': 'pehnvosjijipehnvosjijipehnvosjijipehnvosjijipehnvosjiji', 'name': 'some name', 'country': 'IN'}, {'id': 4, 'key': 'pfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnewpfvvjwovnew', 'name': 'some name', 'country': 'IN'} 

My request

 keys = db.collection.distinct('key', {}) all_data = db.collection.find({'key': {$in: keys}}) 

As you can see, one query requires two queries. Combine it with one because the database is very large

I could also create a unique key in key , but the value will be so long (152 characters) that it will not help me.

Or will it be?

Answer 1

To do this, you need to use the aggregation structure. There are several ways to do this, the solution below uses the $$ROOT variable to get the first document for each group:

 db.data.aggregate([{ "$sort": { "_id": 1 } }, { "$group": { "_id": "$key", "first": { "$first": "$$ROOT" } } }, { "$project": { "_id": 0, "id":"$first.id", "key":"$first.key", "name":"$first.name", "country":"$first.country" } }])