MeteorJS: why so long in foreach? Should I use MongoDB aggregation operations like?

Question

I have a collection where one document looks like this:

{
    _id: 'string'
    date: Date,
    user: 'usera',
    sections: [
        { heading: 'string a', score: 10 },
        { heading: 'string a', score: 104 },
        { heading: 'string b', score: 123 },
        { heading: 'string b', score: 15 },
        { heading: 'string b', score: 7 },
        { heading: 'string c', score: 1 }
    ]
    totalScore: 259
}

The collection contains hundreds of thousands of documents.

I need to search from the collection by date and give the total points for the sections and the amount totalScores.

In Meteor publications, if I do

Meteor.publish('mydata', function() {
  var rets = {}

  Mycollections.find({date: {$gte: new Date(2016, 8, 1), $lt: new Date (2016, 12, 24)}}.forEach(function (doc) {
     if (!rets[doc.user]) {
       rets[doc.user] = {

         tot: 0, 
         sections: {}
       }
     }

     rets[doc.user].tot += doc.totalScore;
     for (var i in doc.sections) {
        if (!rets[doc.user].sections[doc.sections[i].heading]) {
          sections[doc.sections[i].heading] = 0;
        }

        rets[doc.user].sections[doc.sections[i].heading] += sections[doc.sections[i].score; 
     }
  });

  return rets;

});

Mycollections.find({date: {$gte: new Date(2016, 8, 1), $lt: new Date (2016, 12, 24)}} less than 0 ms running.

For a single forEach loop, it takes about 0-1ms to complete, if I retrieve 20,000 documents, it takes 20 seconds to publish.

How can MongoDB aggregates help here? or how can I do the calculations faster on the server side?

I need the total points and the total points in each section for each user in the collection.

{
  usera: {
   tot: 235
   sections[
    {'string a': 200},
    {'string b': 35},
    {...}
  ]
 }, 
 userb: {
 ...
 }

}

Answer 1

:

• totalScore
• score
• user

:

• $match
• $group totalScore $$ROOT , .
• $unwind $$ROOT json
• $unwind ( )
• $group, user sections
• $group, user

:

db.document.aggregate([{
    $match: {
        date: {
            $gte: ISODate("2016-08-01T00:00:00.0Z"),
            $lte: ISODate("2016-12-24T23:59:00.0Z")
        }
    }
}, {
    $group: {
        _id: {
            "user": "$user"
        },
        totalScore: { $sum: "$totalScore" },
        document: { $push: "$$ROOT" }
    }
}, {
    $unwind: "$document"
}, {
    $unwind: "$document.sections"
}, {
    $group: {
        _id: {
            "user": "$_id.user",
            "sections": "$document.sections.heading"
        },
        score: { $sum: "$document.sections.score" },
        totalScore: { $first: "$totalScore" }
    }
}, {
    $group: {
        _id: "$_id.user",
        sections: { $push: { "name": "$_id.sections", "score": "$score" } },
        totalScore: { $first: "$totalScore" }
    }
}])

- :

{ "_id" : "userc", "sections" : [ { "name" : "string c", "score" : 1 }, { "name" : "string b", "score" : 145 }, { "name" : "string a", "score" : 114 } ], "totalScore" : 259 }
{ "_id" : "usera", "sections" : [ { "name" : "string c", "score" : 12 }, { "name" : "string b", "score" : 1873 }, { "name" : "string a", "score" : 230 } ], "totalScore" : 269 }

Meteor :

MyCollection.aggregate(
    // aggregation query here
);