LINQ Grouping Big Query What's Going Backstage

Question

Take the following LINQ query as an example. Please do not comment on the code itself, as I just typed it to help with this question.

The following LINQ query uses 'group by' and computes the resulting information. As you can see, there are numerous calculations that are performed on the data, but how efficient LINQ is behind the scenes.

var NinjasGrouped = (from ninja in Ninjas group pos by new { pos.NinjaClan, pos.NinjaRank } into con select new NinjaGroupSummary { NinjaClan = con.Key.NinjaClan, NinjaRank = con.Key.NinjaRank, NumberOfShoes = con.Sum(x => x.Shoes), MaxNinjaAge = con.Max(x => x.NinjaAge), MinNinjaAge = con.Min(x => x.NinjaAge), ComplicatedCalculation = con.Sum(x => x.NinjaGrade) != 0 ? con.Sum(x => x.NinjaRedBloodCellCount)/con.Sum(x => x.NinjaDoctorVisits) : 0, ListOfNinjas = con.ToList() }).ToList(); 

• How many times is the ninja list renamed to calculate each of the values?
• Is it faster to use a foreach loop to speed up the execution of such a request?
• Adding ".AsParallel ()" after Ninjas leads to any performance improvements?
• Is there a better way to calculate the summary for List?

Any advice is appreciated since we use this type of code in all of our software, and I would really like to get a better idea of ​​what LINQ does under the hood (so to speak). Perhaps there is a better way?

Answer 1

Assuming this is a LINQ to Objects query:

• Ninjas repeated only once; groups are created in internal specific lists, which are then repeated several times (once for aggregation).
• Using a foreach almost certainly not speed things up - you can benefit from cache coherence a little more (since each time you iterate over a group, you probably have to extract data from a higher level cache or main memory), but I really doubt that it would be important. An increase in pain during its implementation would probably be significant, though :)
• Using AsParallel can speed up the process - it looks pretty easy to parallelize. Worth trying...
• Not better for LINQ for objects, to be honest. It would be nice to be able to perform aggregation as you group, and Reactive Extensions would allow you to do something similar, but for now this is probably the easiest approach.

You might want to see a GroupBy on my GroupBy blog blog for more details on a possible implementation.