Migrating from LinqToXYZ to LinqToObjects

Question

Answering this question , I became interested ...

I often use this template:

collectionofsomestuff //here it LinqToEntities .Select(something=>new{something.Name,something.SomeGuid}) .ToArray() //From here on it LinqToObjects .Select(s=>new SelectListItem() { Text = s.Name, Value = s.SomeGuid.ToString(), Selected = false }) 

Perhaps I would split it into a couple of lines, but essentially, at ToArray , I actually enumerate my query and save the resulting sequence so that I can continue to process it with all kindness full CLR.

Since I am not interested in any kind of manipulation of the staging list, I use ToArray over ToList , since there is less overhead.

I do this all the time, but I wonder if there is a better pattern for this kind of problem?

Answer 1

There is a much better option: AsEnumerable

Similar usage:

 collectionofsomestuff //here it LinqToEntities .Select(something=>new{something.Name,something.SomeGuid}) .AsEnumerable() //From here on it LinqToObjects .Select(s=>new SelectListItem() { Text = s.Name, Value = s.SomeGuid.ToString(), Selected = false }) 

This, however, does not cause the full copy to be made as ToList() or ToArray() and saves any delayed execution from your provider.

Answer 2

The read response is indeed correct if you are performing simple assignments in the remainder of the LINQ query. However, if you are doing significant work or calculation in the LinqToObjects section of your query, its solution has a small problem if you are considering connecting to the underlying data source:

Consider:

 collectionofsomestuff //here it LinqToEntities .Select(something=>new{something.Name,something.SomeGuid}) .AsEnumerable() //From here on it LinqToObjects .Select(s=>new SelectListItem() { Text = s.Name, Value = s.SomeGuid.ToString(), OtherValue = someCrazyComputationOnS(s) }) 

If you can imagine the second code for the LinqToEntities selection function (very simplified, but you should get an image), it might look something like this:

 using(SqlConnection con = createConnection()) { using(SqlCommand com = con.CreateCommand()) { con.Open(); com.CommandText = createQuery(expression); using(SqlDataReader reader = com.ExecuteReader()) { while(reader.Read()) { yield return createClrObjectFromReader(reader); } } } } 

This method supports traditional Linq deferred execution patterns. This means that whenever the result is read from the reader, it will be “returned” back to the caller, and the next value will not be read until the caller requests it.

So, in the above code, the execution sequence for the result set of 5 entries will be:

 con.Open(); reader.Read(); createClrObjectFromReader(reader); // at this point there is a yield back to the caller someCrazyComputationOnS(s); reader.Read(); createClrObjectFromReader(reader); // at this point there is a yield back to the caller someCrazyComputationOnS(s); reader.Read(); createClrObjectFromReader(reader); // at this point there is a yield back to the caller someCrazyComputationOnS(s); reader.Read(); createClrObjectFromReader(reader); // at this point there is a yield back to the caller someCrazyComputationOnS(s); reader.Read(); createClrObjectFromReader(reader); // at this point there is a yield back to the caller someCrazyComputationOnS(s); // ONLY here does the connection finally get closed: con.Close(); 

Although this retains the deferred execution pattern. In this situation, this is not optimal. Calling ToList () or ToArray () will buffer all unprocessed requests into an array or list, after which SqlConnection can be closed. Only after closing SqlConnection would someCrazyComputOnS (s) be called.

In most cases, this is not a problem, and Reed's answer is really correct, but in the rare case you do a lot of work on your dataset, you definitely want to buffer the results before proceeding with large LinqToObjects queries.