How is a streaming operator different from deferred execution?

Question

LINQ Where runs a stream statement. Where - like OrderByDescending - an operator without streaming. AFAIK, the streaming operator only collects the next item that is needed. An operator without streaming immediately processes the entire data stream.

I do not see the relevance of the definition of Streaming Operator. For me, this is redundant with delayed execution. Take an example where I wrote a custom extension and used it using the where and orderby operator.

 public static class ExtensionStuff { public static IEnumerable<int> Where(this IEnumerable<int> sequence, Func<int, bool> predicate) { foreach (int i in sequence) { if (predicate(i)) { yield return i; } } } } public static void Main() { TestLinq3(); } private static void TestLinq3() { int[] items = { 1, 2, 3,4 }; var selected = items.Where(i => i < 3) .OrderByDescending(i => i); Write(selected); } private static void Write(IEnumerable<int> selected) { foreach(var i in selected) Console.WriteLine(i); } 

In any case, Where it is necessary to evaluate each element in order to determine which elements correspond to this condition. The fact that it gives seems to be relevant only because the operator receives deferred execution.

So what is the importance of stream operators?

Answer 1

There are two aspects: speed and memory.

The speed aspect becomes more apparent if you use a method like .Take() to consume only part of the original result set.

 // Consumes ten elements, yields 5 results. Enumerable.Range(1, 1000000).Where(i => i % 2 == 0) .Take(5) .ToList(); // Consumes one million elements, yields 5 results. Enumerable.Range(1, 1000000).Where(i => i % 2 == 0) .OrderByDescending(i => i) .Take(5) .ToList(); 

Since the first example uses only stream operators before calling Take , you end up outputting values ​​from 1 to 10 before Take stops evaluating. In addition, only one value is loaded into memory at a time, so you have a very small amount of memory.

In the second example, OrderByDescending not translated, so the Take pull moment of the first element, the entire result that passed through the Where filter, must be placed in memory for sorting. This can take a lot of time and create a large amount of memory.

Even if you have not used Take , a memory issue can be important. For instance:

 // Puts half a million elements in memory, sorts, then outputs them. var numbers = Enumerable.Range(1, 1000000).Where(i => i % 2 == 0) .OrderByDescending(i => i); foreach(var number in numbers) Console.WriteLine(number); // Puts one element in memory at a time. var numbers = Enumerable.Range(1, 1000000).Where(i => i % 2 == 0); foreach(var number in numbers) Console.WriteLine(number); 

Answer 2

The fact that it gives seems to be relevant only because the operator receives deferred execution.

So what is the importance of stream operators?

those. you cannot process infinite sequences using buffering / non-streaming methods - while you can “run” such a sequence (until you stop execution) is just fine using only stream extension methods.

Take for example this method:

 public IEnumerable<int> GetNumbers(int start) { int num = start; while(true) { yield return num; num++; } } 

You can use Where just fine:

 foreach (var num in GetNumbers(0).Where(x => x % 2 == 0)) { Console.WriteLine(num); } 

OrderBy() will not work in this case, since it will have to exhaustively list the results before emitting a singular.

Answer 3

Just to be explicit; in the case you mentioned, there is no advantage in that where the threads are, since orderby sucks it all up anyway. However, there are times when the benefits of streaming are used (examples are given in other answers / comments), therefore all LINQ operators transmit to the best of their ability . Orderby streams , as far as possible , which is not very. Where threads are very efficient.