LINQ Performance - Delayed v / s Immediate Execution

Question

I have seen that sometimes the performance of LINQ to Objects queries can be significantly improved if they are forced to execute a command immediately .ToArray()but cannot understand why. For example, in the example below, the execution of a function is Deferred()much slower than the function Immediate(), and grows exponentially with a value limit(maybe it exponentially in both functions, but the execution time with Immediate()was too short for me to say definitively).

public void Deferred()
{
    var all = Range(limit);
    var even = from e in EvenRange(limit) where all.Contains(e) select e;
    var odd = from o in OddRange(limit) where !even.Contains(o) select o;

    var query = from q in odd select q;

    foreach(var i in query) { var j = i+1; }
}

public void Immediate()
{
    var all = Range(limit);
    var even = (from e in EvenRange(limit) where all.Contains(e) select e)
        .ToArray();
    var odd = (from o in OddRange(limit) where !even.Contains(o) select o)
        .ToArray();

    var query = (from q in odd select q).ToArray();

    foreach(var i in query) { var j = i+1; }
}

public static IEnumerable<int> OddRange(int stop)
{
    for (int i = 1; i < stop; i+=2) yield return i;
}

public static IEnumerable<int> EvenRange(int stop)
{
    for (int i = 2; i < stop; i+=2) yield return i;
}

public static IEnumerable<int> Range(int stop)
{
    for (int i = 0; i < stop; ++i) yield return i;
}

? , , , Deferred() , " O"?

Answer 1

, IEnumerable , , even.Contains( ), , , , . .NET( F #) ( , F # Seq.cache)

Answer 2

emaster70, Deferred even even.Contains. limit.

Immediate , all :

var all = Range(limit).ToArray();

.

Contains - O (n), . , Contains.

, HashSet , Contains O (1).

a limit 10000 100 , Immediate:

public void Joined() {
    var all = Range(limit);
    var even = from e in EvenRange(limit) join a in all on e equals a select e;
    var evenSet = new HashSet<int>(even);
    var odd = from o in OddRange(limit) where !evenSet.Contains(o) select o;

    var query = from q in odd select q;

    foreach (var i in query) { var j = i + 1; }
}

Answer 3

, emaster70, , , , .

Answer 4

var all = Range(limit);    
var even = from e in EvenRange(limit) where all.Contains(e) select e;    
var odd = from o in OddRange(limit) where !even.Contains(o) select o;

since IEnumerables built from iterator blocks or LINQ queries are lazy, evaluating each odd element requires recalculating the entire even sequence. As a result, the great code for this code is terrible. It is instructive to use a very small "limit" and put Console.WriteLine () inside the loop in Range () and observe the behavior.