Speed ​​to repeat several times over the generator compared to the list

Question

I expected that in the case of repeating several loops, the iteration would be much faster than using the generator, and my code suggests that this is a lie.

My understanding (by operation I mean any expression defining an element):

• list requires initialization of n operations
• but then each loop over the list just grabs the item from memory
• Thus, m loops over a list require only n operations
• The generator does not require initialization of any operations.
• however, a loop over the generator triggers operations in fly
• Thus, for one cycle of the generator requires n operations
• but m loops over the generator require nxm operations
  

And I checked my expectations using the following code:

from timeit import timeit

def pow2_list(n):
    """Return a list with powers of 2"""

    results = []

    for i in range(n):
        results.append(2**i)

    return results

def pow2_gen(n):
    """Generator of powers of 2"""

    for i in range(n):
        yield 2**i

def loop(iterator, n=1000):
    """Loop n times over iterable object"""

    for _ in range(n):
        for _ in iterator:
            pass

l = pow2_list(1000) # point to a list
g = pow2_gen(1000)  # point to a generator


time_list = \
    timeit("loop(l)", setup="from __main__ import loop, l", number=10)

time_gen = \
    timeit("loop(g)", setup="from __main__ import loop, g", number=10)

print("Loops over list took: ", time_list)
print("Loops over generator took: ", time_gen)

And the results surprised me ...

Loops over list took:  0.20484769299946493
Loops over generator took:  0.0019217690005461918

- , , 1000 . ! ?

EDIT:

. . , , , :

>>> x = range(10)
>>> sum(x)
45
>>> sum(x)
45

( ...).

: , .

Answer 1

. pow2_gen, g ; loop, StopIteration. loop, next(g) ( g.next() Python 2) StopIteration, g .

, .

, , , append , , , . .

. create_list create_gen , , . time_loop loop, time_apply - loop, .

def create_list(n=1000):
    return [2**i for i in range(n)]

def create_gen(n=1000):
    return (2**i for i in range(n))

def time_loop(iterator, n=1000):
    for t in range(n):
        for v in iterator:
            pass

def time_apply(create_fn, fn_arg, n=1000):
    for t in range(n):
        iterator = create_fn(fn_arg)
        time_loop(iterator, 1)

print('time_loop(create_list): %.3f' % timeit("time_loop(create_list(1000))",
                                              setup="from __main__ import *",
                                              number=10))

print('time_loop(create_gen): %.3f' % timeit("time_loop(create_gen(1000))",
                                             setup="from __main__ import *",
                                             number=10))

print('time_apply(create_list): %.3f' % timeit("time_apply(create_list, 1000)",
                                               setup="from __main__ import *",
                                               number=10))

print('time_apply(create_gen): %.3f' % timeit("time_apply(create_gen, 1000)",
                                              setup="from __main__ import *",
                                              number=10))

, (time_apply(create_list)) (, , ) (time_apply(create_gen)).

time_loop(create_list): 0.244
time_loop(create_gen): 0.028
time_apply(create_list): 21.190
time_apply(create_gen): 21.555

, , : time_loop(create_gen) , time_loop(create_list). , , , .

, (time_loop(create_list)) , (time_apply(create_gen)) .

, . 1000 , , . 100 000 -.

print('create big list: %.3f' % timeit("l = create_list(100000)",
                                       setup="from __main__ import *",
                                       number=10))

print('create big gen: %.3f' % timeit("g = create_gen(100000)",
                                      setup="from __main__ import *",
                                      number=10))

:

create big list: 209.748
create big gen: 0.023

Python 700 800 , ; . Python , , ; - .

Answer 2

. , . , . .

l = [0, 1, 2, 4, 5]
g = iter(l) # creates an iterator (a type of generator) over the list

sum_list0 = sum(l)
sum_list1 = sum(1)
assert sum_list0 == sum_list1 # all working normally

sum_gen0 = sum(g) # consumes generator
sum_gen1 = sum(g) # sum of empty generator is 0
assert sum_gen0 == sum_list1 # result is correct
assert sum_gen1 == sum_list1, "second result was incorrect" # because generator was exhausted

, , , timeit.

from timeit import timeit

n = 1000
repeats = 10000

list_powers = [2**i for i in range(n)]
def gen_powers():
    for i in range(n):
        yield 2**i

time_list = timeit("min(list_powers)", globals=globals(), number=repeats)
time_gen = timeit("min(gen_powers())", globals=globals(), number=repeats)

print("Loops over list took: ", time_list)
print("Loops over generator took: ", time_gen)

:

Loops over list took:  0.24689035064701784
Loops over generator took:  13.551637053904571

, . , . n , . , , . , , , , . n 1000 (), repeats n, .

Answer 3

, loop(). , ( ).

Here is an illustration of this. I use a generator expression and list comprehension (this is more optimized than using appendin a loop for), but the concept is the same:

>>> gen = (i for i in range(3))
>>> for n in range(2):
...     for i in gen:
...         print(i)
... 
0 # 1st print
1
2 # after one loop the iterator is exhausted
>>> 
>>> lst = [x for x in range(3)]
>>> for n in range(2):
...     for i in lst:
...         print(i)
... 
0 # 1st print
1
2
0 # 2nd print
1
2 
>>> 

For an equivalent test, you must rebuild the generator after each iteration of the outer loop:

>>> for n in range(2):
...     gen = (i for i in range(3))
...     for i in gen:
...         print(i)
... 
0 # 1st print
1
2
0 # 2nd print
1
2
>>>