All () vs for loop with break performance

Question

I found an interesting performance optimization. Instead of using all() :

 def matches(self, item): return all(c.applies(item) for c in self.conditions) 

I profiled that it is faster when a loop is used:

 def matches(self, item): for condition in self.conditions: if not condition.applies(item): return False return True 

With all() profiler shows 1160 additional <genexpr> calls:

  4608 function calls (4600 primitive calls) in 0.015 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 580 0.002 0.000 0.008 0.000 rule.py:23(matches) 1160 0.002 0.000 0.005 0.000 rule.py:28(<genexpr>) 

There are no <genexpr> calls in the for loop:

  2867 function calls (2859 primitive calls) in 0.012 seconds Ordered by: internal time ncalls tottime percall cumtime percall filename:lineno(function) 580 0.002 0.000 0.006 0.000 rule.py:23(matches) 

My question is, where does this difference come from? My first was that all evaluates all conditions, but it is not:

 def foo(): print('foo') return False all(foo() for _ in range(1000)) foo 

Answer 1

To get the next sequence in the generator, next is called on the generator object, so each iteration will have this call. The for loop does not carry this cost, so in this case it can be a little faster. Here the generator does nothing for you in the way of productivity.

My first, although it was that everyone appreciates all the conditions

all stops as soon as a false value is encountered. On the contrary, any stops at the first meaning of the truth; which may help to better define the value of all .

Consider the following functions

 def genfunc(): return all(i for i in range(1, 100)) def forfunc(): for i in range(1, 100): if not i: return False return True 

If we use dis.dis to find out what happens ...

 dis.dis(genfunc) 0 LOAD_GLOBAL 0 (all) 2 LOAD_CONST 1 (<code object <genexpr> at 0x04D4E5A0, file "<ipython-input-2-60c0c9eff4e2>", line 2>) 4 LOAD_CONST 2 ('genfunc.<locals>.<genexpr>') 6 MAKE_FUNCTION 0 8 LOAD_GLOBAL 1 (range) 10 LOAD_CONST 3 (1) 12 LOAD_CONST 4 (100) 14 CALL_FUNCTION 2 16 GET_ITER 18 CALL_FUNCTION 1 20 CALL_FUNCTION 1 22 RETURN_VALUE 

and in the loop cycle version.

 dis.dis(forfunc) 0 SETUP_LOOP 26 (to 28) 2 LOAD_GLOBAL 0 (range) 4 LOAD_CONST 1 (1) 6 LOAD_CONST 2 (100) 8 CALL_FUNCTION 2 10 GET_ITER >> 12 FOR_ITER 12 (to 26) 14 STORE_FAST 0 (i) 16 LOAD_FAST 0 (i) 18 POP_JUMP_IF_TRUE 12 20 LOAD_CONST 3 (False) 22 RETURN_VALUE 24 JUMP_ABSOLUTE 12 >> 26 POP_BLOCK >> 28 LOAD_CONST 4 (True) 30 RETURN_VALUE 

You will notice that in the version of the generator expression there are 2 additional function calls ( CALL_FUNCTION ). This takes into account an additional 1,160 calls (2 calls for each of the 580 cycles) that you see in expression.version.