Why does exec not work in a function with a subfunction?

Question

It looks like you cannot use exec in a function that has a subfunction ...

Does anyone know why this Python code is not working? I get an error in exec in test2. In addition, I know that exec is not a very good style, but believe me, I am using exec for the appropriate reason. I would not use it otherwise.

#!/usr/bin/env python # def test1(): exec('print "hi from test1"') test1() def test2(): """Test with a subfunction.""" exec('print "hi from test2"') def subfunction(): return True test2() 

EDIT: I narrowed the error down to a function in a subfunction. This has nothing to do with the raise keyword.

Answer 1

Right. You cannot use exec in a function that has a subfunction unless you specify a context. From the docs:

If exec is used in a function, and the function contains a nested block with free variables, the compiler will raise SyntaxError, unless exec explicitly specifies the local namespace for exec. (In others, the words “exec obj” would be illegal, but “exec obj in ns” would be legal.)

There are good reasons for this, which I probably will understand if it weren’t for Sunday night. Now, the next question: why are you using exec? This is very rarely needed. You say that you have a good reason. I am skeptical about this.;) If you have a good reason, I will tell you a workaround. :-P

Well, one way or another:

 def test2(): """Test with a subfunction.""" exec 'print "hi from test2"' in globals(), locals() def subfunction(): return True 

Answer 2

Although in Python it looks like local variables are stored in the locals() dictionary, they are usually not. Instead, they are mostly stored on the stack and are accessible by index. This makes searching for local variables faster than if you had to do a dictionary search each time. If you use the locals() function, then you get a fresh dictionary created from all local variables, and why the purpose of locals() usually does not work.

There are a few exceptions to this scenario:

When you use unskilled exec inside a function, Python disables optimization and uses a real dictionary for local variables. This means that you can create or update variables from exec , but it also means that access to a local variable in this function will be slower.

Another exception is that when you insert functions, the inner function can access local variables in the outer scope. When this happens, the variable is stored in the "cell" object, and not stored on the stack. An additional level of indirection makes all use of variables with scope slower, regardless of whether you get them from an internal or external function.

The catch you encounter is that these two exceptions, as local variables are usually stored, are incompatible. You cannot have a variable stored in a dictionary, and at the same time access it through a cell reference. Python 2.x fixes this by prohibiting exec, even in cases where you are not trying to use any variables with scope.

Answer 3

This works well in Python 3.1.3 after changing the print statement to use the print function. A.

In Python 2.6, it produces a SyntaxError: unqualified exec is not allowed in function 'test2' it contains a nested function with free variables , I don't think this is an error.

Answer 4

This is a pretty interesting case:

 >>> def func(): ... exec('print "hi from func"') ... def subfunction(): ... return True ... File "<stdin>", line 2 SyntaxError: unqualified exec is not allowed in function 'func' because it contains a nested function with free variables 

The reason this doesn't work is because the subfunction contains a free variable, and since in Python 2, exec could theoretically modify locales in the content area, it would be impossible to decide whether the variable should be associated with a global or parent function. One of the poems in Zen Python is "In the face of ambiguity, give up the temptation to guess." and this is what Python 2 does.

Now the question is: what is this unrelated variable? Well, that’s True !

Indeed, it is played using None :

 >>> def func(): ... exec('print "hi from func"') ... def subfunction(): ... return None ... File "<stdin>", line 2 SyntaxError: unqualified exec is not allowed in function 'test2' because it contains a nested function with free variables 

Even if None cannot be assigned, and it is considered a constant in the bytecode, the buggy parser considers this to be an unrelated variable here.

But if you replace it with 1 , and it works without problems:

 >>> def test2(): ... exec('print "hi from func"') ... def subfunction(): ... return 1 ... >>> 

To avoid this error, specify explicitly global and possibly local that exec will use, say:

 >>> def test2(): ... exec 'print "hi from test2"' in {} ... def subfunction(): ... return None ... >>> 

---

In Python 3, exec is just a simple function and is not specifically handled by the parser or bytecode compiler. In Python 3, exec cannot restore function-local names, and thus this syntaxError and ambiguity do not exist.

---

One of the special cases of Python 2 vs 3 compatibility is that while the Python 2.7 Documentation states that

The form exec(expr, globals) equivalent to exec expr in globals , and the form exec(expr, globals, locals) equivalent to exec expr in globals, locals . The exec tuple form provides compatibility with Python 3, where exec is a function, not an operator.

The tuple form was not always 100% compatible, since there was an error processing exec in functions with nested functions (number 21591) ; up to Python 2.7.8, the following code may throw an exception:

 def func(): exec('print "hi from test2"', {}) def subfunction(): return None 

This has been fixed in Python 2.7.9 and it no longer throws.

Answer 5

The error seems pretty obvious to me:

 SyntaxError: unqualified exec is not allowed in function 'test2' it contains a nested function with free variables 

See pep 227 for more details: http://www.python.org/dev/peps/pep-0227/