Rules for not recursively importing packages

Question

This appeared in the context of answering another question today.

Suppose the following files, where comments indicate file names:

# level1/__init__.py import level1.level2 answer = level1.level2.answer # level1/level2/__init__.py from .b import answer # level1/level2/b.py from .a import answer from ..level2.a import answer from level1.level2.a import answer import level1.level2.a if answer != 42: answer = level1.level2.a.answer # <-- Fails here #level1/level2/a.py answer = 42 

With this code, python -c "import level1" works fine (both in 2.7 and 3.4) as it is. But change the answer to anything but 42, and it's not with AttributeError: 'module' object has no attribute 'level2' in the specified location.

It seems that from / import it can import the package submodule and push the variables out of its namespace before all the parent namespaces are created, but (obviously) the parent namespaces must be configured before the sub-namespace attributes can go through regular calls to attributes.

Sometimes, however, the namespace is set up "good enough" for import to work. For example, the code below with the top level disabled always works with python -c "import level2" , although we have not finished initializing the level2 namespace when we import level2.a from level2.b .

 # level2/__init__.py from .b import answer # level2/b.py from .a import answer import level2.a if answer != 42: answer = level2.a.answer # <-- Works here #level1/level2/a.py answer = 41 

EDIT

It appears that import xyz inserts a link to z in y, but will not insert a link to y in x. For example, when I change level1/level2/b.py from my first example:

 # level1/level2/b.py from sys import modules def dump(): print '\n Dumping...' for key, value in sorted(modules.items()): if key.startswith('level'): print key, [ x for x in dir(value) if not x.startswith('_')] dump() import level1.level2.a dump() from .a import answer from ..level2.a import answer from level1.level2.a import answer if answer != 42: answer = level1.level2.a.answer 

I get the following results before tracing:

  Dumping... level1 [] level1.level2 [] level1.level2.b ['dump', 'modules'] level1.level2.sys [] Dumping... level1 [] level1.level2 ['a'] level1.level2.a ['answer'] level1.level2.b ['dump', 'level1', 'modules'] level1.level2.level1 [] level1.level2.sys [] 

However, if immediately after the second call to dump() I add the line:

 setattr(modules['level1'], 'level2', modules['level1.level2']) 

then this will not work, because I linked level2 to level1 before accessing it through an attribute search. However, it seems that if the interpreter can bind a in level1 to this import, it can also bind level2 to level exactly the same way.

Is there a good reason why only the lowest-level package is updated during the import operation, or is it an error (or, rather, a function that was added for a single package that needs to be expanded to nested packages)?

NOTE According to Python documentation. When a submodule is loaded using any mechanism ... the binding is placed in the namespace parent modules for the submodule object. .

I believe this is happening. But not so soon in all cases. In addition, the documentation for it may be relatively new .

Answer 1

I filed a problem in Python. The text of this problem is reproduced below:

PEP 8 recommends absolute imports for relative imports, and section 5.4.2 of the import documentation states that imports will place the binding in the import parent space of the imported module.

However, since (with all current versions of Python) this binding is not performed until the module body is completed, there are times when relative imports will work fine, but absolute imports will fail. Consider the simple case of these five files:

 xyz.py: import x x/__init__.py: import xy x/y/__init__.py: import xya x/y/a/__init__.py: import xyb; foo = xybfoo x/y/b/__init__.py: foo = 1 

This will fail in fashion, which can be very surprising for the uninitiated. It will not fail any of the import statements; rather, it will fail with AttributeError in the assignment operator in xya, because the import of y is not complete yet, so y is not yet bound to x.

Perhaps this could be fixed in machine import by binding before exec was executed. Whether this can be done cleanly so as not to cause compatibility issues with existing bootloaders is a question for major developers.

But if a decision is made on the acceptability of the current behavior, then at least PEP 8 and the import documentation should have an explanation of this corner case and how it can be solved with relative imports.