Differences between the “import module” and “from the package import module”

Question

I studied the dynamic "plugin loading" for python and noticed a not very problematic, but an interesting difference between import module and from package import module .

I created a test script consisting of four files (similar to my own setup, for which I want to achieve)

The file tree looks like this:

• test (main package)
  • sup (package, folder with plugins)
    • __ __ INIT. RU
    • uber.py (plugin)
  • __ __ INIT. RU
  • bar.py ('main'-program)
  • foo.py (an object that requires dynamically added functionality)
  • poo.py (decorator)

poo.py:

 from test import foo def decorate(method): print "before:", method.__name__ in dir(foo.Foo) setattr(foo.Foo, method.__name__, method) print "after :", method.__name__ in dir(foo.Foo) return method 

foo.py:

 import os class Foo(object): def __init__(self): self.__loadplugins("sup") @classmethod def __loadplugins(cls, plugindir): for f in os.listdir(os.path.join(os.path.dirname(__file__), plugindir)): if f.endswith(".py"): __import__(("%s.%s" % (plugindir, f))[0:-3]) 

uber.py:

 from test import poo @poo.decorate def aFunction(self, anArg): print anArg 

I have two versions of bar.py, this does not work:

 import foo f = foo.Foo() f.aFunction("print goes here") # pylint: disable-msg=E1101 

It works:

 from test import foo f = foo.Foo() f.aFunction("print goes here") # pylint: disable-msg=E1101 

The only difference between the two columns is import. One of them is relative, the other is not. But relative does not work, and absolute does. Can someone reproduce this on their machine and can give some explanation why this is happening?

Update
Thought it would be useful to mention my python version as well: Using the regular version of python 2.7.2 x86

Update
The output of the "wrong" bar.py:

 before: False after : True Traceback (most recent call last): File "C:\Users\Daan\workspace\python\mytests\src\test\bar.py", line 6, in <module> f.aFunction("print goes here") # pylint: disable-msg=E1101 AttributeError: 'Foo' object has no attribute 'aFunction' 

The output of the "correct" bar.py:

 before: False after : True print goes here 

Answer 1

I assume that you are in the “test” directory and have the environment variable PYTHONPATH = .. (so you can do “import foo” and “from test import foo”).

In this case, foo and test.foo are two different modules that are loaded separately (although they are loaded from the same file).

"decorate" the function from the test.poo module adds the method to the "Foo" class, which lives in test.foo (the first line is poo.py: "from test import foo"), while the "Foo" class from the foo module remains unchanged.

Answer 2

How do you execute bar.py? I assume that you are going to run it as a package, because that is how you developed it. Or else, from test import foo in the second version of bar.py does not make much sense, since test will not be recognized by bar.py if it is not running as a package.

Take a look at the python docs for in -package links . They mainly talk about two types of links. Say you are trying to import uber.py from foo.py. One way to do this is to use an explicit relative link of the form from .sup import uber . Another way to do this is via an absolute link, and this will be from test.sup import uber or import test.sup.uber

It looks like you are using absolute links except foo.py Here you actually call __import__('sup.uber') , where it should have been __import__('test.sup.uber') . I'm not sure if this is what causes the error message, but I managed to run both versions of bar.py using

I got both versions of bar.py to run using this in foo.py :

 __import__(("%s.%s" % ('test.'+plugindir, f))[0:-3]) 

Also, how do you use bar.py as a package? One way to do this is to include the import test.bar line in the script outside the test directory, and then run it.

Answer 3

Short answer to your questions:

Is there anyone who can reproduce this on their machine and can give some explanation why this is happening?

Yes.: -)

Somewhat longer:

Yes - I was able to reproduce it.

Here is my explanation:

In the first case (where the error occurs), python includes the foo module twice (you can easily check it with the print statement). The foo class is created a second time, and aFunction added to the second incarnation - yet the main program uses the first.

In the second case, the module foo read only once - and therefore the class foo exists only once. The same incarnation of the class is used, adding aFunction and basically.

The reason for the simultaneous import of the file is IMHO somewhat strange arrangement of modules. It looks like for python, the internal path / name used for the foo module is different in the first example.

(I used a lot of print from id(foo.Foo) and globals() to get this.)