What is __path__ useful for?

Question

I have never noticed the __path__ attribute, which is defined on some of my packages until today. According to the documentation:

Packages support another special __path__ attribute. This is initialized as a list containing the name of the directory in which the __init__.py packages are located before the code in this file. This variable can be changed; doing this affects future searches for modules and subpackages contained in the package.

Although this feature is not often necessary, it can be used to expand the set of modules found in the package.

Can someone explain to me what exactly this means and why I will ever want to use it?

Answer 1

This is usually used with pkgutil so that the package is placed on disk. For example, zope.interface and zope.schema are separate distributions ( zope is a "namespace package"). You can have zope.interface installed in /usr/lib/python2.6/site-packages/zope/interface/ , while you use zope.schema locally in /home/me/src/myproject/lib/python2.6/site-packages/zope/schema .

If you put pkgutil.extend_path(__path__, __name__) in /usr/lib/python2.6/site-packages/zope/__init__.py , then both zope.interface and zope.schema will be imported because pkgutil will have a __path__ change - ['/usr/lib/python2.6/site-packages/zope', '/home/me/src/myproject/lib/python2.6/site-packages/zope'] .

pkg_resources.declare_namespace (part of Setuptools) is similar to pkgutil.extend_path , but knows more about zips on the way.

Manually modifying __path__ is unusual and probably not necessary, although it is useful to consider the variable when debugging import problems with namespace packages.

You can also use __path__ for monkeypatching, for example, I have a monkey passed through distutils, creating a distutils/__init__.py , which is in the early stage of sys.path :

 import os stdlib_dir = os.path.dirname(os.__file__) real_distutils_path = os.path.join(stdlib_dir, 'distutils') __path__.append(real_distutils_path) execfile(os.path.join(real_distutils_path, '__init__.py')) # and then apply some monkeypatching here... 

Answer 2

If you change __path__ , you can force the interpreter to search in a different directory for modules belonging to this package.

This will allow you, for example, to download different versions of the same module depending on the execution conditions. You can do this if you want to use different implementations of the same functionality on different platforms.

Answer 3

In addition to choosing different versions of the module based on the execution conditions, as Syntactic says, this functionality will also allow you to split your package into several parts / downloads / installs, while preserving the appearance of a single logical package.

Consider the following.

• I have two packages mypkg and _mypkg_foo .
• _mypkg_foo contains an add-on module for mypkg , foo.py
• as downloaded and installed, mypkg does not contain foo.py

mypkg __init__.py might do something like this:

 try: import _mypkg_foo __path__.append(os.path.abspath(os.path.dirname(_mypkg_foo.__file__))) import mypkg.foo except ImportError: pass 

If someone installed the _mypkg_foo package, then mypkg.foo is available to them. If they did not, it does not exist.

Answer 4

A special situation that I encountered is when a package becomes so large that I want to split its parts into subdirectories without changing any code that refers to it.

For example, I have a package called views that collects a number of auxiliary utility functions that get confused with the main purpose of a top-level package. I managed to move these helper functions to the utils subdirectory and add the following line to __init__.py for the views package:

 __path__.append(os.path.join(os.path.dirname(__file__), "utils")) 

With this change, too, of views/__init_.py , I could run the rest of the software with a new file structure without any further changes to the files.

(I tried to do something similar with the import statements in the views/__init__.py , but the views/__init__.py modules were still not visible through the import of the view package - I’m not quite sure if I’m missing something, comments about this is welcome!)

(This answer is based on installing Python 2.7)