The python submodule mask from its __init__.py package

Question

Short version

For a specific installation, I need to create local variables in __init__.py , which should mask the modules from one package.

eg. the y variable (in the local context __init__.py ) should hide the y.py module. The import xy statement should give a local variable instead of loading the module.

If you do not want to read about a particular setting, scroll down to the question; this is understandable without details.

Detailed description

I implemented a set of Python 2.7 packages, each of which may require individual configuration settings. For convenience, I planned to provide default settings for each package, which can be locally overwritten by those who use one of the packages.

(The rationale for this is to extend the default settings when deploying the application on a machine running in a specific environment (server, workstation, laptop, etc.), but at the same time allow configuration overrides without spoiling the local repository or reset local adaptations when updating the code.)

Example directory structure:

 ~/pkg/ | +- package_a/ | | | +- __init__.py | +- mod_x.py | +- mod_y.py | +- package_b/ | | | +- __init__.py | +- mod_z.py | +- config/ | | | +- __init__.py | +- package_a.py # Should locally override <pkg>_sample.py | +- package_a_sample.py | +- package_b_sample.py | +- test_this.py 

I would like to access the settings stored in config/ , for example, to regularly import modules, for example:

 # ~/pkg/test_this.py import config.package_a as cfg_a 

... but it implicitly switches to an override file if one exists.

My approach to solving

To automate the process in some way, I dynamically create local variables that point to the correct import of configuration files. Using the imp package, I can import the module and, in particular, name it at the same time. (Ie at runtime you cannot tell whether <pkg>_sample.py or <pkg>.py loaded to serve the configuration.)

Finally I ended up with this:

 # ~/pkg/config/__init__.py import os import imp __all__ = ['datastore'] _cfgbase = os.path.dirname(os.path.realpath(__file__)) for cfgmodule in __all__: if os.path.isfile(os.path.join(_cfgbase, cfgmodule + '.py')): locals()[cfgmodule] = imp.load_source( cfgmodule, os.path.join(_cfgbase, cfgmodule + '.py')) else: locals()[cfgmodule] = imp.load_source( cfgmodule, os.path.join(_cfgbase, cfgmodule + '_sample.py')) 

This actually creates a local link to the required source files (excluding <pkg>_sample.py when <pkg>.py exists in config/ .

I can use it from other modules / scripts when using from config import package_a as cfg_a .

Question

Essentially, this question can return to the famous import xy vs from x import y -thing.

But there is a difference.

I know that import xy requires y be a module. Is there any way to hide the module in the __init__.py package and provide a local instad variable upon import?

• from x import y returns the local variable y from x __init__.py
• import xy always imports the module, even if the local variable y exists in __init__.py .

I can’t get everyone to always use the old import statement, people like to use the latter in their code.

Can I advise here?

Edited: Fixed name. Unfortunately.

Decision

Thanks to @ martijn-pieters for pointing out sys.modules .

In fact, my approach would work completely without explicitly adding a new import to sys.modules , as I simply could not correctly name the new imports:

 locals()[cfgmodule] j= imp.load_source( 'config.' + cfgmodule, os.path.join(_cfgbase, cfgmodule + '.py')) 

This solves the problem because it does not register a new submodule with its canonical name (here: package_a ), but registers it as a submodule of my config package.

Thanks a lot!

Answer 1

import xy does not really require y be a module. import xy looks at the 'x' and 'x.y' in the sys.modules structure. If both are found, then x bound to sys.modules['x'] . Only if 'x.y' does not exist, Python will look for a loadable module.

Then the trick is to stuff your y on sys.modules :

 sys.modules['x.y'] = y