Python module paths

Question

I created a folder with all my modules for my GAE application and with external libraries such as Jinja2 so that everything is sorted in one place. I have a folder structure as follows:

lib\ \utils\ \__init__.py \firepython \jinja2 \jsonpickle __init__.py sessions.py 

When I try to load Jinja from utils__init__.py, I get an ImportError: No module named jinja2.environment . When I look at the Jinja2 import instructions, I see that they look like from jinja2.loaders . I am trying to modify them to be like from lib.jinja2.loaders , but some errors appear after import. Moreover, I do not think that it is good practice to change this import in the source of external libraries if there is a more convenient and correct way to import modules properly. I also added some paths to PYTHONPATH , but it does not solve all the problems. How can I correctly import an external package that is placed in another folder, maybe with a deep structure?

Answer 1

Indeed, you do not need to change the import in external libraries, although depending on your environment you may even have one.

PYTHONPATH

Modification of your PYTHONPATH should be sufficient; PYTHONPATH should contain the path "lib", which is absolute or relative to your home, for example.

Then you could just do

 from jinja2 import WHATEVER 

sys.path.append

Another way to navigate without PYTHONPATH is to use sys.path.append() and add your paths from your Python code. I really approve of this, as it also allows you to have paths for each application.

use virtualenv

The details will be a little long so that they can be placed here, but please follow the official document

These options apply to general python development, not GAE features; if it does not work on your development machine, you should post more detailed information (exact import, absolute paths, pythonpath ...).

The proper design of the project and the use of appcfg.py should be workout dependent on google loading: please take a look at this good answer: How do I manage third-party Python libraries using the Google App Engine? (virtualenv? pip?) and follow these recommendations.

A good way to go with GAE is through the yaml application directives. Please take a look at the document, which includes: http://code.google.com/appengine/docs/python/config/appconfig.html#Includes

Also remember that GAE officially supports python 2.5 and 2.7 supports is experimental

Python 2.7 is now officially supported

Answer 2

To correctly import a module, you need to make sure python knows where to find it. To do this, for each external library, add the parent directory to sys.path (at run time) or set the PYTHONPATH environment (before starting).

For instance:

 import sys sys.path.append('/my/lib') # now we can import from lib import jsonpickle # will load /my/lib/jsonpickle/__init__.py 

See http://docs.python.org/tutorial/modules.html#the-module-search-path . to understand what python does when import is called.