Tox and lib and lib64 and package sites

Question

I use tox and coverage.py to run tests of my Python project on my continuous build server. I also have a pkg_x package from a provider (not available in PyPI) that I installed with python3.5 setup.py install , which puts it in /usr/lib/python3.5/site-packages . Now I need to make this package available for test code.

My current tox.ini as follows:

 [tox] envlist = py35 [testenv] deps = nose coverage commands = coverage run -m nose [] sitepackages = True 

and I run the tests as follows:

 python3.5 -m tox -- --verbose --with-doctest 

This is inefficient - none of the dependency packages listed in my local setup.py (e.g., public stuff like more_itertools ) can be found, although it creates directories like .tox/py35/lib/python3.5/site-packages/more_itertools , which appear to contain the appropriate packages. If I run .tox/py35/bin/python3.5 , sys.path looks like this:

 >>> [re.compile('.*\\.tox').sub('.tox', x) for x in sys.path] ['', '.tox/py35/lib64/python35.zip', '.tox/py35/lib64/python3.5', '.tox/py35/lib64/python3.5/plat-linux', '.tox/py35/lib64/python3.5/lib-dynload', '/usr/lib64/python3.5', '/usr/lib/python3.5', '.tox/py35/lib/python3.5/site-packages'] 

If I delete the line sitepackages = True from my tox.ini , then I get further, in packages like more_itertools , and the rest of the contents in my setup.py dependencies can now be found, but the package provider pkg_x I mentioned above is still I can not find. And sys.path looks like this:

 >>> [re.compile('.*\\.tox').sub('.tox', x) for x in sys.path] ['', '.tox/py35/lib64/python35.zip', '.tox/py35/lib64/python3.5', '.tox/py35/lib64/python3.5/plat-linux', '.tox/py35/lib64/python3.5/lib-dynload', '/usr/lib64/python3.5', '/usr/lib/python3.5', '.tox/py35/lib/python3.5/site-packages', '/usr/lib64/python3.5/site-packages', '/usr/lib/python3.5/site-packages'] 

Anyway .tox/py35/ seems to contain the pkg_x provider package anywhere. And although the /usr/lib/python3.5/site-packages directory is specified when shooting .tox/py35/bin/python3.5 manually, pkg_x not actually found when running the tests.

It looks like sitepackages = True has the opposite effect from what it documented at http://tox.readthedocs.io/en/latest/config.html#confval-sitepackages=True|False , right?

Tips are greatly appreciated!

Answer 1

Tox creates virtualenv , then runs the tests from within this environment.

The --sitepackages argument is a switch that determines whether virtualenv is gaining access to globally installed packages.

The “normal” way to start a tex would be to just say Tox ; installing it through pip or through an OS package should put it in your path. i.e:

 $ tox 

This is the same as saying:

 $ tox -c tox.ini 

If you directly call the kernel, python -m tox , it can do something, but for me it is a red flag. This command seems unlikely to activate the appropriate virtual environment, which explains the problems with package availability. It fits because when you omit sitepackages , it actually adds global packages, because it thinks it is inside virtualenv, and therefore adds that it considers local sitepackages, although they are actually global. The opposite happens when you make it true, because when he searches for global packages, he cannot find them. Regardless of the fact that you do not cause current, as expected, it is confused, as we are.

So just use the provided Tox command.

But wait there again: you say that you have a package that is required but not available on pypi. So how toxic is going to install it? The toxicity docs offer several ways (use the .txt requirements file), but the most direct description here is to activate env and install it manually.

Also good if you need to debug further: go to the .tox directory and activate venv manually. eg:

 $ source .tox/testenv/bin/activate 

(where testenv is the name you used between the brackets in tox.ini ).

Now install the package, however you did it before, for example: pip install pkg_x

Deactivate venv when you are done with this:

 $ deactivate 

Try Tox Now?

If we are on the right track learn more about virtualenv here

Answer 2

It seems like the solution to my problem is to change the tox.ini line in my tox.ini as follows:

 commands = coverage run -m nose [] 

:

 commands = python -m coverage run -m nose [] 

The effect is that python now a virtual command that loads the correct Python interpreter, loads the coverage module, and runs it. Without python -m it just finds that the coverage executable is in my PATH and runs it, with unpredictable results. whitelist_externals can be used, but this will only help solve the problem.

Then, if I also add the line sitepackages = True , it successfully sees the pkg_x that I installed from the provider, and my tests failed.