Permission denied while trying to install inside virtualenv using available

Question

I have this game in my indispensable play

- name: Setup virtualenv pip: requirements="/vagrant/webapp/requirements/{{ requirements_filename }}" virtualenv="$HOME/.envs/{{ project_name }}" 

However, I get an error resolved by Permission. It seems that ansible is trying to install the package in the system path, not in virtualenv.

Here's the full trace:

 TASK: [deploy | Setup virtualenv] ********************************************* failed: [default] => {"cmd": "/usr/bin/pip install -r /vagrant/webapp/requirements/dev.txt", "failed": true} msg: stdout: New python executable in /home/vagrant/.envs/venv/bin/python Installing setuptools, pip...done. Downloading/unpacking django>=1.8 (from -r /vagrant/webapp/requirements/requirements.txt (line 3)) Requirement already satisfied (use --upgrade to upgrade): wsgiref>=0.1.2 in /usr/lib/python2.7 (from -r /vagrant/webapp/requirements/requirements.txt (line 4)) Downloading/unpacking django-allauth>=0.19.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 5)) Running setup.py (path:/tmp/pip_build_vagrant/django-allauth/setup.py) egg_info for package django-allauth Downloading/unpacking psycopg2>=2.6 (from -r /vagrant/webapp/requirements/requirements.txt (line 6)) Running setup.py (path:/tmp/pip_build_vagrant/psycopg2/setup.py) egg_info for package psycopg2 Downloading/unpacking django-currencies>=0.3.3 (from -r /vagrant/webapp/requirements/requirements.txt (line 7)) Downloading django-currencies-0.3.3.tar.gz Running setup.py (path:/tmp/pip_build_vagrant/django-currencies/setup.py) egg_info for package django-currencies Downloading/unpacking django-countries>=3.2 (from -r /vagrant/webapp/requirements/requirements.txt (line 8)) Running setup.py (path:/tmp/pip_build_vagrant/django-countries/setup.py) egg_info for package django-countries Downloading/unpacking stripe>=1.22.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 9)) Running setup.py (path:/tmp/pip_build_vagrant/stripe/setup.py) egg_info for package stripe Downloading/unpacking pillow>=2.8.0 (from -r /vagrant/webapp/requirements/requirements.txt (line 10)) Running setup.py (path:/tmp/pip_build_vagrant/pillow/setup.py) egg_info for package pillow Single threaded build, not installing mp_compile: 1 processes warning: no files found matching 'LICENSE' under directory 'docs' Downloading/unpacking django-compressor>=1.4 (from -r /vagrant/webapp/requirements/requirements.txt (line 11)) Downloading/unpacking djangorestframework>=3.1.1 (from -r /vagrant/webapp/requirements/requirements.txt (line 12)) Downloading/unpacking django-filter>=0.9.2 (from -r /vagrant/webapp/requirements/requirements.txt (line 13)) Downloading django_filter-0.9.2-py2.py3-none-any.whl Obtaining formtools from git+https://github.com/django/django-formtools.git#egg=formtools (from -r /vagrant/webapp/requirements/requirements.txt (line 14)) Cloning https://github.com/django/django-formtools.git to ./src/formtools Running setup.py (path:/tmp/src/formtools/setup.py) egg_info for package formtools Installing extra requirements: 'egg' Downloading/unpacking django-widget-tweaks>=1.3 (from -r /vagrant/webapp/requirements/requirements.txt (line 15)) Downloading django-widget-tweaks-1.3.tar.gz Running setup.py (path:/tmp/pip_build_vagrant/django-widget-tweaks/setup.py) egg_info for package django-widget-tweaks Downloading/unpacking django-braces>=1.4.0 (from -r /vagrant/webapp/requirements/requirements.txt (line 16)) Downloading django_braces-1.4.0-py2.py3-none-any.whl Downloading/unpacking django-debug-toolbar>=1.3.0 (from -r /vagrant/webapp/requirements/dev.txt (line 4)) Downloading/unpacking python-openid>=2.2.5 (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5)) Running setup.py (path:/tmp/pip_build_vagrant/python-openid/setup.py) egg_info for package python-openid warning: no files found matching 'CHANGELOG' warning: no files found matching '*.css' under directory 'doc' warning: no files found matching '*.html' under directory 'doc' Downloading/unpacking requests-oauthlib>=0.3.0 (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5)) Downloading requests_oauthlib-0.4.2-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): requests>=1.0.3 in /usr/lib/python2.7/dist-packages (from django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5)) Downloading/unpacking django-appconf>=0.4 (from django-compressor>=1.4->-r /vagrant/webapp/requirements/requirements.txt (line 11)) Downloading django_appconf-1.0.1-py2.py3-none-any.whl Requirement already satisfied (use --upgrade to upgrade): six in /usr/local/lib/python2.7/dist-packages (from django-braces>=1.4.0->-r /vagrant/webapp/requirements/requirements.txt (line 16)) Downloading/unpacking sqlparse (from django-debug-toolbar>=1.3.0->-r /vagrant/webapp/requirements/dev.txt (line 4)) Running setup.py (path:/tmp/pip_build_vagrant/sqlparse/setup.py) egg_info for package sqlparse Downloading/unpacking oauthlib>=0.6.2 (from requests-oauthlib>=0.3.0->django-allauth>=0.19.1->-r /vagrant/webapp/requirements/requirements.txt (line 5)) Running setup.py (path:/tmp/pip_build_vagrant/oauthlib/setup.py) egg_info for package oauthlib Installing collected packages: django, django-allauth, psycopg2, django-currencies, django-countries, stripe, pillow, django-compressor, djangorestframework, django-filter, formtools, django-widget-tweaks, django-braces, django-debug-toolbar, python-openid, requests-oauthlib, django-appconf, sqlparse, oauthlib Cleaning up... Exception: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main status = self.run(options, args) File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 283, in run requirement_set.install(install_options, global_options, root=options.root_path) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1435, in install requirement.install(install_options, global_options, *args, **kwargs) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 671, in install self.move_wheel_files(self.source_dir, root=root) File "/usr/lib/python2.7/dist-packages/pip/req.py", line 901, in move_wheel_files pycompile=self.pycompile, File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 206, in move_wheel_files clobber(source, lib_dir, True) File "/usr/lib/python2.7/dist-packages/pip/wheel.py", line 193, in clobber os.makedirs(destsubdir) File "/usr/lib/python2.7/os.py", line 157, in makedirs mkdir(name, mode) OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/Django-1.8.dist-info' Storing debug log for failure in /home/vagrant/.pip/pip.log FATAL: all hosts have already failed -- aborting 

Answer 1

I solved the problem by setting the path explicitly and not using the $HOME variable.

 - name: Setup virtualenv pip: virtualenv="/home/vagrant/.envs/{{ project_name }}" requirements="/vagrant/webapp/requirements/{{ requirements_filename }}" 

Answer 2

Try also specifying the pip executable path:

 executable="$HOME/.envs/{{ project_name }}/bin/pip" 

Not looking for the exact cause of TBH, but had to do the same to get the material installed in virtual environments. It seemed like he was trying to install in a virtual environment using the global pip . Perhaps this can be done in some circumstances, this is not my business.

Answer 3

The pip module seems to be using the Python system, so it will try to install the package on the expected system path on site-packages , and since you are not using it as root or using sudo, it cannot write in '/usr/local/lib/python2.7 / ... '

I assume you want to install the pip package under virtualenv, right?

If so, beware that Ansible modules always use the default Python interpreter by default unless you set ansible_python_interpreter . An example of how to do this can be found on the best way to always run ansible inside virtualenv on remote machines?

A link to a short discussion about the subject and why Ansible people don’t want to change #!/usr/bin/python for more virtual friendly #!/usr/bin/env python can be found.

As a side note, you will have the same problem if you use Ansible from a local virtualenv and try using the ec2 module without having boto installed at the system level (but installed at virtualenv level).

Answer 4

It looks like it uses /usr/bin/pip , not pip from virtualenv. Not sure why this happens - Ansible bug? Can you manually verify that your virtualenv has a pip command? One thing I can think of from pip head is being created for root (sudo), and you are trying to run this command as a regular user and do not have read access to the virtual pip.

You can probably use pip manually, like this for a workaround.

 - name: Create virtualenv for app using chosen Python shell: virtualenv --python=python3.4 {{deploy_location}}/venv sudo: yes - name: Install Python packages sudo: yes command: {{deploy_location}}/venv/bin/pip -r {{deploy_location}}/requirements.txt