Install pip using installed ArcGIS Python 2.7

Question

I am trying to install Scrapy for Python 2.7 on Windows 8.1, and I understand that I need to install pip first. Since I have Python installed through ArcGIS 10.2, I think I need to install pip in C: \ Python27 \ ArcGIS10.2 \ lib \ site-packages. After pip is installed in this directory, I get an error code:

>>> import pip Traceback (most recent call last): File "<interactive input>", line 1, in <module> File "C:\Python27\ArcGIS10.2\lib\site-packages\pip\__init__.py", line 10, in <module> from pip.util import get_installed_distributions, get_prog File "C:\Python27\ArcGIS10.2\lib\site-packages\pip\util.py", line 18, in <module> from pip._vendor.distlib import version File "C:\Python27\ArcGIS10.2\lib\site-packages\pip\_vendor\distlib\version.py", line 14, in <module> from .compat import string_types File "C:\Python27\ArcGIS10.2\lib\site-packages\pip\_vendor\distlib\compat.py", line 38, in <module> from HTMLParser import HTMLParser File "C:\Python27\ArcGIS10.2\lib\HTMLParser.py", line 47, in <module> """, re.VERBOSE) File "C:\Python27\ArcGIS10.2\lib\re.py", line 190, in compile return _compile(pattern, flags) File "C:\Python27\ArcGIS10.2\lib\re.py", line 242, in _compile raise error, v # invalid expression error: nothing to repeat 

I also installed pip in C: \ Python27 \ lib \ site-packages. However, when it is installed only in this directory, PyScripter does not recognize that it is installed. Anyone have any suggestions?

Answer 1

I had the same problem and solved it by doing a really CLEAN reinstall of python.

My version of ArcGIS went up over and over again from 10.2.2 to 10.3 at 10.2 at 10.2.2. When ArcGIS (or just the Python functions) is uninstalled using Add or Remove Programs, most files from C: \ Python27 will be deleted. However, this will not remove the python dll from your system folder. Depending on your Windows OS, this will either:

• C: \ Windows \ System32 \ python27.dll
• C: \ Windows \ SysWOW64 \ python27.dll

Remove the DLL manually and then install python again.

This should give you a really clean python installation and then run get-pip.py again, and pip should work as expected!

PS DLL removal is really important if you lower your python. Since python27.dll seems to be replaced when updating python, but not when lowering it. Therefore, there is incompatibility between python27.dll and scripts in C: \ Python27.

Answer 2

I have a similar setup (Python is installed through ArcGIS 10.2, but on machines running Windows 7 not 8.1). I used PIP to install another package (bird instead of scrapy) and made it work. I think your problem may arise from the Python interpreter, and not from the command line (oh, you are probably Unix users with your permanent command line). Here is what worked for me:

• Go to http://pip.readthedocs.org/en/latest/installing.html
• Download the get-pip.py file and put it in the python folder, for example: C: \ python27 \ arcgis10.2 \
• Run the command prompt (Start menu → Accessories> Command Prompt)
• Change directories to the python folder by typing: cd c: \ python27 \ arcgis10.2
• Install PIP by typing: python get-pip.py
• Change directories in the scripts folder by typing: cd scripts
• Use pip to install your package (e.g. scrapy) by typing: pip install scrapy

If this works, you should now enter Python and import scrapy. This worked for me on every computer in my lab ... just not on my own laptop ... soon I will write my own question (arghh!).

Answer 3

Modify the HTMLParser.py file as follows ( C:\Python27\ArcGIS10.2\lib\HTMLParser.py for me):

Before:

 locatestarttagend = re.compile(r""" <[a-zA-Z][-.a-zA-Z0-9:_]* # tag name (?:[\s/]* # optional whitespace before attribute name (?:(?<=['"\s/])[^\s/>][^\s/=>]* # attribute name (?:\s*=+\s* # value indicator (?:'[^']*' # LITA-enclosed value |"[^"]*" # LIT-enclosed value |(?!['"])[^>\s]* # bare value ) )?(?:\s|/(?!>))* )* )? \s* # trailing whitespace """, re.VERBOSE) 

After:

 locatestarttagend = re.compile(r""" <[a-zA-Z][-.a-zA-Z0-9:_]* # tag name (?:[\s/]* # optional whitespace before attribute name (?:(?<=['"\s/])[^\s/>][^\s/=>]* # attribute name (?:\s*=+\s* # value indicator (?:'[^']*' # LITA-enclosed value |"[^"]*" # LIT-enclosed value |(?!['"])[^>\s]* # bare value ) )?(?:\s|/(?!>))* )* ) # >>>>>>>>>>>>>>>>>>>>>>>>>>> Remove the ? <<<<<<<<<<<<<<<<<< \s* # trailing whitespace """, re.VERBOSE) 

Then use pip . I do not know what this modification will affect. It may be safer to add a polling point after using pip