"There is no module named _scproxy" on OSX

Question

I'm on OSX 10.6 with python 2.6 preinstalled and would like to install python packages via easy_install or setup.py (in the downloaded package). In my case, I am trying to install MySQLdb. In both cases, I get a stack trace that ends like this:

... File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/command/easy_install.py", line 21, in <module> from setuptools.package_index import PackageIndex, parse_bdist_wininst File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/package_index.py", line 2, in <module> import sys, os.path, re, urlparse, urllib2, shutil, random, socket, cStringIO File "/System/Library/Frameworks/Python/framework/Versions/2.6/lib/python2.6/urllib2.py", line 111, in <module> from urllib import (unwrap, unquote, splittype, splithost, quote, File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.py", line 1335, in <module> from _scproxy import _get_proxy_settings, _get_proxies ImportError: No module named _scproxy 

The python installation is an unmodified preinstalled version 2.6.1, except that I added the source files to the lib folder. "Find / System / Library / Frameworks / Python.framework / -name scproxy" does not produce any results.

How to install a missing module?

Answer 1

Background

_scproxy is a Mac-specific urllib helper associated with OS-specific libraries for executing HTTP requests. It seems that my system is also missing (10.6.7). Previously, I think this seems like a problem with the Python build system (I can't find anything like /System/Libraries ).

Hack-o-rama solution

It is (sort of) possible to install the missing module. But first, some tips:

^TM cannot work too much with your Python system. Do yourself a favor by learning to use virtualenv and apply potentially dangerous operations to new, new virtualenv. This way you will not be prone to installing problematic packages.

Anyway: Python stock on Snow Leopard is 2.6.1. I did my experiments with the most recent 2.6, 2.6.6, a safer way would be to download this instead. However, my experience is that the various point releases work great together.

Anyway, I downloaded 2.6.6 into my ~/src directory as follows:

 ~/src/ext/python$ wget http://www.python.org/ftp/python/2.6.6/Python-2.6.6.tgz ~/src/ext/python$ tar zxf Python-2.6.6.tgz ~/src/ext/python$ cd Python-2.6.6 ~/src/ext/python$ ./configure 

console spam like crazy

 ~/src/ext/python$ make sharedmods 

I hope there are no errors, more mailing to the console

Find the newly built _scproxy.so :

 ~/src/ext/python/Python-2.6.6$ find . -name '_scproxy.so' ./build/lib.macosx-10.4-x86_64-2.6/_scproxy.so # <- exact path may vary 

Now you can copy _scproxy.so to /System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynlo ad (and then remember that your system Python has a built-in somewhat alien module). Or, which is much better, add it to the lib/python2.6/ virtual subdirectory. After completing these steps, I could import _scproxy as indicated in your trace:

 Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from _scproxy import _get_proxy_settings, _get_proxies >>> 

This is a strong indication that installing packages using a method requiring urllib requests will work. From there you continue your work, though, since I do not want to test the installation of MySQL myself.