I am trying to add proxy authentication support to an existing script since the script connects to the https url (with urllib2.Request and urllib2.urlopen), resets the page and performs some actions based on what it found. Initially, I was hoping this would be as simple as adding urllib2.ProxyHandler ({"http: MY_PROXY}) as an argument to urllib2.build_opener, which in turn is passed to urllib2.install_opener. Unfortunately, this does not work when trying to execute urllib2.Request (ANY_HTTPS_PAGE). Googling around gives me the opportunity to believe that proxy support in urllib2 in python 2.5 does not support https requests. This surprised me, to say the least.
It seems that there are solutions floating around the network, for example http://bugs.python.org/issue1424152 contains a patch for urllib2and httplib, which is for solving the problem (when I tried to solve this problem, I started getting the following error:) urllib2.URLError: <urlopen error (1, 'error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol')>. Here is the recipe for the cookbook http://code.activestate.com/recipes/456195 , which I plan to try next. In general, although I am surprised that this is not supported out of the box, which makes me wonder if I just skipped the obvious solutions, so in short, does anyone have a simple method for retrieving https pages using an authentication proxy with urllib2 in Python 2.5? Ideally, this would work:
import urllib2
proxy_handler = urllib2.ProxyHandler({"http": "http://user:pass@myproxy:port"})
urllib2.install_opener( urllib2.build_opener( urllib2.HTTPHandler,
urllib2.HTTPSHandler,
proxy_handler ))
request = urllib2.Request(A_HTTPS_URL)
response = urllib2.urlopen( request)
print response.read()
Many thanks