How can I use SOCKS 4/5 proxies with urllib2?

Question

How can I use SOCKS 4/5 proxy with urllib2 to load a web page?

Answer 1

You can use the SocksiPy module. Just copy the socks.py file to the Python lib / site-packages directory and you are ready to go.

You must use socks before urllib2. (Try pip install PySocks )

For example:

 import socks import socket socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "127.0.0.1", 8080) socket.socket = socks.socksocket import urllib2 print urllib2.urlopen('http://www.google.com').read() 

You can also try pycurl lib and tsocks, for more details click here .

Answer 2

Adding an alternative for panning response when you need to use many different proxies at the same time.

In this case, you need to create an opener, as with the http proxy. GitHub has code available at https://gist.github.com/869791

 opener = urllib2.build_opener(SocksiPyHandler(socks.PROXY_TYPE_SOCKS4, 'localhost', 9999)) print opener.open('http://www.whatismyip.com/automation/n09230945.asp').read() 

Answer 3

Since SOCKS is a socket level proxy, you need to replace the socket object used by urllib2 . Please see this solution. If fixing a monkey is not enough for you, you can try a subclass or copy-modify the code from the standard urllib2 library.