If I understand correctly, you are basically asking how to make url request in python using proxy?
If so, you can do it as follows:
import urllib2 import urllib import random CAproxies = [{"http":"199.201.122.175:3128", "https":"199.201.122.175:3128"},{"http":"192.99.3.129:3128", "https":"192.99.3.129:3128"},{"http":"192.99.246.101:8118", "https":"192.99.246.101:8118"},{"http":"205.205.129.130:443", "https":"205.205.129.130:443"} ] proxies = urllib2.ProxyHandler(random.choice(CAproxies)) url = 'https://store.playstation.com/#!/en-ca/movies/the-house/cid=UV0130-NPVA92773_CN-0000000000236063' request = urllib2.Request(url) request.add_header("User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:25.0) Gecko/20100101 Firefox/25.0") request.add_header("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") opener = urllib2.build_opener(proxies) urllib2.install_opener(opener) r = urllib2.urlopen(request, timeout=15) html = r.read()
The headings are good if you want the service to think that you are using a browser, they usually have protection against bots. You need to replace the proxy address with your own proxy server, this is just an invented proxy for illustration.
The list of proxies can be found here, for example: http://www.proxy-listen.de/Proxy/Proxyliste.html In case the above proxy does not work. In case one of the proxies works best for your specific location (let's say that the second one works best for you, it might be a good idea to change the random proxy selection only to the second one. Ie
random.choice(CAproxies) -> CAproxies[1]
CAproxies [3] works best for me. The first 250 characters from html:
>>> html[0:250] '<!DOCTYPE html>\n\n<html class="ctry mobvportA rgba">\n <head>\n <meta http-equiv="x-ua-compatible" content="IE=edge" />\n <meta charset="utf-8"/>\n\n <link rel="dns-prefetch" href="//ajax.googleapis.com">\n <link rel="dns-prefetch" href="//ssl.'
source share