Python - should not be redirected to only one URL

Question

I am wondering how you can prevent urllib2 after requesting a redirect to my chosen URL. I found this piece of code while browsing, but it seems to work globally, and I just want it to disable redirection to a specific URL:

import urllib2 class RedirectHandler(urllib2.HTTPRedirectHandler): def http_error_302(self, req, fp, code, msg, headers): result = urllib2.HTTPError(req.get_full_url(), code, msg, headers, fp) result.status = code return result http_error_301 = http_error_303 = http_error_307 = http_error_302 opener = urllib2.build_opener(RedirectHandler()) webpage = opener.open('http://www.website.com').geturl() print webpage 

I should also mention that I am requesting a URL using urllib.urlopen ('site.com') and I want the first redirect to be allowed, for example, redirecting the site.com/redirect site, but then it tries redirect again from site.com/redirect to site.com/secondredirect. I would like the script to recognize "secondredirect" in the url and terminate this request. Hopefully I explained all this well and hopefully see some answers as I spent hours on hours trying to figure it out: headache:

Answer 1

It is not possible to disable call forwarding for each request using urllib2. You have the option to use httplib , which is usually the low-level module used by modules such as urllib2.

 >>> import httplib >>> conn = httplib.HTTPConnection("www.bogosoft.com") >>> conn.request("GET", "") >>> r1 = conn.getresponse() >>> print r1.status, r1.reason 301 Moved Permanently >>> print r1.getheader('Location') http://www.bogosoft.com/new/location 

Another option is to use the Python Requests library, which gives you finer control over how to handle redirects . Requests are the best choice here, in my opinion, if you have the opportunity to use another library.

Answer 2

 import urllib.request class RedirectFilter(urllib.request.HTTPRedirectHandler): def redirect_request(self, req, fp, code, msg, hdrs, newurl): if newurl.endswith('.jpg'): return None # do not redirect, HTTPError will be raised return urllib.request.HTTPRedirectHandler.redirect_request(self, req, fp, code, msg, hdrs, newurl) opener = urllib.request.build_opener(RedirectFilter) opener.open('http://example.com/') 

This is for Python 3. For Python 2, replace urllib.request with urllib2 .