Is there an easy way to request a url in python and NOT follow a redirect?

Question

Looking at the source of urllib2, the easiest way to do this seems to be to subclass the HTTPRedirectHandler and then use build_opener to override the default HTTPRedirectHandler, but it seems like a lot of (relatively complicated) work to make it seem like it should be pretty simple.

Answer 1

Here is the Requests method:

import requests r = requests.get('http://github.com', allow_redirects=False) print(r.status_code, r.headers['Location']) 

Answer 2

Dive Into Python has a good chapter on handling redirects using urllib2. Another solution is httplib .

 >>> import httplib >>> conn = httplib.HTTPConnection("www.bogosoft.com") >>> conn.request("GET", "") >>> r1 = conn.getresponse() >>> print r1.status, r1.reason 301 Moved Permanently >>> print r1.getheader('Location') http://www.bogosoft.com/new/location 

Answer 3

This is the urllib2 handler that will not follow the redirects:

 class NoRedirectHandler(urllib2.HTTPRedirectHandler): def http_error_302(self, req, fp, code, msg, headers): infourl = urllib.addinfourl(fp, headers, req.get_full_url()) infourl.status = code infourl.code = code return infourl http_error_300 = http_error_302 http_error_301 = http_error_302 http_error_303 = http_error_302 http_error_307 = http_error_302 opener = urllib2.build_opener(NoRedirectHandler()) urllib2.install_opener(opener) 

Answer 4

I suppose that would help

 from httplib2 import Http def get_html(uri,num_redirections=0): # put it as 0 for not to follow redirects conn = Http() return conn.request(uri,redirections=num_redirections) 

Answer 5

The redirections keyword in the httplib2 request httplib2 is a red herring. Instead of returning the first request, it throws a RedirectLimit exception if it receives a redirect status code. To return the original response, you need to set follow_redirects to False of the Http object:

 import httplib2 h = httplib2.Http() h.follow_redirects = False (response, body) = h.request("http://example.com") 

Answer 6

The second pointer to Immersion in Python . Here's an implementation using urllib2 redirect handlers, more work than it should be? Maybe with a shrug.

 import sys import urllib2 class RedirectHandler(urllib2.HTTPRedirectHandler): def http_error_301(self, req, fp, code, msg, headers): result = urllib2.HTTPRedirectHandler.http_error_301( self, req, fp, code, msg, headers) result.status = code raise Exception("Permanent Redirect: %s" % 301) def http_error_302(self, req, fp, code, msg, headers): result = urllib2.HTTPRedirectHandler.http_error_302( self, req, fp, code, msg, headers) result.status = code raise Exception("Temporary Redirect: %s" % 302) def main(script_name, url): opener = urllib2.build_opener(RedirectHandler) urllib2.install_opener(opener) print urllib2.urlopen(url).read() if __name__ == "__main__": main(*sys.argv) 

Answer 7

The shortest way is

 class NoRedirect(urllib2.HTTPRedirectHandler): def redirect_request(self, req, fp, code, msg, hdrs, newurl): pass noredir_opener = urllib2.build_opener(NoRedirect())