HttpClient 4 - how to capture the last redirect URL

Question

I have a fairly simple HttpClient 4 code that calls HttpGet to get HTML output. HTML is returned with scripts and image locations, all set to local (for example, <img src="/images/foo.jpg"/> ), so I need the url of the call to make them absolute ( <img src="http://foo.com/images/foo.jpg"/> ). Now the problem is that there may be one or two 302 redirects during the call, so the original URL no longer reflects the location of the HTML.

How to get the last URL of the returned content given by all redirected by me (or not necessarily)?

I looked at HttpGet#getAllHeaders() and HttpResponse#getAllHeaders() - I did not find anything.

Edited: HttpGet#getURI() returns the source address of the call

Answer 1

This will be the current URL you can get by calling

  HttpGet#getURI(); 

EDIT: You did not mention how you are doing the redirection. This works for us because we process 302 ourselves.

It looks like you are using DefaultRedirectHandler. We did it. It is difficult to get the current URL. You must use your own context. Here are the relevant code snippets,

  HttpGet httpget = new HttpGet(url); HttpContext context = new BasicHttpContext(); HttpResponse response = httpClient.execute(httpget, context); if (response.getStatusLine().getStatusCode() != HttpStatus.SC_OK) throw new IOException(response.getStatusLine().toString()); HttpUriRequest currentReq = (HttpUriRequest) context.getAttribute( ExecutionContext.HTTP_REQUEST); HttpHost currentHost = (HttpHost) context.getAttribute( ExecutionContext.HTTP_TARGET_HOST); String currentUrl = (currentReq.getURI().isAbsolute()) ? currentReq.getURI().toString() : (currentHost.toURI() + currentReq.getURI()); 

The default redirect did not work for us, so we changed it, but I forgot what the problem was.

Answer 2

In HttpClient 4, if you use LaxRedirectStrategy or any subclass of DefaultRedirectStrategy , this is recommended (see the source code of DefaultRedirectStrategy ):

 HttpContext context = new BasicHttpContext(); HttpResult<T> result = client.execute(request, handler, context); URI finalUrl = request.getURI(); RedirectLocations locations = (RedirectLocations) context.getAttribute(DefaultRedirectStrategy.REDIRECT_LOCATIONS); if (locations != null) { finalUrl = locations.getAll().get(locations.getAll().size() - 1); } 

Since HttpClient 4.3.x, the above code can be simplified as:

 HttpClientContext context = HttpClientContext.create(); HttpResult<T> result = client.execute(request, handler, context); URI finalUrl = request.getURI(); List<URI> locations = context.getRedirectLocations(); if (locations != null) { finalUrl = locations.get(locations.size() - 1); } 

Answer 3

  HttpGet httpGet = new HttpHead("<put your URL here>"); HttpClient httpClient = HttpClients.createDefault(); HttpClientContext context = HttpClientContext.create(); httpClient.execute(httpGet, context); List<URI> redirectURIs = context.getRedirectLocations(); if (redirectURIs != null && !redirectURIs.isEmpty()) { for (URI redirectURI : redirectURIs) { System.out.println("Redirect URI: " + redirectURI); } URI finalURI = redirectURIs.get(redirectURIs.size() - 1); } 

Answer 4

An improved IMHO method based on ZZ Coder's solution is to use the ResponseInterceptor to simply track the last redirect location. This way you do not lose information, for example. after the hashtag. Without an interceptor, you lose the hashtag. Example: http://j.mp/OxbI23

 private static HttpClient createHttpClient() throws NoSuchAlgorithmException, KeyManagementException { SSLContext sslContext = SSLContext.getInstance("SSL"); TrustManager[] trustAllCerts = new TrustManager[] { new TrustAllTrustManager() }; sslContext.init(null, trustAllCerts, new java.security.SecureRandom()); SSLSocketFactory sslSocketFactory = new SSLSocketFactory(sslContext); SchemeRegistry schemeRegistry = new SchemeRegistry(); schemeRegistry.register(new Scheme("https", 443, sslSocketFactory)); schemeRegistry.register(new Scheme("http", 80, new PlainSocketFactory())); HttpParams params = new BasicHttpParams(); ClientConnectionManager cm = new org.apache.http.impl.conn.SingleClientConnManager(schemeRegistry); // some pages require a user agent AbstractHttpClient httpClient = new DefaultHttpClient(cm, params); HttpProtocolParams.setUserAgent(httpClient.getParams(), "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1"); httpClient.setRedirectStrategy(new RedirectStrategy()); httpClient.addResponseInterceptor(new HttpResponseInterceptor() { @Override public void process(HttpResponse response, HttpContext context) throws HttpException, IOException { if (response.containsHeader("Location")) { Header[] locations = response.getHeaders("Location"); if (locations.length > 0) context.setAttribute(LAST_REDIRECT_URL, locations[0].getValue()); } } }); return httpClient; } private String getUrlAfterRedirects(HttpContext context) { String lastRedirectUrl = (String) context.getAttribute(LAST_REDIRECT_URL); if (lastRedirectUrl != null) return lastRedirectUrl; else { HttpUriRequest currentReq = (HttpUriRequest) context.getAttribute(ExecutionContext.HTTP_REQUEST); HttpHost currentHost = (HttpHost) context.getAttribute(ExecutionContext.HTTP_TARGET_HOST); String currentUrl = (currentReq.getURI().isAbsolute()) ? currentReq.getURI().toString() : (currentHost.toURI() + currentReq.getURI()); return currentUrl; } } public static final String LAST_REDIRECT_URL = "last_redirect_url"; 

use it the same as ZZ Coder solution:

 HttpResponse response = httpClient.execute(httpGet, context); String url = getUrlAfterRedirects(context); 

Answer 5

I think it’s easier to find the last URL - use DefaultRedirectHandler.

 package ru.test.test; import java.net.URI; import org.apache.http.HttpResponse; import org.apache.http.ProtocolException; import org.apache.http.impl.client.DefaultRedirectHandler; import org.apache.http.protocol.HttpContext; public class MyRedirectHandler extends DefaultRedirectHandler { public URI lastRedirectedUri; @Override public boolean isRedirectRequested(HttpResponse response, HttpContext context) { return super.isRedirectRequested(response, context); } @Override public URI getLocationURI(HttpResponse response, HttpContext context) throws ProtocolException { lastRedirectedUri = super.getLocationURI(response, context); return lastRedirectedUri; } } 

Code for using this handler:

  DefaultHttpClient httpclient = new DefaultHttpClient(); MyRedirectHandler handler = new MyRedirectHandler(); httpclient.setRedirectHandler(handler); HttpGet get = new HttpGet(url); HttpResponse response = httpclient.execute(get); HttpEntity entity = response.getEntity(); lastUrl = url; if(handler.lastRedirectedUri != null){ lastUrl = handler.lastRedirectedUri.toString(); } 

Answer 6

I found this on the HttpComponents Client Documentation

 CloseableHttpClient httpclient = HttpClients.createDefault(); HttpClientContext context = HttpClientContext.create(); HttpGet httpget = new HttpGet("http://localhost:8080/"); CloseableHttpResponse response = httpclient.execute(httpget, context); try { HttpHost target = context.getTargetHost(); List<URI> redirectLocations = context.getRedirectLocations(); URI location = URIUtils.resolve(httpget.getURI(), target, redirectLocations); System.out.println("Final HTTP location: " + location.toASCIIString()); // Expected to be an absolute URI } finally { response.close(); } 

Answer 7

In version 2.3, Android still does not support the following redirects (HTTP code 302). I just read the location header and booted up again:

 if (statusCode != HttpStatus.SC_OK) { Header[] headers = response.getHeaders("Location"); if (headers != null && headers.length != 0) { String newUrl = headers[headers.length - 1].getValue(); // call again the same downloading method with new URL return downloadBitmap(newUrl); } else { return null; } } 

There is no circular redirection protection here, so be careful. Blog Details Follow 302 Redirects with AndroidHttpClient

Answer 8

Here's how I managed to get the redirect URL:

 Header[] arr = httpResponse.getHeaders("Location"); for (Header head : arr){ String whatever = arr.getValue(); } 

Or, if you are sure that there is only one redirect location, do the following:

 httpResponse.getFirstHeader("Location").getValue();