Multiple posts with Httpclient 4.0.3 randomly hanging

Question

Let me explain the situation.

I have a servlet that redirects outgoing GET / POST to another project in another domain (some kind of proxy), whose task is to process it and return some things (params and gif). I am using HttpClient 4.0.3 for this.

There are several GET / POST sent by my application at startup, so I set it once, when ThreadSafeClientConnManager processes several threads this way.

cm_params = new BasicHttpParams(); ConnManagerParams.setMaxTotalConnections(cm_params, 200); ConnPerRouteBean connPerRoute = new ConnPerRouteBean(); HttpHost localhost = new HttpHost("localhost"); connPerRoute.setMaxForRoute(new HttpRoute(localhost), 50); ConnManagerParams.setMaxConnectionsPerRoute(cm_params, connPerRoute); SchemeRegistry schemeRegistry = new SchemeRegistry(); schemeRegistry.register( new Scheme("http", PlainSocketFactory.getSocketFactory(), 80)); cm = new ThreadSafeClientConnManager(cm_params, schemeRegistry); 

Then I create a new HttpClient with these parameters, which should be enough to handle a bunch of the request at the same time. Of course, I do this in public void service () for each GET / POST, but I use the same Httpclient object after creating it.

 httpclient = new DefaultHttpClient(cm, cm_params); 

After that, I create my POST and send it via execute, with all the required parameters and with such triple confirmation.

  HttpPost httpPost = new HttpPost(target+tmpString); httpPost.setHeader("Host", request.getHeader("host")); httpPost.setHeader("User-Agent", request.getHeader("user-agent")); httpPost.setHeader("Accept-Encoding", request.getHeader("accept-encoding")); httpPost.setHeader("Accept", request.getHeader("accept")); ..etc.. UrlEncodedFormEntity urlEncodedFormEntity = new UrlEncodedFormEntity(params); urlEncodedFormEntity.setContentEncoding(HTTP.UTF_8); httpPost.setEntity(urlEncodedFormEntity); HttpResponse response = httpclient.execute(httpPost); 

Finally, I read the stream and processed the entities ...

  OutputStream os = res.getOutputStream(); InputStream is = response.getEntity().getContent(); byte[] buf = new byte[1024]; for(int n;(n=is.read(buf))!=-1;) { os.write(buf, 0, n); } // Make sure to close is.close(); os.close(); // Flush entities just in case EntityUtils.consume(urlEncodedFormEntity); EntityUtils.consume(response.getEntity()); urlEncodedFormEntity.getContent().close(); response.getEntity().getContent().close(); 

So my problem is that the code works fine when I load my page. 4 requests sent (1 GET, 3 POST). Initially, I return some parameters and 1 small gif, which I print on my page.

But as soon as I begin to experience a stress test of my application, I.E. loading the same page in 4-5 tabs, my application seems to accidentally fit when I execute several POSTs at the same time. I thought that I would not have problems, even if I use the same Httpclient object , since I correctly formulated my ThreadSafeClientConnManager (I think?), So that it works with multiple threads.

Does anyone know what I'm doing wrong? If I load my tabs 1 on 1, it does not freeze. Just when I update more than one tab at a time.

Who has the key ?: S (sry english is not my first language ^^;)

Answer 1

@Apache Fan by installing this:

ConnManagerParams.setMaxTotalConnections (cm_params, 200); connPerRoute.setDefaultMaxPerRoute (50);

I doubt that my connection may end, I open 3-4 tabs and end ... maybe 4 POST per tab, so that it does not add up: S

Answer 2

You can take a look at this from the HttpClient API -

ThreadSafeClientConnManager supports the maximum connection limit for each route and in general. By default, this implementation will create no more than two simultaneous connections for each route and no more than 20 connections. For many real-world applications, these restrictions may be too limited, especially if they use HTTP as the transport protocol for their services. However, connection restrictions can be configured using the HTTP settings.

Your application may have ended in an empty connection and you need to increase the pool size.

Answer 3

I recently tried to achieve this, but I had a new problem, for example, when I first sent a request and then sent without a cookie, but the second time httpContext added a cookie automatically but this is a standard way to send, but sometimes your requested server accepts without a cookie, so for this I used INTERCEPTOR. The second time it deletes these automatically added cookies after it works fine, use this httpClient interceptor using the code below:

 httpClient.addRequestInterceptor(new HttpRequestInterceptor() { @Override public void process(HttpRequest request, HttpContext context) throws HttpException, IOException { LOG.info("***************** Entered My Interceptor ****************************"); Header[] headers = request.getAllHeaders(); for (Header eachHeader : headers) { LOG.info("Headers -- Name: {}, Value: {} ", eachHeader.getName(), eachHeader.getValue()); } request.removeHeaders("Cookie"); } }); 

Answer 4

I got the same error. We noticed that the HttpClient is not closing.

What I did, I tried the HttpClientUtils.closeQuietly () method on my unchecked HttpClient and HttpResponse.It helped me with this and it no longer hung. You can also try and experiment with it.