How to set a limit on the number of simultaneous requests to a servlet?

Question

I got this servlet that returns a PDF file to a client web browser. We do not want to take risks, if the number of requests is too large, the server will be paralyzed.

We would like to make a way for the application level (program) to set a limit on the number of parallel requests and return an error message to the browser when the limit is reached. We need to do this at the application level, because we have a different servlet container at the development level (tomcat) and production level (websphere).

I must emphasize that I want to control the maximum number of requests instead of a session. A user can send multiple requests to a server with the same session.

Any idea? I thought about using a static counter to track the number of requests, but that would cause a race condition problem.

Answer 1

I would suggest writing a simple Filter servlet. Customize it in web.xml to apply to the path you want to limit the number of simultaneous requests. The code will look something like this:

 public class LimitFilter implements Filter { private int limit = 5; private int count; private Object lock = new Object(); public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { try { boolean ok; synchronized (lock) { ok = count++ < limit; } if (ok) { // let the request through and process as usual chain.doFilter(request, response); } else { // handle limit case, eg return status code 429 (Too Many Requests) // see http://tools.ietf.org/html/rfc6585#page-3 } } finally { synchronized (lock) { count--; } } } } 

Or, alternatively, you can simply put this logic in your HttpServlet . It is a little cleaner and reused as a Filter . You might want to make the limit customizable using web.xml , rather than hard coding it.

Ref :.
Check the definition of the HTTP status code 429 .

Answer 2

I thought about using a static counter to track the number of requests, but that would cause a race condition problem.

If you use AtomicInteger for the counter, you will not have problems with the conditions of the race.

Another way would be to use the Java Executor Framework (shipped with Java 1.5). There you can limit the number of running threads and block a new one once until a new free thread appears.

But I think that the counter will work and will be the easiest solution.

Caution: put the counter again in the finally block!

 //psydo code final AtomicInteger counter; ... while(true) { int v = counter.getValue() if (v > max) return FAILURE; if(counter.compareAndSet(v, v+1)) break; } try{ doStuff(); } finally{ counter.decrementAndGet(); } 

Answer 3

Perhaps you should take a look at Semaphore .

Semaphores are often used to limit the number of threads than access to some (physical or logical) resource.

Or better yet, try to figure out the server settings. This of course depends on the server.

Answer 4

You can use RateLimiter. See this article for an explanation.

Answer 5

If you serve static files, this is unlikely to cause the server to crash. The network bandwidth will be the bottleneck, and it wears out gracefully - when more requests arrive, each of them is still serviced, a little slower.

If you set a hard limit on general requests, be sure to set a limit on IP requests. Otherwise, one bad guy can issue N requests, intentionally read the answers very slowly and completely clog your service. This works even if it dials a number and your server network has a large bandwidth.