Tomcat 7 Async Processing

Question

I am wondering how Tomcat 7 implements asynchronous processing. I understand that the request stream is immediately returned, allowing the request stream to immediately listen to the new request and respond to it.

How is an async request handled? Is there a separate thread pool that handles asynchronous requests? I assume that I / O blocking is handled using something like java.nio.Selector for performance. What about threads that block CPU calculations?

Answer 1

You mix different concepts. You must distinguish between:

• Asynchronous request processing in accordance with Servlet 3.0 ; An API that allows you to separate an incoming servlet request from a web container's thread pool. It does not create streams on the fly. It provides the user with an interface for implementing the right multi-threaded solution. This does not apply to non-blocking IO.
• Thread pool provides a mechanism for receiving and managing flows in a web container. When it comes to asynchronous request, you have 2 options. You can define your own ExecutorService and use it to further process the request, or you can create a new Runnable and send it to the received AsyncContext by calling AsyncContext.start() . For Tomcat, the latter approach uses the Tomcat thread pool defined in server.xml .
• Non-blocking IO (NIO) ; Although it is asynchronous, it is a completely different story. It refers to non-blocking I / O, such as a disk or network I / O. If you want to enable NIO to handle HTTP requests, look at the Tomcat documentation .

---

The example below describes how it can work. It uses only one thread for work tasks. If you run it from two different browsers in parallel, the output is as follows (I use my own log):

  DATE THREAD_ID LEVEL MESSAGE 2011-09-03 11:51:22.198 +0200 26 I: >doGet: chrome 2011-09-03 11:51:22.204 +0200 26 I: <doGet: chrome 2011-09-03 11:51:22.204 +0200 28 I: >run: chrome 2011-09-03 11:51:27.908 +0200 29 I: >doGet: firefox 2011-09-03 11:51:27.908 +0200 29 I: <doGet: firefox 2011-09-03 11:51:32.227 +0200 28 I: <run: chrome 2011-09-03 11:51:32.228 +0200 28 I: >run: firefox 2011-09-03 11:51:42.244 +0200 28 I: <run: firefox 

You see that the doGet methods end immediately and the worker is still working. 2 test requests: http://localhost:8080/pc/TestServlet?name=chrome and http://localhost:8080/pc/TestServlet?name=firefox .

A simple servlet example

 @WebServlet(asyncSupported = true, value = "/TestServlet", loadOnStartup = 1) public class TestServlet extends HttpServlet { private static final Logger LOG = Logger.getLogger(TestServlet.class.getName()); private static final long serialVersionUID = 1L; private static final int NUM_WORKER_THREADS = 1; private ExecutorService executor = null; @Override public void init() throws ServletException { this.executor = Executors.newFixedThreadPool(NUM_WORKER_THREADS); } protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { final String name = request.getParameter("name"); LOG.info(">doGet: " + name); AsyncContext ac = request.startAsync(); // obtain async context ac.setTimeout(0); // test only, no timeout /* Create a worker */ Runnable worker = new TestWorker(name, ac); /* use your own executor service to execute a worker thread (TestWorker) */ this.executorService.execute(worker); /* OR delegate to the container */ // ac.start(worker); LOG.info("<doGet: " + name); } } 

... and TestWorker

 public class TestWorker implements Runnable { private static final Logger LOG = Logger.getLogger(TestWorker.class.getName()); private final String name; private final AsyncContext context; private final Date queued; public TestWorker(String name, AsyncContext context) { this.name = name; this.context = context; this.queued = new Date(System.currentTimeMillis()); } @Override public void run() { LOG.info(">run: " + name); /* do some work for 10 sec */ for (int i = 0; i < 100; i++) { try { Thread.sleep(100); } catch (InterruptedException e) { throw new RuntimeException(e); } } ServletResponse response = this.context.getResponse(); response.setContentType("text/plain"); try { PrintWriter out = response.getWriter(); out.println("Name:\t\t" + this.name); out.println("Queued:\t\t" + this.queued); out.println("End:\t\t" + new Date(System.currentTimeMillis())); out.println("Thread:\t\t" + Thread.currentThread().getId()); out.flush(); } catch (IOException e) { throw new RuntimeException(e); } this.context.complete(); LOG.info("<run: " + name); } }