Long poll in Spring

Question

We have a somewhat unique case where we need to interact with an external API, which requires us to take a long survey of their endpoint for what they call real-time events.

The fact is that at any given time up to 80,000 people / devices can be involved, affecting this endpoint, listening to events, 1 connection per device / person.

When a client makes a request to our Spring service for a long poll for events, our service, in turn, calls an asynchronous call to an external API for a long poll for events. The external API defined a minimum long polling timeout, which can be set to 180 seconds.

So, we have a situation where the thread pool with the queue will not work, because if we have a thread pool with something like (5 minutes, 10 max, 10 queues), then 10 threads that will work can be spotlighted and 10 in the queue will not get a chance until one of the current 10 is completed.

We need a feed or failure (we will put load balancers, etc.), but we do not want to leave the client hanging without an actual survey.

We studied the use of DeferredResult for this and returned it from the controller.

Something to the tune

@RequestMapping(value = "test/deferredResult", method = RequestMethod.GET) DeferredResult<ResponseEntity> testDeferredResult() { final DeferredResult<ResponseEntity> deferredResult = new DeferredResult<ResponseEntity>(); CompletableFuture.supplyAsync(() -> testService.test()).whenCompleteAsync((result, throwable) -> deferredResult.setResult(result)); return deferredResult; } 

I am wondering if I am on the right track, and also have to provide the executor and which executor (and configuration) the CompletableFuture.supplyAsync() method in order to best accomplish our task.

I read various articles, posts, etc., and I want to find out if anyone has knowledge that can help in our specific situation.

Answer 1

The problem you are describing is not like the one that can be solved well if you use I / O lock. Thus, you are on the right track because DeferredResult allows you to create a result using any thread without blocking the servlet container stream.

As for calling the API with a long pool up, you also need a NIO solution. If you use the Netty client, you can manage several thousand sockets with a single thread. When the NIO selector in Netty detects the data, you will receive a channel callback and ultimately delegate the stream to the Netty workflow pool, and you can call deferredResult.setResult . If you do not block IO, the work pool usually has a size after the number of CPU cores, otherwise you may need more threads.

There are a number of problems.

• You may need more than one server (or network interface) since there are only 65K ports.
• Sockets in Java do not have write timeouts, so if the client refuses to read data from the socket, and you send more data than your socket buffer, you block the Netty workflow, and then everything stops (slow reverse Lori attack). This is a classic problem in large asynchronous settings and one of the reasons for using frameworks like Hystrix (Netflix).