Akka HTTP and long-running requests

Question

We have an API implemented in the bare bones of Scala Akka HTTP - a pair of routes for heavy computing (CPU and memory intensity). No clustering - everything works on the same muscular machine. The calculation is quite heavy - it may take more than 60 seconds to complete one isolated request. And we don’t really care about speed. There is no I / O lock, just processor processing.

When I started to test the performance, an interesting pattern appeared: let's say requests A1, A2, ..., A10. They use resources quite heavily, and it turns out that Akka will return HTTP 503 for A5-A10 requests that are upside down. The problem is that this calculation still works, although no one can find the result.

And from there we see a cascading performance failure: A11-A20 requests go to a server still working on A5-A10 requests . Obviously, these new requests also have a chance of exceeding - even higher if the server is more busy. Thus, some of them will work by the time Akka starts the timeout, making the server even more loaded and slow, and then starts a new batch of requests ... therefore, starting the system a bit, you see that almost all requests after certain point start does not work with timeouts. And after you stop the download that you see in the logs, some requests are still being processed.

I tried to perform the calculation in a separate ExecutionContext, as well as the system manager, trying to make it completely asynchronous (via Future composition), but the result is still the same. Long jobs make the server so busy that in the end, almost every request fails.

A similar case is described in https://github.com/zcox/spray-blocking-test , but the focus is shifted - it /pingdoes not matter for us as a more or less stable responsibility for an endpoint that processes long-term requests.

Question : how do I configure my application to better interrupt hanging requests? I can tolerate some small percentage of failed requests under heavy load, but interrupting the entire system to stop after a few seconds is unacceptable.

Answer 1

Akka HTTP , . , , , . , - , TBH, Akka HTTP.

, - , , , .

, (. " Scala?" ). -, Thread.interrupt() . Deadline Promise , , , :

// in the HTTP server class:
val responseTimeout: Duration = 30.seconds

val routes = 
  path("slowComputation") {
    complete {
      val responseTimeoutDeadline: Deadline = responseTimeout.fromNow
      computeSlowResult(responseTimeoutDeadline)
    }
  }

// in the processing code:
def computeSlowResult(responseDeadline: Deadline): Future[HttpResponse] = Future {
  val gatherInputs: List[_] = ???
  gatherInputs.fold(0) { (acc, next) =>

    // check if the response has timed out
    if (responseDeadline.isOverdue())
      throw new TimeoutException()

    acc + next // proceed with the calculation a little
  }
}

( Promise , Deadline. , .)

Answer 2

spray-blocking-test , , , Akka HTTP. , :

application.conf

blocking-io-dispatcher {
  type = Dispatcher
  executor = "thread-pool-executor"
  thread-pool-executor {
    fixed-pool-size = 16
  }
  throughput = 1
}

---

complete {
  Try(new URL(url)) match {
    case scala.util.Success(u) => {
      val src = Source.fromIterator(() => parseMovies(u).iterator)

      src
        .via(findMovieByTitleAndYear)
        .via(persistMovies)
        .completionTimeout(5.seconds)
        .toMat(Sink.fold(Future(0))((acc, elem) => Applicative[Future].map2(acc, elem)(_ + _)))(Keep.right)
        // run the whole graph on a separate dispatcher
        .withAttributes(ActorAttributes.dispatcher("blocking-io-dispatcher"))
        .run.flatten
        .onComplete {
            _ match {
               case scala.util.Success(n) => logger.info(s"Created $n movies")
               case Failure(t) => logger.error(t, "Failed to process movies")
            }
        }

      Accepted
    }
    case Failure(t) => logger.error(t, "Bad URL"); BadRequest -> "Bad URL"
  }
}

, .

:

http://doc.akka.io/docs/akka/current/scala/dispatchers.html http://blog.akka.io/streams/2016/07/06/threading-and-concurrency-in-akka-streams-explained