Akka Worker Pool

Question

As described in the akka thread documentation, I tried to create a pool of workers (threads):

def balancer[In, Out](worker: Flow[In, Out, NotUsed], workerCount: Int): Flow[In, Out, NotUsed] = { import GraphDSL.Implicits._ Flow.fromGraph(GraphDSL.create() { implicit b => val balancer = b.add(Balance[In](workerCount)) val merge = b.add(Merge[Out](workerCount)) for (_ <- 1 to workerCount) { balancer ~> worker ~> merge } FlowShape(balancer.in, merge.out) }) } 

then I used this function to start the thread in parallel:

 def main(args: Array[String]) { val system = ActorSystem() implicit val mat = ActorMaterializer.create(system) val flow = Flow[Int].map(e => { println(e) Thread.sleep(1000) // 1 second e }) Source(Range.apply(1, 10).toList) .via(balancer(flow, 3)) .runForeach(e => {}) } 

I get the expected output 1, 2, 3, 4, 5, 6, 7, 8, 9 , but the numbers are displayed at a speed of 1 per second (without parallelism). What am I doing wrong?

Answer 1

As Endre Varga pointed out, the stream itself must be marked with .async .

But even then, the behavior is not deterministic, since the asynchronous stages have a default buffer size of 16, and the balancer can send all messages to the same worker.

As a result, balancer ~> worker.async.addAttributes(Attributes.inputBuffer(1, 1)) ~> merge will result in the desired behavior.

For the answer asked by the project member, see: https://github.com/akka/akka/issues/20146#issuecomment-201381356

Answer 2

The documents in this section are outdated, which will be fixed in the next version. Basically all you need to do is call .async on the thread itself. By doing this, you can draw a “box” around the stream (which you can imagine as a box with one input and output ports), which will prevent merging in this field. By doing this, in fact, all the workers will be dedicated to the actors. The rest of the schedule (translation and merging stages) will be shared by another actor (they will not work on individual actors, the asynchronous block protects only the stream, everything outside will still be burned).