Parallel Processing in Scala

I am trying to use concurrent programming in Scala. Based on this example here StackOverflow, I made a program based on Problem 1 Project Euler. I try three methods: the first is simple execution, without paralysis. the second uses the java.util.concurrency API through Executors and Callables. The third, based on the page mentioned above, using Scala. Futures. My goal is to compare runtime.

This is the code:

package sandbox import java.util.concurrent._ import scala.actors._ object TestPool { def eval(n: Int): Boolean = (n % 3 == 0) || (n % 5 == 0) def runSingle(max: Int): Int = (1 until max).filter(eval(_)).foldLeft(0)(_ + _) def runPool(max: Int): Int = { def getCallable(i: Int): Callable[Boolean] = new Callable[Boolean] { def call = eval(i) } val pool = Executors.newFixedThreadPool(5) val result = (1 until max).filter(i => pool.submit(getCallable(i)).get).foldLeft(0)(_ + _) pool.shutdown pool.awaitTermination(Math.MAX_LONG, TimeUnit.SECONDS) result } def runFutures(max: Int): Int = (1 until max).filter(i => Futures.future(eval(i)).apply).foldLeft(0)(_ + _) /** * f is the function to be runned. it returns a Tuple2 containing the sum and the * execution time. */ def test(max: Int, f: Int => Int): (Int, Long) = { val t0 = System.currentTimeMillis val result = f(max) val deltaT = System.currentTimeMillis - t0 (result, deltaT) } def main(args : Array[String]) : Unit = { val max = 10000 println("Single : " + test(max, runSingle)) println("Pool : " + test(max, runPool)) println("Futures: " + test(max, runFutures)) } } 

Here are the results:

max = 10:

  • Single: (23.31)
  • Pool: (23.16)
  • Futures: (23.31)

max = 100:

  • Single: (2318.33)
  • Pool: (2318.31)
  • Futures: (2318.55)

max = 1000:

  • Single: (233,168.42)
  • Pool: (233,168,111)
  • Futures: (233,168,364)

max = 10000:

  • Single: (23331668,144)
  • Pool: (23331668,544)
  • Futures: ... I canceled the performance after 3 minutes.

Obviously, I could not use the concurrency API from Java and Scala correctly. So I ask: Where is my mistake? What is a more suitable form of using Concurrency? What about Scala Actors? Is it possible to use them?

+4
source share
2 answers

What result do you expect? Do you expect one of these methods to be better than the other? Do you expect the program to scale differently for different execution methods?

How many cores does your machine have? If you have only one core, you should expect time to increase linearly with the work that needs to be done. What does your CPU usage look like during the runs? Are numbers repeatable?

You also did not take into account the influence of the JVM Hotspot warm-up time, which can cause significant problems for such micro-tests.

+1
source

I assume you are using Scala 2.7. In principle, the filter and map on the Range (the result is 1 until max ) is non-strict, that is, it will be calculated by request, and it will be calculated every time you try to get results to it.

Try this for example:

 val y = (1 to 10).filter{x => println("Filtering "+x); x % 2 == 0}.map{x => println("Mapping "+x); x * 2} println(y(0)) println(y(1)) println(y(2)) println(y(0)) 

The result, in any case, is that your parallel material works in series. Add .force to the range and everything will be fine.

+1
source

Source: https://habr.com/ru/post/1300128/


All Articles