What operations make parallel code run slow?

Question

Consider this: The Concurrency Hitchhiking Guide , namely the Amdahl Law section, is the idea that a parallel program runs only faster than its slowest part, and that the larger the parallel program from the start, the faster it will be after introducing more cores , I am wondering: how can I guarantee that I write code that will be as parallel as possible from the very beginning? How can I guarantee that my code takes full advantage of adding multiple cores? And also, what kinds of operations can cause the code to not be parallel or parallel code to be slow? Of course, code examples will be appreciated.

Answer 1

Not all code, by its nature, can execute in parallel.

Parallel execution simultaneously means simultaneously. The code can only work at the same time, if it does not rely on the final result, another code working next to it. If you do this equation:

((((x+y)+z)+a)*b)

Each bracket must be developed before the next step, so operations cannot be performed sequentially. To make the program parallel, it is important to determine when there is a big task that can be broken down into parts that can be completed at the same time.

Consider the summation, I need to add 100,000 numbers.

sum = 0;
for (i = 0; i < 100000; i++) {
   sum += numbers[i];
}

, a + b + c + d (a + b + c) + d, a + (b + c) + d, (a + b) + (c + d) .. - , , .

, , :

sumA = 0;
for (i = 0; i < 100000 / 2; i++) {
   sumA += numbers[i];
}

sumB = 0;
for (i = 100000 / 2; i < 100000; i++) {
   sumB += numbers[i];
}

sum = sumA + sumB;

, , .

, (cpu/ node/machine), . .

, . , , , . , , , (, ) - .

, . , , , , .

Answer 2

( ) - (: Haskell, Lisp ..)

OO-, , ( Java: ) , , , .

, , ! .

!

Answer 3

; .

, Erlang EVM. EVM . , , .

: http://www.erlang.org/doc/efficiency_guide/processes.html

SMP, Erlang- . Erlang Erlang , . , .

. Erlang/OTP . , - node / . ( ), , " ". Erlang/OTP , .

, (.. : foldl/3) . - .

, . , , , , IO. , , parallelism, .

.

• , .
• , .
• , .
• Erlang/OTP. , .
• " " .
• , .
• .

Answer 4

, . , , , , . . .

, . - , , . , . , . parallelism, .

, . , . parallelism , , , , ( - -). Sharding .

Answer 5

, , ?

, . , , . . , , 2 . , "" . , (., , wikipedia Carry Select Adders). , , , , 99%. , . , . , ., , Branch Prediction .

, . , , , "" .

, ?

, , - . , . , , . : , , - , , , . , , .

parallelism. FP , , . Haskell - , , , - par.

, , . .

Erlang , Erlang , Erlang parallelism. Erlang , , . . Erlangs - : - - , , -.

: ​​ , , . , , , .

. " ", , Amdahl Gustafsson , .