How does a sequential loop work faster than a parallel loop in C #?

Question

I tried a very minimal example:

using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.Threading; using System.Collections.Concurrent; using System.Diagnostics; namespace TPLExample { class Program { static void Main(string[] args) { int[] dataItems = new int[100]; double[] resultItems = new double[100]; for (int i = 0; i < dataItems.Length; ++i) { dataItems[i] = i; } Stopwatch stopwatch = new Stopwatch(); stopwatch.Reset(); stopwatch.Start(); Parallel.For(0, dataItems.Length, (index) => { resultItems[index] = Math.Pow(dataItems[index], 2); }); stopwatch.Stop(); Console.WriteLine("TPL Time elapsed: {0}", stopwatch.Elapsed); stopwatch.Reset(); stopwatch.Start(); for (int i = 0; i < dataItems.Length; ++i) { resultItems[i] = Math.Pow(dataItems[i], 2); } stopwatch.Stop(); Console.WriteLine("Sequential Time elapsed: {0}", stopwatch.Elapsed); WaitForEnterKey(); } public static void WaitForEnterKey() { Console.WriteLine("Press enter to finish"); Console.ReadLine(); } public static void PrintMessage() { Console.WriteLine("Message printed"); } } } 

The output was:

 TPL Time elapsed: 00:00:00.0010670 Sequential Time elapsed: 00:00:00.0000178 Press enter to finish 

Sequential loop faster than TPL! How is this possible? In my opinion, the calculation inside Parallel.For will be done in parallel, should it be faster?

Answer 1

Simply put: in order to repeat only more than a hundred elements and perform a small mathematical operation, the appearance of new threads and their expectation will lead to an increase in overhead than just looping.

From my point of view, the calculation in Parallel.For will be performed in parallel, should it be faster?

As is usually the case when people make broad statements about computer performance, there are many more variables that play here, and you cannot really make that assumption. For example, inside the for loop you are doing nothing more than Math.Pow , which the processor can execute very quickly. If it was an intensive I / O operation, requiring each thread to wait a long time, or even if it was a series of intensive operations with a processor, you would get more from parallel processing (if you have a multi-threaded processor), But be that as it may, the overhead of creating and synchronizing these threads is much more than any benefit that parallelism can provide.

Answer 2

Processing parallel loops is useful when the operation performed in a loop is relatively expensive. All you do in your example is calculating the exponent, which is trivial. The overhead of multithreading far outweighs the benefits you get in this case.

Answer 3

This code example is a practical proof, the really nice answers above.

I simulated processor intensive work by simply blocking the Thead.Sleep thread.

Conclusion:

• Serial loop - 00: 00: 09.9995500
• Parallel loop - 00: 00: 03.0347901

_

 class Program { static void Main(string[] args) { const int a = 10; Stopwatch sw = new Stopwatch(); sw.Start(); //for (long i = 0; i < a; i++) //{ // Thread.Sleep(1000); //} Parallel.For(0, a, i => { Thread.Sleep(1000); }); sw.Stop(); Console.WriteLine(sw.Elapsed); Console.ReadLine(); } } 

Answer 4

Parallelization overhead is much more than just running Math.Pow 100 times in a row. Others said that.

More importantly, however, memory access is trivial in a serial version, but with a parallel version, threads must exchange memory (resultItems), and such a thing will really kill you, even if you have a million elements.

See page 44 of this excellent Microsoft parallel programming document: http://www.microsoft.com/en-us/download/details.aspx?id=19222 . Here is the MSDN magazine article on the topic: http://msdn.microsoft.com/en-us/magazine/cc872851.aspx