Parallelization Issues in MATLAB

Question

I cannot understand the problem in the following short script, which should compare one CPU calculation with parallelization relative to the calculation time.

Link to full image: LINK

Code:

n = 700; ranksSingle = zeros(1,n); tic for ind = 1:n ranksSingle(ind) = rank(magic(ind)); end toc matlabpool local 4 tic ranks = zeros(1,n); parfor (ind = 1:n) ranks(ind) = rank(magic(ind)); end toc isequal(ranksSingle, ranks) matlabpool close 

I also tried it with matlabpool 2 . As can be seen from the process window, all cores are occupied up to 100% in parallel computing (marked in red).

When performing a calculation with one processor (marked in blue), strange 4 cores are also more busy than before. I would expect only one core to start growing. I searched the Internet to see if perhaps the magic() or rank function is built into parallelism, but as you can read here: http://www.walkingrandomly.com/?p=1894 it is not. So it’s good that these 4 cores are not fully occupied, but still I wonder why all the cores grow.

Secondly, I’m really interested to know the calculation time in the parallel version. I know that there is some overhead by distributing tasks to individual cores, but it should not be so high that in the end there is no use: (

Maybe someone can tell me something :( I'm really stuck with this because I want to speed up some of my for-loops. The second question is: if there is any command that always sets the working size to the number of physical cores that do I have on my computer? (as well as using Hyper Threading, if that's an added benefit?)

Thanks a lot!

Answer 1

If you want to run a parallel task, you must remember that it is bad to have too many fast iterations and it is bad to have too few slow iterations. If you do a million iterations, each of which takes several milliseconds, the overhead from parallelization will destroy any possible gain. If you perform nine iterations that take an hour each and you run it on eight processors in parallel, seven processors will idle for an hour, waiting for iteration # 9 to finish.

So your example is pretty bad for checking the effects of parallelization, since both magic and rank are too fast.

  function testParfor2 tic for i=1:4 pause(1); %# wait for 1 second end toc matlabpool open 4 tic parfor i=1:4 pause(1); %# wait for 1 second end toc Elapsed time is 4.050287 seconds. Elapsed time is 1.534534 seconds. 

Please note that I performed the second parallel task at the same time, but roughly, the result should be reproducible: there is a bit of overhead (note that I did not take into account the time used by matlabpool !), But there is acceleration. If you increase the length of the pause, you should see the same amount of overhead. In addition, you should test your actual loops (try parallelizing the outermost loop, by the way).

To your second question:

  matlabpool open 

Creates as many workers as there are physical cores. Hyperthreading helps you ensure that your computer remains responsive when running a parallel job.

Finally, although magic and rank can be completely incompatible with each other, they can make calls for multi-threaded procedures.