Error trying to use parfor (parallel for loop) in MATLAB

Question

I am dealing with a very large matrix and therefore wanted to use parallel computing in MATLAB to run in clusters. Here I created a sparse matrix using:

Ad = sparse(length(con)*length(uni_core), length(con)*length(uni_core)); 

I have a written adj function in which I can populate the Ad matrix. Every time the loop works, from the adj function I get a square symmetric matrix that must be assigned Ad from 3682*(i-1)+1 to 3682 *(i-1)+3682 in the first index and similarly in the second index. This is shown here:

 parfor i = 1:length(con) Ad((3682*(i-1))+1:((3682*(i-1))+3682), ... (3682*(i-1))+1:((3682*(i-1))+3682)) = adj(a, b, uni_core); end 

In a normal cycle, it works without any problems. But in parfor in parallel computing, I get the error message that there is a problem with using truncated arrays with parfor .

Answer 1

The outputs from the PARFOR loops must be either reduction variables (for example, summation calculation) or “chopped”. See this page in the document for details.

In your case, you are trying to generate a “clipped” output, but your indexing expression is too complicated for PARFOR. In PARFOR, chopped output must be indexed: a loop variable for one index and some constant expression for other indices. The constant expression must be either : end , or a literal. The following example shows several cut outputs:

 x3 = zeros(4, 10, 3); parfor ii = 1:10 x1(ii) = rand; x2(ii,:) = rand(1,10); x3(:,ii,end) = rand(4,1); x4{ii} = rand(ii); end 

In your case, your indexing expression in Ad is too complicated to handle PARFOR. Probably the easiest thing you can do is return the calculations as an array of cells and then enter them in Ad on the host side using a regular FOR loop, for example:

 parfor i = 1:length(con) tmpout{i} = ....; end for i = 1:length(con) Ad(...) = tmpout{i}; end 

Answer 2

Edric has already explained why you get the error message, but I wanted to make another suggestion for the solution. The Ad matrix you create consists of a series of 3682-x-3682 blocks on the main diagonal with zeros all over the world. One solution is to create your blocks in the PARFOR circuit , storing them in an array of cells. Then you can combine them all into one matrix with a call to the BLKDIAG function:

 cellArray = cell(1,length(con)); %# Preallocate the cell array parfor i = 1:length(con) cellArray{i} = sparse(adj(a,b,uni_core)); %# Compute matrices in parallel end Ad = blkdiag(cellArray{:}); 

The resulting Ad matrix will be sparse because each block has been converted to a sparse matrix before being placed in an array of cells.