Matlab bottleneck

Question

I work with large arrays (~ 6x40 million) and my code shows large bottlenecks. I am an experienced programmer at MatLab, but I don’t know much about internal processes (such as memory, etc.).

My code looks like this (only the main things, of course, all variables are initialized, especially arrays in loops, I just don't want to bomb everyone with the code):

First I read the file,

disp('Point cloud import and subsampling') tic fid=fopen(strcat(Name,'.dat')); C=textscan(fid, '%d%d%f%f%f%d'); %<= Big! fclose(fid); 

then create arrays from the content,

  y=C{1}(1:Subsampling:end)/Subsampling; x=C{2}(1:Subsampling:end)/Subsampling; %... and so on for the other rows clear C %No one wants 400+ millon doubles just lying around. 

And clear the cell array ( 1 ) and create some images and arrays with new values

 for i=1:length(x) PCImage(y(i)+SubSize(1)-maxy+1,x(i)+1-minx)=Reflectanse(i); PixelCoordinates(y(i)+SubSize(1)-maxy+1,x(i)+1-minx,:)=Coordinates(i,:); end toc 

Everything works more or less smoothly while here, but then I manipulate some arrays

 disp('Overlap alignment') tic PCImage=PCImage(:,[1:maxx/2-Overlap,maxx/2:end-Overlap]); %-30 overlap? PixelCoordinates=PixelCoordinates(:,[1:maxx/2-Overlap,maxx/2:end-Overlap],:); Sphere=Sphere(:,[1:maxx/2-Overlap,maxx/2:end-Overlap],:); toc 

and this is a big bottleneck, but in the next step it gets worse.

 disp('Planar view and point cloud matching') tic CompImage=zeros(max(SubSize(1),PCSize(1)),max(SubSize(2),PCSize(2)),3); CompImage(1:SubSize(1),1:SubSize(2),2)=Subimage; %ExportImage Cyan CompImage(1:SubSize(1),1:SubSize(2),3)=Subimage; CompImage(1:PCSize(1),1:PCSize(2),1)=PCImage; %PointCloudImage Red toc 

Output

Import and subsample cloud point clouds

The elapsed time is 181.157182 seconds.

Overlap Alignment

The elapsed time is 408.750932 seconds.

Alignment of planar view and point cloud

Elapsed time - 719.383807 seconds.

My questions are: will the removal of unused objects such as C in 1 ? (it doesn't look like this)

Am I observing any other important mechanisms or rules of thumb, or is this all too much and should this happen?

Answer 1

When subsref used, Matlab creates a copy of the elements referenced. This can be expensive for large arrays. Often it will be faster to snap vectors like

 res = [a,b,c]; 

This is not possible with the current code, as described above, but if the code can be modified to do this, it can save some time.

EDIT For multidimensional arrays you need to use cat

 CompImage = cat(dim,Subimage,Subimage,PCImage); 

where dim for this example is 3.