Out of memory in Matlab - how to do an in-place operation on matrix elements?

Question

I am loading a fairly large matrix into Matlab. Downloading this matrix is ​​pushing Matlab to its limits - but it does.

Then I do the following and I get an error from memory.

data( :, 2:2:end, :, : ) = - data( :, 2:2:end, :, : );

Is Matlab allocating a new matrix for this operation? I would suggest that this operation does not require additional memory. How to make Matlab be more efficient for this?

Bonus question:

'data = permute (data, [1 2 3 4 5 12 8 7 6 9 10 11]);

Can Matlab do this on the spot?

Answer 1

There are several limitations (in addition to those from the Loren block John quotes):

• Your code should work inside a function
• You must not have other aliases for "data"

Alias ​​are important and potentially difficult to understand. MATLAB uses copy-on-write, which means that when you call the function, the argument you pass is not duplicated immediately, but can be copied if you change it inside the function. For example, consider

 x = rand(100); y = myfcn(x); % with myfcn.m containing: function out = myfcn(in) in(1) = 3; out = in * 2; end 

In this case, the variable x is passed to myfcn . MATLAB has semantics of values, so any changes to the input argument in should not be displayed in the workspace of the call. So, the first line of myfcn causes the in argument to become a copy of x , and not just an alias. Consider what happens with try / catch - it could be a killer in place, because MATLAB should keep the values ​​if you are mistaken. In the following:

 % consider this function function myouterfcn() x = rand(100); x = myfcn(x); end % with myfcn.m containing function arg = myfcn( arg ) arg = -arg; end 

then this should get in-place optimization for x in myouterfcn . But the following cannot:

 % consider this function function myouterfcn() x = rand(100); x = myfcn(x); end % with myfcn.m containing function arg = myfcn( arg ) try arg = -arg; catch E disp( 'Oops.' ); end end 

Hope this info helps ...

Answer 2

Matlab supports field operations. Here's a discussion: http://blogs.mathworks.com/loren/2007/03/22/in-place-operations-on-data/ . Sorry, I can’t help anymore.

Answer 3

I don't think there is a good way to find out what MATLAB really does under the sheets. I would recommend you:

• Make sure to clear all variables that are not used before trying to perform the operation.
• If you still do not have enough memory, write and compile a simple mex file to complete the operation. With a massive array and your special requirement, it is likely to be faster than the MATLAB approach.

Similarly, you can write your own permutation algorithm as a .mex file in C if you need it to run in place (i.e. you run out of memory again).