Move 2D array values, knowing new coordinates with mask

Question

I would like to "move" the elements of a 2D array to the new coordinates, which are stored in two other arrays. I want to automate this because my arrays are actually large (400x200x100). Some values ​​will not find its coordinates and will not be used. Some of these coordinates are masked, which I indicated in the example below using the value 0. If the coordinate is masked, the elements in the array that I want to shuffle will not be used.

import numpy as np #My new coordinates in X and Y directions mx = np.array([[ 1., 2., 3., 4., 0.], [ 1., 2., 3., 4., 0.], [ 1., 2., 3., 4., 0.], [ 1., 2., 3., 4., 0.], [ 1., 2., 3., 4., 0.]]) my = np.array([[ 0., 2., 2., 2., 2.], [ 0., 3., 3., 3., 3.], [ 0., 4., 4., 4., 4.], [ 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0.]]) #The array with values to move IRtest = np.array([[-0.07383495, -0.08606554, -0.08480594, -0.08099556, -0.08218414], [-0.07866761, -0.08373 , -0.08253587, -0.08106102, -0.08220205], [-0.07727436, -0.08271511, -0.0807254 , -0.07832416, -0.08021686], [-0.07612349, -0.08190446, -0.07996929, -0.07842754, -0.08024891], [-0.07488144, -0.08150557, -0.08038229, -0.07895656, -0.07997815]]) #Creation of zeros array to get new array b = np.zeros((5,5)) # I tried this but it doesn't work... for i in range(IRtest.shape[0]): for j in range(IRtest.shape[1]): b[my[i,j], mx[i,j]] = IRtest[i,j] plt.imshow(b) plt.colorbar() plt.show() 

So, the expected array looks like this:

 array_expected = np.array([[-0.08271511, -0.0807254 , -0.07832416, -0.08021686, 0], [-0.08190446, -0.07996929, -0.07842754, -0.08024891, 0], [-0.08150557, -0.08038229, -0.07895656, -0.07997815, 0], [0, 0, 0, 0, 0], [0, 0, 0, 0, 0]]) 

----- EDIT LATER -----------------

Better in this "orientation":

 for i in range(IRtest.shape[0]): for j in range(IRtest.shape[1]): b[j, i] = IRtest[my[j,i],mx[j,i]] 

And I realized:

 array([[-0.08606554, -0.0807254 , -0.07832416, -0.08021686, -0.07727436], [-0.08606554, -0.07996929, -0.07842754, -0.08024891, -0.07612349], [-0.08606554, -0.08038229, -0.07895656, -0.07997815, -0.07488144], [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495], [-0.08606554, -0.08480594, -0.08099556, -0.08218414, -0.07383495]]) 

So, the last problem is to handle masked values ​​...

So I'm trying:

 mask_mx = np.array([[False, False, False, False, True], [False, False, False, False, True], [False, False, False, False, True], [False, False, False, False, True], [False, False, False, False, True]], dtype=int) mask_my = np.array([[True, False, False, False, False], [True, False, False, False, False], [True, False, False, False, False], [True, True, True, True, True], [True, True, True, True, True]], dtype=int) mx3 = np.where(mask_mx, 'nan', mx) my3 = np.where(mask_my, 'nan', my) for i in range(IRtest.shape[0]): for j in range(IRtest.shape[1]): b[j, i] = IRtest[my3[j,i],mx3[j,i]] 

But I get the error below, she doesn't like 'nan' as coordinates: invalid literal for int () with base 10: 'nan'

Answer 1

You probably want to use -1 as a mask, not 0, so you can access index 0 in IRtest.

 mx = np.array([[ 1., 2., 3., 4., -1.], [ 1., 2., 3., 4., -1.], [ 1., 2., 3., 4., -1.], [ 1., 2., 3., 4., -1.], [ 1., 2., 3., 4., -1.]]) my = np.array([[ 2., 2., 2., 2., 2.], [ 3., 3., 3., 3., 3.], [ 4., 4., 4., 4., 4.], [ -1., -1., -1., -1., -1.], [ -1., -1., -1., -1., -1.]]) b = np.zeros_like(IRtest) for i in range(IRtest.shape[0]): for j in range(IRtest.shape[1]): b[j, i] = IRtest[my[j,i],mx[j,i]]*(mx[j,i]!=-1)*(my[j,i]!=-1) 

Answer 2

Your code really works, but your coordinate arrays are not what you mean. Analyze your double loop:

During the iteration, where i is 2 and j is 1, you are effectively handling the case:

 b[my_test[2,1], mx_test[2,1]] = IRtest[2,1] 

IRtest[2,1] - -0.08271511 . my[2,1] is 4 , and mx[2,1] is 2 , so the line above comes down to the following:

 b[4, 2] = -0.08271511 

which is what you see in the output.

Basically the problem is that you have to change your arrays of indices my and mx to get the desired result.

Answer 3

Thanks to numpy fancy-indexing, a loop is not needed, and in fact your code will run much faster without it.

Also note that:

• your index arrays are best represented as int arrays (fantastic indexing won't work with floats)
• best represented as bool arrays (which allow logical operations to be performed as bitwise operations, are more memory efficient and much faster).

Therefore, some pre-processing is required:

 mx = np.array(mx, dtype=int) my = np.array(my, dtype=int) mask_mx = np.array(mask_mx, dtype=bool) mask_my = np.array(mask_my, dtype=bool) unified_mask = mask_my | mask_mx # logical-or as bitwise-or 

---

So, first without a mask, you can just imagine your array:

 b = IRtest[my,mx] 

Then, to apply the mask, you can:

 b *= ~unified_mask 

or explicitly assign zeros (again, fancy-indexing, this time using a mask):

 b[unified_mask] = 0 

As single line:

 b = IRtest[my,mx] * ~unified_mask 

It is shorter and faster than a loop. Be careful, ah?

With numpy, more often than not, you can and should avoid loops.