Index 2D Numpy Array with 2 Index Lists

Question

I have a strange situation.

I have a 2D Numpy array, x:

x = np.random.random_integers(0,5,(20,8)) 

And I have 2 indexers - one with indexes for rows and one with indexes for a column. To index X, I need to do the following:

 row_indices = [4,2,18,16,7,19,4] col_indices = [1,2] x_rows = x[row_indices,:] x_indexed = x_rows[:,column_indices] 

Instead simply:

 x_new = x[row_indices,column_indices] 

(which is not with an error: cannot broadcast (20,) with (2,))

---

I would like to be able to do indexing on a single line using broadcast, as this will keep the code clean and readable ... Also, I don't know everything about python under the hood, but as I understand it, it should be faster to do it in one line (and I will work with fairly large arrays).

---

Test case:

 x = np.random.random_integers(0,5,(20,8)) row_indices = [4,2,18,16,7,19,4] col_indices = [1,2] x_rows = x[row_indices,:] x_indexed = x_rows[:,col_indices] x_doesnt_work = x[row_indices,col_indices] 

Answer 1

Choices or assignments with np.ix_ with indexing or logical arrays / masks

1. With indexing-arrays

A. Choice

We can use np.ix_ to get np.ix_ index arrays that can be broadcast against each other, which leads to multidimensional index combinations. Thus, when this tuple is used for indexing in the input array, we get the same multidimensional array. Therefore, to make a selection based on two 1D index arrays, it would be:

 x_indexed = x[np.ix_(row_indices,col_indices)] 

B. Purpose

We can use the same record to assign a scalar or broadcast array to these indexed positions. Consequently, the following work for assignments -

 x[np.ix_(row_indices,col_indices)] = # scalar or broadcastable array 

masks

We can also use logical arrays / masks with np.ix_ , similar to how index arrays are used. This can be used again to select a block from the input array, as well as for assignments in it.

A. Choice

Thus, with boolean arrays row_mask and col_mask as masks for selecting rows and columns respectively, we can use the following to select:

 x[np.ix_(row_mask,col_mask)] 

B. Purpose

And the following work for assignments -

 x[np.ix_(row_mask,col_mask)] = # scalar or broadcastable array 

---

Test runs

Using np.ix_ with indexing-arrays

Input Array and Index Arrays -

 In [221]: x Out[221]: array([[17, 39, 88, 14, 73, 58, 17, 78], [88, 92, 46, 67, 44, 81, 17, 67], [31, 70, 47, 90, 52, 15, 24, 22], [19, 59, 98, 19, 52, 95, 88, 65], [85, 76, 56, 72, 43, 79, 53, 37], [74, 46, 95, 27, 81, 97, 93, 69], [49, 46, 12, 83, 15, 63, 20, 79]]) In [222]: row_indices Out[222]: [4, 2, 5, 4, 1] In [223]: col_indices Out[223]: [1, 2] 

A tuple of indexed arrays using np.ix_ -

 In [224]: np.ix_(row_indices,col_indices) # Broadcasting of indices Out[224]: (array([[4], [2], [5], [4], [1]]), array([[1, 2]])) 

Make a choice -

 In [225]: x[np.ix_(row_indices,col_indices)] Out[225]: array([[76, 56], [70, 47], [46, 95], [76, 56], [92, 46]]) 

As suggested by OP , this is actually the same as performing old-school broadcasting with a version of the two-dimensional row_indices array, whose elements / indices are sent along axis=0 and thus creating a singleton dimension to axis=1 which allows broadcasting with col_indices , So we would have an alternative solution, like so -

 In [227]: x[np.asarray(row_indices)[:,None],col_indices] Out[227]: array([[76, 56], [70, 47], [46, 95], [76, 56], [92, 46]]) 

As mentioned earlier, for assignments we just do it.

Row, col, array indexing -

 In [36]: row_indices = [1, 4] In [37]: col_indices = [1, 3] 

Doing tasks with a scalar -

 In [38]: x[np.ix_(row_indices,col_indices)] = -1 In [39]: x Out[39]: array([[17, 39, 88, 14, 73, 58, 17, 78], [88, -1, 46, -1, 44, 81, 17, 67], [31, 70, 47, 90, 52, 15, 24, 22], [19, 59, 98, 19, 52, 95, 88, 65], [85, -1, 56, -1, 43, 79, 53, 37], [74, 46, 95, 27, 81, 97, 93, 69], [49, 46, 12, 83, 15, 63, 20, 79]]) 

Make assignments with a 2D block (broadcast array) -

 In [40]: rand_arr = -np.arange(4).reshape(2,2) In [41]: x[np.ix_(row_indices,col_indices)] = rand_arr In [42]: x Out[42]: array([[17, 39, 88, 14, 73, 58, 17, 78], [88, 0, 46, -1, 44, 81, 17, 67], [31, 70, 47, 90, 52, 15, 24, 22], [19, 59, 98, 19, 52, 95, 88, 65], [85, -2, 56, -3, 43, 79, 53, 37], [74, 46, 95, 27, 81, 97, 93, 69], [49, 46, 12, 83, 15, 63, 20, 79]]) 

Using np.ix_ with masks

Input Array -

 In [19]: x Out[19]: array([[17, 39, 88, 14, 73, 58, 17, 78], [88, 92, 46, 67, 44, 81, 17, 67], [31, 70, 47, 90, 52, 15, 24, 22], [19, 59, 98, 19, 52, 95, 88, 65], [85, 76, 56, 72, 43, 79, 53, 37], [74, 46, 95, 27, 81, 97, 93, 69], [49, 46, 12, 83, 15, 63, 20, 79]]) 

Entering a string, col masks -

 In [20]: row_mask = np.array([0,1,1,0,0,1,0],dtype=bool) In [21]: col_mask = np.array([1,0,1,0,1,1,0,0],dtype=bool) 

Make a choice -

 In [22]: x[np.ix_(row_mask,col_mask)] Out[22]: array([[88, 46, 44, 81], [31, 47, 52, 15], [74, 95, 81, 97]]) 

Doing tasks with a scalar -

 In [23]: x[np.ix_(row_mask,col_mask)] = -1 In [24]: x Out[24]: array([[17, 39, 88, 14, 73, 58, 17, 78], [-1, 92, -1, 67, -1, -1, 17, 67], [-1, 70, -1, 90, -1, -1, 24, 22], [19, 59, 98, 19, 52, 95, 88, 65], [85, 76, 56, 72, 43, 79, 53, 37], [-1, 46, -1, 27, -1, -1, 93, 69], [49, 46, 12, 83, 15, 63, 20, 79]]) 

Make assignments with a 2D block (broadcast array) -

 In [25]: rand_arr = -np.arange(12).reshape(3,4) In [26]: x[np.ix_(row_mask,col_mask)] = rand_arr In [27]: x Out[27]: array([[ 17, 39, 88, 14, 73, 58, 17, 78], [ 0, 92, -1, 67, -2, -3, 17, 67], [ -4, 70, -5, 90, -6, -7, 24, 22], [ 19, 59, 98, 19, 52, 95, 88, 65], [ 85, 76, 56, 72, 43, 79, 53, 37], [ -8, 46, -9, 27, -10, -11, 93, 69], [ 49, 46, 12, 83, 15, 63, 20, 79]]) 

Answer 2

What about:

 x[row_indices][:,col_indices] 

For instance,

 x = np.random.random_integers(0,5,(5,5)) ## array([[4, 3, 2, 5, 0], ## [0, 3, 1, 4, 2], ## [4, 2, 0, 0, 3], ## [4, 5, 5, 5, 0], ## [1, 1, 5, 0, 2]]) row_indices = [4,2] col_indices = [1,2] x[row_indices][:,col_indices] ## array([[1, 5], ## [2, 0]]) 

Answer 3

 import numpy as np x = np.random.random_integers(0,5,(4,4)) x array([[5, 3, 3, 2], [4, 3, 0, 0], [1, 4, 5, 3], [0, 4, 3, 4]]) # This indexes the elements 1,1 and 2,2 and 3,3 indexes = (np.array([1,2,3]),np.array([1,2,3])) x[indexes] # returns array([3, 5, 4]) 

Note that numpy has very different rules depending on which indexes you use. Therefore, indexing multiple elements should be tuple np.ndarray (see indexing guide ).

So, you only need to convert your list to np.ndarray , and it should work as expected.

Answer 4

I think you are trying to do one of the following (equlvalent) operations:

 x_does_work = x[row_indices,:][:,col_indices] x_does_work = x[:,col_indices][row_indices,:] 

This will actually create a subset of x with only the selected rows, then select the columns from this or vice versa in the second case. The first case can be considered as

 x_does_work = (x[row_indices,:])[:,col_indices]