Numpy Array Column Slicing Throws IndexError: Invalid Index Exception

Question

I am using version 1.5.1 from numpy and Python 2.6.6.

I read the binary into a numpy array:

>>> dt = np.dtype('<u4,<i2,<i2,<i2,<i2,<i2,<i2,<i2,<i2,u1,u1,u1,u1') >>> file_data = np.fromfile(os.path.join(folder,f), dtype=dt) 

This works great. Studying the result:

 >>> type(file_data) <type 'numpy.ndarray'> >>> file_data array([(3571121L, -54, 103, 1, 50, 48, 469, 588, -10, 0, 102, 0, 0), (3571122L, -78, 20, 25, 45, 44, 495, 397, -211, 0, 102, 0, 0), (3571123L, -69, -48, 23, 60, 19, 317, -26, -151, 0, 102, 0, 0), ..., (3691138L, -53, 52, -2, -11, 76, 988, 288, -101, 1, 102, 0, 0), (3691139L, -11, 21, -27, 25, 47, 986, 253, 176, 1, 102, 0, 0), (3691140L, -30, -19, -63, 59, 12, 729, 23, 302, 1, 102, 0, 0)], dtype=[('f0', '<u4'), ('f1', '<i2'), ('f2', '<i2'), ... , ('f12', '|u1')]) >>> file_data[0] (3571121L, -54, 103, 1, 50, 48, 469, 588, -10, 0, 102, 0, 0) >>> file_data[0][0] 3571121 >>> len(file_data) 120020 

When I try to cut the first column:

 >>> file_data[:,0] 

I get:

 IndexError: invalid index. 

I looked through simple examples and was able to perform slicing:

 >>> a = np.array([(1,2,3),(4,5,6)]) >>> a[:,0] array([1, 4]) 

The only difference I see in my case and a simple example is to use dtype. What am I doing wrong?

Answer 1

When you set the dtype type, you create a Record Array . Numpy treats as a 1D array of elements of your dtype. There is a fundamental difference between

 file_data[0][0] 

and

 file_data[0,0] 

In the first, you request the first element of the 1D array, and then retrieve the first element of this returned element. In the second case, you request an element in the first row of the first column of the 2D array. That is why you get an IndexError .

If you want to access a single element using 2D notation, you can create a view and work with it. Unfortunately, AFAIK, if you want to treat your object as a 2D array, all elements must have the same dtype type.