Copy 2D array to 3D - Python / NumPy

Question

I programmed a bit when I was younger, but I was never really good. I find Python to be ideal for what I want to do.

I have an Excel file that contains data (64 columns, 18496 rows) that I read using the numpy genfromtxt function. I want to put everything in a three-dimensional matrix called H. I use three loops for this, but I know that this is not the most efficient.

data = np.genfromtxt(filename, delimiter = ";",skiprows = 11) 
H = np.zeros((N,N,Nt))

for k in np.arange(N):
    for l in np.arange(N):            
        for m in np.arange(Nt):    
            H[k,l,m] = data[m+Nt*k,l]

Is there a cut-off mechanism (faster calculation) for this. I though about using a numpy form, but I can not do it.

thanks

Answer 1

np.reshape, np.transpose, -

H = data.reshape(N,Nt,N).transpose(0,2,1)

np.transpose np.swapaxes, axes 1,2 , -

H = data.reshape(N,Nt,N).swapaxes(1,2)

-

In [300]: N = 2
     ...: Nt = 3
     ...: data = np.random.randint(0,9,(N*Nt,N))
     ...: 

In [301]: data
Out[301]: 
array([[3, 6],
       [7, 4],
       [8, 1],
       [8, 7],
       [4, 8],
       [2, 3]])

In [302]: H = np.zeros((N,N,Nt),dtype=data.dtype)
     ...: for k in np.arange(N):
     ...:     for l in np.arange(N):            
     ...:         for m in np.arange(Nt):    
     ...:             H[k,l,m] = data[m+Nt*k,l]
     ...:             

In [303]: H
Out[303]: 
array([[[3, 7, 8],
        [6, 4, 1]],

       [[8, 4, 2],
        [7, 8, 3]]])

In [304]: data.reshape(N,Nt,N).transpose(0,2,1)
Out[304]: 
array([[[3, 7, 8],
        [6, 4, 1]],

       [[8, 4, 2],
        [7, 8, 3]]])

-

In [8]: # Input
   ...: N = 10
   ...: Nt = 10*50
   ...: data = np.random.randint(0,9,(N*Nt,N))
   ...: 
   ...: def original_app(data):
   ...:     H = np.zeros((N,N,Nt),dtype=data.dtype)
   ...:     for k in np.arange(N):
   ...:         for l in np.arange(N):            
   ...:             for m in np.arange(Nt):    
   ...:                 H[k,l,m] = data[m+Nt*k,l]
   ...:     return H
   ...: 

In [9]: np.allclose(original_app(data),data.reshape(N,Nt,N).transpose(0,2,1))
Out[9]: True

In [10]: %timeit original_app(data)
10 loops, best of 3: 56.1 ms per loop

In [11]: %timeit data.reshape(N,Nt,N).transpose(0,2,1)
1000000 loops, best of 3: 1.25 µs per loop