All geek questions in one place

Optimizing assignment to an array from various arrays - NumPy

I have four square matrices with size 3Nx3N called A, B, C and D.

I want to combine them into one matrix. Code for loops:

import numpy N = 3 A = numpy.random.random((3*N, 3*N)) B = numpy.random.random((3*N, 3*N)) C = numpy.random.random((3*N, 3*N)) D = numpy.random.random((3*N, 3*N)) final = numpy.zeros((6*N, 6*N)) for i in range(N): for j in range(N): for k in range(3): for l in range(3): final[6*i + k][6*j + l] = A[3*i+k][3*j+l] final[6*i + k + 3][6*j + l + 3] = B[3*i+k][3*j+l] final[6*i + k + 3][6*j + l] = C[3*i+k][3*j+l] final[6*i + k][6*j + l + 3] = D[3*i+k][3*j+l]

Is it possible to write down previous ones for numpythonic loops?

+5
source share
4 answers

A big problem to practice array-slicing into multidimensional tensors / arrays!

We will initialize the output array as a 6D multidimensional array and simply cut it and assign four arrays to be changed as 4D arrays. The goal is to avoid any styling / concatenation, as they would be expensive, especially when working with large arrays, instead working with changing the input arrays, which would be just a lookup.

Here's the implementation -

 out = np.zeros((N,2,3,N,2,3),dtype=A.dtype) out[:,0,:,:,0,:] = A.reshape(N,3,N,3) out[:,0,:,:,1,:] = D.reshape(N,3,N,3) out[:,1,:,:,0,:] = C.reshape(N,3,N,3) out[:,1,:,:,1,:] = B.reshape(N,3,N,3) out.shape = (6*N,6*N)

To explain a little more, we had:

  |------------------------ Axes for selecting A, B, C, D np.zeros((N,2,3,N,2,3),dtype=A.dtype) |------------------------- Axes for selecting A, B, C, D

Thus, these two axes (second and fifth) of length (2x2) = 4 used to select between the four inputs.

Runtime test

Approaches -

 def original_app(A, B, C, D): final = np.zeros((6*N,6*N),dtype=A.dtype) for i in range(N): for j in range(N): for k in range(3): for l in range(3): final[6*i + k][6*j + l] = A[3*i+k][3*j+l] final[6*i + k + 3][6*j + l + 3] = B[3*i+k][3*j+l] final[6*i + k + 3][6*j + l] = C[3*i+k][3*j+l] final[6*i + k][6*j + l + 3] = D[3*i+k][3*j+l] return final def slicing_app(A, B, C, D): out = np.zeros((N,2,3,N,2,3),dtype=A.dtype) out[:,0,:,:,0,:] = A.reshape(N,3,N,3) out[:,0,:,:,1,:] = D.reshape(N,3,N,3) out[:,1,:,:,0,:] = C.reshape(N,3,N,3) out[:,1,:,:,1,:] = B.reshape(N,3,N,3) return out.reshape(6*N,6*N)

Timing and Verification -

 In [147]: # Setup input arrays ...: N = 200 ...: A = np.random.randint(11,99,(3*N,3*N)) ...: B = np.random.randint(11,99,(3*N,3*N)) ...: C = np.random.randint(11,99,(3*N,3*N)) ...: D = np.random.randint(11,99,(3*N,3*N)) ...: In [148]: np.allclose(slicing_app(A, B, C, D), original_app(A, B, C, D)) Out[148]: True In [149]: %timeit original_app(A, B, C, D) 1 loops, best of 3: 1.63 s per loop In [150]: %timeit slicing_app(A, B, C, D) 100 loops, best of 3: 9.26 ms per loop
+4
source

I'll start with a few general observations

For numpy arrays, we usually use the syntax [,] rather than [] [] final [6 * i + k] [6 * j + l] final [6 * i + k, 6 * j + l]

For new arrays built from others, we often use things like reshape and slicing so that we can add them together as blocks, rather than with iterative loops

As a simple example, the following differences can be made:

  y = x[1:] - x[:-1]

As for the name, “matrix creation” is clearer. "load" makes more sense to read data from a file, as in np.loadtxt .

==================

So, with N=1 ,

 In [171]: A=np.arange(0,9).reshape(3,3) In [172]: B=np.arange(10,19).reshape(3,3) In [173]: C=np.arange(20,29).reshape(3,3) In [174]: D=np.arange(30,39).reshape(3,3) In [178]: final Out[178]: array([[ 0, 1, 2, 30, 31, 32], [ 3, 4, 5, 33, 34, 35], [ 6, 7, 8, 36, 37, 38], [20, 21, 22, 10, 11, 12], [23, 24, 25, 13, 14, 15], [26, 27, 28, 16, 17, 18]])

What can be created with a single bmat call:

 In [183]: np.bmat([[A,D],[C,B]]).A Out[183]: array([[ 0, 1, 2, 30, 31, 32], [ 3, 4, 5, 33, 34, 35], [ 6, 7, 8, 36, 37, 38], [20, 21, 22, 10, 11, 12], [23, 24, 25, 13, 14, 15], [26, 27, 28, 16, 17, 18]])

bmat uses a combination of hstack and vstack . It also creates np.matrix , hence the need for .A . @Divakar's solution should be faster.

This does not correspond to N = 3. 3x3 blocks do not work. But expanding the array to 6d (as Divakar does) and changing some axes, we put the subunits in the correct order.

For N=3 :

 In [57]: block=np.bmat([[A,D],[C,B]]) In [58]: b1=block.A.reshape(2,3,3,2,3,3) In [59]: b2=b1.transpose(1,0,2,4,3,5) In [60]: b3=b2.reshape(18,18) In [61]: np.allclose(b3,final) Out[61]: True

In tests with fast times (N = 3), my approach is about half the slicing_app speed.

As a curiosity, bmat works with string input: np.bmat('A,D;C,B') . This is because np.matrix tried many years ago to give a MATLAB feel.

+2
source

you can just contact em

concat A and B horizontally concat C and D horizontally

AB concatenated with vertical narrowing of the CD

Example:

 AB = numpy.concatenate([A,B],1) CD = numpy.concatenate([C,D],1) ABCD = numpy.concatenate([AB,CD],0)

Hope this helps :)

+1
source

Another way to do this using view_as_blocks :

 from skimage.util import view_as_blocks def by_blocks(): final = numpy.empty((6*N,6*N)) a,b,c,d,f= [view_as_blocks(X,(3,3)) for X in [A,B,C,D,final]] f[0::2,0::2]=a f[1::2,1::2]=b f[1::2,0::2]=c f[0::2,1::2]=d return final

You just need to think in blocks, letting view_as_blocks control the steps and shapes for you. It is as fast as other numpy solutions.

+1
source

Source: https://habr.com/ru/post/1263273/

More articles:


All Articles