Take the dot product of the first and middle records, the second and middle + 1 records to mid-1 and the last python / numpy record

Question

Sorry for the vague sound question, I really couldn't think of a better way to describe what I'm trying to do

I have an array of 81x990000x3 and for each of 81 entries; I need to summarize the point product of the first record with the 495000th input (medium), the second with the record 495001st, etc. Until the 494999th record with the last record.

Now I am doing this in a loop as follows:

import numpy as np
summed = 0

Nt = 990000
i = Nt/2

for k in xrange(81):
  for j in xrange(Nt-i-1):
    vec1 = array[k][j]
    vec2 = array[k][j+i]

    summed += np.dot(vec1,vec2)

This is obviously quite slow as it goes through 81x990000 = 80190000 records.

Are there more countless ways to do this?

Answer 1

NB, , , , for j in xrange(Nt-i):, xrange .

, , , einsum:

import numpy as np
summed = 0

dim1 = 2  # this is 81 in your case
dim2 = 4  # this is 990000 in your case
array = np.random.random(size=(dim1, dim2, 3))

Nt = dim2
i = Nt // 2

for k in xrange(dim1):
    summed = 0
    for j in xrange(dim2-i):
        vec1 = array[k][j]
        vec2 = array[k][j+i]
        summed += np.dot(vec1,vec2)
    print summed

print '='*70

for k in xrange(dim1):
    summed = np.einsum('ij,ij', array[k][:Nt//2], array[k][Nt//2:])
    print summed

.

2.0480375425
1.89065215839
======================================================================
2.0480375425
1.89065215839

, ( , , ):

np.einsum('kij,kij->k', array[:,:Nt//2,:], array[:,Nt//2:,:])

[ 2.0480375425  1.89065215839]

Answer 2

0 494999th, , jth for j in xrange(Nt-i). , , 3D slice , . , , 2D-, axis =(1,2).

, -

• : .reshape().
• betwen : .prod().
• .sum().

, :

m,n,r = array.shape
out = array.reshape(m,2,-1,r).prod(1).sum()

-

:

def vectorized_app(array):
    m,n,r = array.shape
    return array.reshape(m,2,-1,r).prod(1).sum()

def org_app(array):
    m,n,r = array.shape
    Nt = n
    i = Nt/2
    summed = 0
    for k in xrange(m):
      for j in xrange(Nt-i):
        vec1 = array[k][j]
        vec2 = array[k][j+i]
        summed += np.dot(vec1,vec2)
    return summed

Runtimes:

In [94]: array = np.random.rand(81,9900,3)

In [95]: org_app(array)
Out[95]: 300450.24128635536

In [96]: vectorized_app(array)
Out[96]: 300450.24128635362

In [97]: %timeit org_app(array)
1 loops, best of 3: 1.29 s per loop

In [98]: %timeit vectorized_app(array)
100 loops, best of 3: 16.1 ms per loop