Vector point product along one dimension for multidimensional arrays

Question

I want to calculate the total product by the size of one of two multidimensional arrays using Theano.

First I will describe what I want to do using numpy. numpy.tensordotand numpy.dotalways seem like a matrix product, while I'm basically looking for an equivalent vector product. Given xand y, I want to calculate zas follows:

x = np.random.normal(size=(200, 2, 2, 1000))
y = np.random.normal(size=(200, 2, 2))

# this is how I now approach it:
z = np.sum(y[:,:,:,np.newaxis] * x, axis=1)

# z is of shape (200, 2, 1000)

Now I know that numpy.einsumI can probably help me here, but again, I want to do this specific calculation in Theano , which has no equivalent einsum. I will need to use dot, tensordotor a specialized subset of entropy function Anano batched_dotor batched_tensordot.

The reason I want to change my approach to this is performance; I suspect that using embedded (CUDA) point products will be faster than relying on broadcasting, the element product, and the amount.

Answer 1

Theano - . . Numpy . - T.patternbroadcast. , this.

. , . - T.shape_padaxis. :

import theano
from theano import tensor as T
import numpy as np

X = T.ftensor4('X')
Y = T.ftensor3('Y')
Y_broadcast = T.shape_padaxis(Y, axis=-1)  # appending extra dimension and making it 
                                           # broadcastable
Z = T.sum((X*Y_broadcast), axis=1)  # element-wise multiplication
f = theano.function([X, Y], Z, allow_input_downcast=True)

# Making sure that it works and gives correct results

x = np.random.normal(size=(3, 2, 2, 4))
y = np.random.normal(size=(3, 2, 2))

theano_result = f(x,y)
numpy_result = np.sum(y[:,:,:,np.newaxis] * x, axis=1)
print np.amax(theano_result - numpy_result)  # prints 2.7e-7 on my system, close enough!

, .