A faster way to get matrix multiplication

Question

I am trying to calculate sinc(vec*dist) for all pairwise distances for each vec , where vec is a 1D array and dist is the pairwise distances of row vectors in the coord matrix. I use:

 dist = scipy.spatial.distance.pdist(coord) for i in range(len(vec)): I_avr[i] = numpy.sum(numpy.sinc(vec[i]*dist)) print vec[i], I_avr[i] 

Usually coord has 10 ^ 5-10 ^ 6 vectors and the vec length is 10 ^ 3 to 10 ^ 4. Can anyone recommend improvements or changes to make this faster?

Answer 1

In theory, you could split the distance matrix into pieces and make it a block block, and then fill it in, but I don't think it is possible at all, because sum(sinc(...)) takes too much time. Try a single block of size 10,000:

 >>> from scipy.spatial import distance >>> import numpy as np >>> np.random.seed(0) >>> coord = np.random.random((10000, 3)) >>> dist = distance.pdist(coord) >>> %timeit np.sum(np.sinc(dist)) 1 loop, best of 3: 1.82 s per loop 

It is just for one element, and there is no multiplication. If vec has a size from 10^3 to 10^4 , the time for one 10k x 10k fragment will be from 30 minutes to 5.5 hours.

Now, if we increase the coord size to 10^5 to 10^6 , the total expected time is expected from 55 hours to 6.3 years, depending on the exact values ​​of the inputs.