Elemental maximum of two sparse matrices

Question

Is there a simple / built-in way to get the maximum size of an element from two (or ideally larger) sparse matrices? That is the rare equivalent of np.maximum .

Answer 1

This did the trick:

def maximum (A, B): BisBigger = AB BisBigger.data = np.where(BisBigger.data < 0, 1, 0) return A - A.multiply(BisBigger) + B.multiply(BisBigger) 

Answer 2

No, there is no built-in way to do this in scipy.sparse . Easy solution

 np.maximum(XA, YA) 

but it will obviously be very memory intensive when the matrices are large, and this can lead to the failure of your machine. An effective (but far from fast) memory solution -

 # convert to COO, if necessary X = X.tocoo() Y = Y.tocoo() Xdict = dict(((i, j), v) for i, j, v in zip(X.row, X.col, X.data)) Ydict = dict(((i, j), v) for i, j, v in zip(Y.row, Y.col, Y.data)) keys = list(set(Xdict.iterkeys()).union(Ydict.iterkeys())) XmaxY = [max(Xdict.get((i, j), 0), Ydict.get((i, j), 0)) for i, j in keys] XmaxY = coo_matrix((XmaxY, zip(*keys))) 

Note that in this case, pure Python is used instead of vectorized idioms. You can try to reduce it from time to time by vectorizing parts of it.

Answer 3

Here's another memory efficient solution, which should be slightly faster than larsmans. This is based on finding a set of unique indices for nonzero elements in two arrays using Jaime's code an excellent answer here .

 import numpy as np from scipy import sparse def sparsemax(X, Y): # the indices of all non-zero elements in both arrays idx = np.hstack((X.nonzero(), Y.nonzero())) # find the set of unique non-zero indices idx = tuple(unique_rows(idx.T).T) # take the element-wise max over only these indices X[idx] = np.maximum(X[idx].A, Y[idx].A) return X def unique_rows(a): void_type = np.dtype((np.void, a.dtype.itemsize * a.shape[1])) b = np.ascontiguousarray(a).view(void_type) idx = np.unique(b, return_index=True)[1] return a[idx] 

Testing:

 def setup(n=1000, fmt='csr'): return sparse.rand(n, n, format=fmt), sparse.rand(n, n, format=fmt) X, Y = setup() Z = sparsemax(X, Y) print np.all(ZA == np.maximum(XA, YA)) # True %%timeit X, Y = setup() sparsemax(X, Y) # 100 loops, best of 3: 4.92 ms per loop 

Answer 4

The last scipy (13.0) defines logical Booleans for sparse matrices. So:

 BisBigger = B>A A - A.multiply(BisBigger) + B.multiply(BisBigger) 

np.maximum does not work (yet) because it uses np.where , which is still trying to get truth value of an array .

Curiously, B>A returns the logical type dtype, and B>=A returns float64.

Answer 5

 from scipy import sparse from numpy import array I = array([0,3,1,0]) J = array([0,3,1,2]) V = array([4,5,7,9]) A = sparse.coo_matrix((V,(I,J)),shape=(4,4)) A.data.max() 9 

If you have not already done so, you should try ipython , you could save your time by making your spare matrix A , then just typing A. , then tab, this will print a list of methods that you can call A From this you will see that A.data gives non-zero entries as an array and therefore you just want the maximum of that.