Address ranges in a sparse Scipy matrix

Question

I have a large matrix, currently in numpy, which I would like to migrate to a scipy sparse matrix, because saving textual representations of the numpy matrix (2000,2000) exceeds 100 MB.

(1) There seems to be an excess of sparse matrices in scipy [for example, lil_matrix or dok_matrix — which one would be optimal for simple incrementing and efficient for storing in a database?

(2) I would like to be able to consider the ranges in the matrix as follows:

>> import numpy as np >> a = np.zeros((1000,1000)) >> a[3:5,4:7] += 1 

It seems that this is not possible for sparse matrices?

Answer 1

I can not say what is the most efficient storage. It will depend on your data.

I can say, however, that the += operator works, you just cannot rely on the usual broadcast rules:

 >>> m = sparse.lil_matrix((100,100)) >>> m[50:56,50:56]+=scipy.ones((6,6)) >>> m[50,50] #1.0