How sparse is the matrix so that it should be presented as rare?

In R, I try to work with a large matrix (39,146,166 rows in 127 columns), and I am having problems with memory with a number of operations on it. I determined that about 35% of the entries in the matrix are nonzero, and the rest are all zeros. Is it sparse enough that I would preserve some memory representing this matrix using one of the R sparse matrix classes? What is a good rule to determine when a matrix is ​​rarely represented?

+5
source share
1 answer

I don't think the sparse view would be much more compact. You need three numbers for each numeric item except implicit zero. Therefore, even if two of them consist of 4 bytes, the memory space will still be larger than the "sequential" storage strategy.

Thus, something above 50% will take up more storage space, but I'm sending it from the iPhone to SF Bay, so I can not test it with the object.size object.

+2
source

Source: https://habr.com/ru/post/1246438/


All Articles