Creating a large matrix in ff

Question

I am trying to create a huge matrix in ff, and I know that ff is good for this kind of thing.

But there is a big problem. Matrix sizes exceed .Machine $ max_integer! I work on a 64 bit machine using 64 bit R and 64 bit ff.

Is there any way around this problem?

It has been suggested that R uses the MAXINT value from stdint.h. Is there a way to fix this without modifying this file and possibly breaking the assembly?

> ffMatrix <- ff(vmode="boolean", dim=c(300000,300000))
Error in if (length < 0 || length > .Machine$integer.max) stop("length must be between 1 and .Machine$integer.max") : 
  missing value where TRUE/FALSE needed
In addition: Warning message:
In ff(vmode = "boolean", dim = c(300000, 300000)) :
  NAs introduced by coercion

> 300000**2 > .Machine$integer.max
[1] TRUE

Answer 1

This is still a very large matrix, you need a parallel computer :-). 300000x300000> 2 ^ 32 (int can be 32 bits, even on a 64-bit machine).

See also link