I know that when I create a line data.table line by line, it is best to allocate space:
library(data.table) dt <- data.table(x=rep(0,1000), y=rep(0,1000)) for(i in 1L:1000L) { set(dt, i, 1L, runif(1)) set(dt, i, 2L, rnorm(1)) }
(In fact, if I do not predetermine, I get a segmentation error with this code.)
If I don't know the number of rows in advance, I need to grow dynamically, perhaps using an exponential distribution or something like that. Do I need to manage this process myself or is there any existing support in data.table for dynamic growth?
Also, when I finished adding lines, I probably have some space left, is there a truncate() method or similar? Or should I just do dt <- dt[1:n,] ?
source share