Data.table: counting rows in a time moving window

Question

library(data.table)
df <- data.table(col1 = c('B', 'A', 'A', 'B', 'B', 'B'), col2 = c("2015-03-06 01:37:57", "2015-03-06 01:39:57", "2015-03-06 01:45:28", "2015-03-06 02:31:44", "2015-03-06 03:55:45", "2015-03-06 04:01:40"))

For each line, I want to count the number of lines with the same "col1" values ​​and the time in the window for the last 10 minutes before the time of this line (including)

I run the following code:

df$col2 <- as_datetime(df$col2)
window = 10L
(counts = setDT(df)[.(t1=col2-window*60L, t2=col2), on=.((col2>=t1) & (col2<=t2)), 
                     .(counts=.N), by=col1]$counts)

df[, counts := counts]

and got the following error:

Error in `[.data.table`(setDT(df), .(t1 = col2 - window * 60L, t2 = col2), : Column(s) [(col2] not found in x

I want to get the result as follows:

col1    col2              counts
B   2015-03-06 01:37:57     1
A   2015-03-06 01:39:57     1
A   2015-03-06 01:45:28     2
B   2015-03-06 02:31:44     1
B   2015-03-06 03:55:45     1
B   2015-03-06 04:01:40     2

Answer 1

Possible Solution:

df[.(col1 = col1, t1 = col2 - gap * 60L, t2 = col2)
   , on = .(col1, col2 >= t1, col2 <= t2)
   , .(counts = .N), by = .EACHI][, (2) := NULL][]

which gives:

   col1                col2 counts
1:    B 2015-03-06 01:37:57      1
2:    A 2015-03-06 01:39:57      1
3:    A 2015-03-06 01:45:28      2
4:    B 2015-03-06 02:31:44      1
5:    B 2015-03-06 03:55:45      1
6:    B 2015-03-06 04:01:40      2

A few notes about your approach:

• You do not need it setDTbecause you have already built dfwith data.table(...).
• Testing is onnot indicated correctly: you need to separate the connection conditions with ,, rather than with &. For instance:on = .(col1, col2 >= t1, col2 <= t2)
• Use by = .EACHIto get the result for each row.

---

:

df[, counts := .SD[.(col1 = col1, t1 = col2 - gap * 60L, t2 = col2)
                   , on = .(col1, col2 >= t1, col2 <= t2)
                   , .N, by = .EACHI]$N][]

.