Julia DataFrame filtering multiple values

Question

There are two ways to filter a DataFrame in the following example:

1. df = df[((df[:field].==1) | (df[:field].==2)), :]
2. df = df[[in(v, [1, 2]) for v in df[:field]], :]

The second approach is slower, but it is suitable for a variable set of values ​​in state. Is there any syntactic sugar I missed, so I can get it as fast as the 1st method, but with some type construct in?

Answer 1

julia> using DataFrames

Function

findin may be another way to complete the task:

julia> function t_findin(df::DataFrames.DataFrame)
        df[findin(df[:A],[1,2]), :]
       end
t3 (generic function with 1 method)

:

julia> function t_compr(df::DataFrames.DataFrame)
        df[[in(v, [1, 2]) for v in df[:A]], :]
       end
t1 (generic function with 1 method)

several conditions:

julia> function t_mconds(df::DataFrames.DataFrame)
        df[((df[:A].==1) | (df[:A].==2)), :]
       end
t2 (generic function with 1 method)

Test Data

julia> df[:B] = rand(1:30,10_000_000);
julia> df[:A] = rand(1:30,10_000_000);

Test results

julia> @time t_findin(df);
  0.489064 seconds (67 allocations: 19.340 MB, 0.49% gc time)

julia> @time t_mconds(df);
  0.222389 seconds (106 allocations: 78.933 MB, 5.98% gc time)

julia> @time t_compr(df);
 23.634846 seconds (100.00 M allocations: 2.563 GB, 1.47% gc time)