Surprisingly low benchmarks for Data.Vector benchmarking

I am comparing Haskell array libraries ( arrayand packages vector) to find the best way to store big data for my use case. I use criterionas a comparison tool.

In short: my code simply selects a vector and proceeds to fill it with simple structures (1M, 10M and 100M elements, respectively). When I compare the Haskell test time with a simple reference implementation that I wrote in C, Haskell is several times faster, and I find this suspicious: C code is a simple loop filling structures in an array.

Question: Is it possible for the Haskell library to vectorbeat C in terms of performance? Or does this mean that my tests are wrong / something is not really being evaluated / is there some gotcha?

Another question is how to make sure that the Haskell vectors are actually evaluated?

Longer explanation: The task is to fill the vector with more structures. They have instances Storable, and the vector used is Data.Vector.Storable.

The data type is as follows:

data Foo = Foo Int Int deriving (Show, Eq, Generic, NFData)

And the instances Storablelook like this:

chunkSize :: Int
chunkSize = sizeOf (undefined :: Int)
{-# INLINE chunkSize #-}

instance Storable Foo where
    sizeOf    _ = 2 * chunkSize ; {-# INLINE sizeOf    #-}
    alignment _ = chunkSize     ; {-# INLINE alignment #-}
    peek ptr = Foo
        <$> peekByteOff ptr 0
        <*> peekByteOff ptr chunkSize
    {-# INLINE peek #-}
    poke ptr (Foo a b) = do
        pokeByteOff ptr 0 a
        pokeByteOff ptr chunkSize b
    {-# INLINE poke #-}

Serialization itself is working fine. Then the vector is highlighted:

mkFooVec :: Int -> IO (Vector Foo)
mkFooVec !i = unsafeFreeze =<< new (i + 1)

And is filled with structures:

populateFooVec :: Int -> Vector Foo -> IO (Vector Foo)
populateFooVec !i !v = do
    v' <- unsafeThaw v
    let go 0 = return ()
        go j = unsafeWrite v' j (Foo j $ j + 1) >> go (j - 1)
    go i
    unsafeFreeze v'

A benchmark is a standard criterion:

    defaultMain [
      bgroup "Storable vector (mutable)"
        $ (\(i :: Int) -> env (mkFooVec (10 ^ i))
        $ \v -> bench ("10e" <> show i)
        $ nfIO (populateFooVec (10 ^ i) v))  <$> [6..8]
    ]

gist contains other tests, trying to force the evaluation in different ways.

C-, , (gist). :

Foo *allocFoos(long n) {
    return (Foo *) malloc(n * sizeof(Foo));
}

// populate the array with structs:
void createFoos(Foo *v, long n) {
    for (long i = 0; i < n; ++i) {
        v[i].name = i;
        v[i].id = i + 1;
    }
}

, : gcc -O2 -o bench benchmark.c && ./bench

, , C 50 , Criterion - 800 (!). : , ? , (, Haskell, -). ? - vector C ( GCC , btw)?

, ;)

+4
1

, . Haskell ( ) C ( 1000 , 1000).

EDIT: , :

  • * , .
  • - . .

? .

% gcc bench.c -O3 && ./a.out
Starting the benchmark
[[ Malloced-array-[10000000] ]]Time taken: 11.904249 ms (cpu) 11.904249 ms (wall)
Done
./a.out  11.78s user 0.14s system 98% cpu 12.131 total

. 11 C 10 ^ 7 .

% ghc -O2 bench.hs && ./bench
benchmarking Storable vector (FAKE mutable)/10e6
time                 2.362 ms   (2.236 ms .. 2.561 ms)
                     0.953 R²   (0.909 R² .. 0.989 R²)
mean                 2.344 ms   (2.268 ms .. 2.482 ms)
std dev              305.0 μs   (169.1 μs .. 477.1 μs)
variance introduced by outliers: 79% (severely inflated)

benchmarking Storable vector (FAKE mutable)/10e7
time                 23.37 ms   (22.13 ms .. 24.73 ms)
                     0.989 R²   (0.979 R² .. 0.996 R²)
mean                 23.19 ms   (22.63 ms .. 23.76 ms)
std dev              1.287 ms   (1.015 ms .. 1.713 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking Storable vector (FAKE mutable)/10e8
time                 232.2 ms   (215.1 ms .. 247.3 ms)
                     0.994 R²   (0.974 R² .. 1.000 R²)
mean                 223.5 ms   (215.9 ms .. 231.5 ms)
std dev              10.41 ms   (7.887 ms .. 13.06 ms)
variance introduced by outliers: 14% (moderately inflated)

. 23 Haskell 10 ^ 7.

macbook GHC 8.2.

+1

Source: https://habr.com/ru/post/1693322/


All Articles