How much faster is Eigen for small fixed size matrices?

Question

I am using Julia at the moment, but I have a critical function that requires a huge number of repetitive matrix operations on small fixed-size matrices (3 dimensional or 4 dimensional). It seems that all matrix operations in Julia are handled by BLAS and LAPACK. There is also a lot of memory allocation in some of these functions.

There is a julia library for small matrices that boasts impressive accelerations for 3x3 matrices, but it has not been updated after 3 years. I am considering rewriting my critical function in Eigen

I know that Eigen claims to be really good for fixed-size matrices, but I'm still trying to judge whether this function should be rewritten in Eigen or not. Performance metrics for dynamic matrices. Does anyone have any data to indicate how much performance comes from fixed size matrices? The types of operations that I perform are matrix x, matrix x vector, positive definite linear solutions.

Answer 1

If you need fast operations for small matrices, I highly recommend it FixedSizeArrays. For example:

using FixedSizeArrays

function foo(A, b, n)
    s = 0.0
    for i = 1:n
        s += sum(A*b)
    end
    s
end

function foo2(A, b, n)
    c = A*b
    s = 0.0
    for i = 1:n
        A_mul_B!(c, A, b)
        s += sum(c)
    end
    s
end

A = rand(3,3)
b = rand(3)
Af = Mat(A)
bf = Vec(b)

foo(A, b, 1)
foo2(A, b, 1)
foo(Af, bf, 1)
@time 1

@time foo(A, b, 10^6)
@time foo2(A, b, 10^6)
@time foo(Af, bf, 10^6)

Results:

julia> include("/tmp/foo.jl")
  0.000005 seconds (148 allocations: 10.151 KB)
  0.325433 seconds (2.00 M allocations: 106.812 MB, 9.07% gc time)
  0.144793 seconds (8 allocations: 304 bytes)
  0.012683 seconds (6 allocations: 192 bytes)

foo2 , FixedSizeArrays.