Write scala performance immutable collections

Question

Quick question. I am currently developing several database queries to retrieve large enough but not massive datasets, say, approximately 10k-100k records.

So far, I have tested loading these result sets into the scala.collection.immutable.Seq file and found that it seems to take a lot of time to create the collection. If I go to a vector or list, writing to memory takes a split second.

My question is why is Seq so slow in this case? If so, in which cases would Seq be more suitable than Vector?

thanks

Answer 1

This will help if you place the corresponding fragment and what operations you call in the sequence - immutable.Seq is represented using List (see https://github.com/scala/scala/blob/v2.10.2/src/library/scala/ collection / immutable / Seq.scala # L42 ). I assume that you used :+ on immutable.Seq , which under the hood joins the end of the list, copying it (possibly giving you a quadratic overall performance), and when you switch to using immutable.List directly, you get attached to the beginning using :: (giving you linear performance).

Since Seq is just a List under the hood, you should use it when you join the beginning of the sequence - the cons :: operator creates only one node and binds it to the rest of the list, which is as fast as it can when it comes goes about immutable data structures. Otherwise, if you add at the end and you insist on immutability, you should use Vector (or the upcoming Conc lists!).

If you want to test these claims, see the link where the performance of two operations is compared using ScalaMeter - lists are 8 times faster than vectors when you add to the beginning.

However, the most appropriate data structure should be either ArrayBuffer or VectorBuilder . These are volatile data structures that dynamically change, and if you create them with += , you get reasonable performance. This assumes that you are not saving primitives.