Why is Elixir message delivery time proportional to message size?

Question

I find that the time it takes to deliver a message to Elixir is proportional to the size of the message when I expect it to be relatively constant. Since data structures are immutable, the runtime should be able to transfer large structures between processes by reference (in constant time). Consider the following test.

use Bitwise defmodule PerfTask do def pack(s) do {millis, packed} = :timer.tc(fn -> Enum.to_list(s) end) IO.puts("packed in #{millis} millis") Task.async(fn -> packed end) end def unpack(t) do {millis, unpacked} = :timer.tc(fn -> Task.await(t) end) IO.puts("unpacked in #{millis} millis") unpacked end def go(n) do IO.puts "n = #{n}" 1..n |> pack |> unpack end end PerfTask.go(1 <<< 20) 

With 2 ^ 20 items in the list, this prints.

 n = 1048576 packed in 106481 millis unpacked in 9916 millis 

It takes about 10 times as long to create a list as he does to get it from Task . (Note that the list is created before the task starts. The whole task is to return the already constructed list.)

With 2 ^ 22 items in the list, it prints

 n = 4194304 packed in 397428 millis unpacked in 38748 millis 

The ratio is still about 10: 1. A 4x longer list takes 4 times to send between processes. What am I missing?

 $ iex Erlang/OTP 18 [erts-7.2] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false] Interactive Elixir (1.2.0) - press Ctrl+C to exit (type h() ENTER for help) 

(I confirmed that the problem is not specific to the Task module, replacing it with simple processes with similar results.)

Answer 1

In this answer from @rvird, your basic assumption is wrong. To quote Mr. Wiring:

., current Erlang versions basically copy everything except large binary files. In the old days before SMP, it was possible not to copy but to transfer links. Although this led to very fast messaging, it created other implementation problems, the first garbage collection is harder and harder. I think that today link passing and data sharing can lead to excessive blocking and synchronization, which, of course, is not a good thing.

In the Elixir context, “large binaries” means very long strings - more than 64K.