What is the maximum (practical) number of nodes in the Erlang system

Question

I want to create a platform as a service in the financial markets using Erlang / Elixir. I will provide lambda-style AWMS features in the financial markets, but instead of being accessible via web / rest / http, I plan to distribute my own ARM-based hardware terminals for clients (Nvidia Jetson TX2 or similar, so decent hardware ), They will access functions from these terminals. I want the indicated terminals to be full nodes in the system. Thus, they will use the actor model to send messages to my central servers, and indeed, the terminals can transmit messages with each other if terminal users decide to post their own functions on the Internet.

Is this a viable model? Can I run 1000 terminals? 100,000? What restrictions can I face? Is Erlang message routing scalable enough to imagine that such a network is still working if we have data flows in financial markets with soft real time? (mainly from central servers to terminals, but a good proportion can move directly from terminal to terminal). We may have a system in which up to 100 thousand are available. Or more different processes of subscribing to a channel, many of which take input and output of data every day.

Basically, I would like the canonical guide to the Erlang scaling capabilities to resemble above. Ideally, I would also like to find guidance on the security implications of such a system, i.e. Will global routing tables or any other part of the system be compromised by the rogue user of the terminal, or can they be partially “covered” by nodes from sensitive parts of the rest of the Erlang network?

Please note that I would like to actively use ports / NIF for highly computational processes.

Answer 1

I would not pursue this path for various reasons, all of which returned to those systems in which the Erlang distribution mechanism was developed - a set of boards on a passive backplane: “free” local bandwidth, and the whole machine sits in the same security domain . Erlang's distribution protocol is probably too frequent to work well on widespread and large networks, and it is certainly too uncertain. If you do not want the nodes to be able to execute :os.cmd("rm -rf /") on each other, of course.

Use the Erlang distribution protocol in your central system for your heart content, and these terminals indicate that the data is only over SSL for this system and each other. In addition, you can simply create a kind of overhead network to do whatever you want.

Answer 2

I recommend that you read this carefully, and I recommend dividing your service into small micro-services.
Another benchmark is Scalability Distribution Erlang .
In Erlang's Joe Armstorng programming book, he said:
“A few years ago, when I had my research hat, I worked with PlanetLab. I had access to the PlanetLab network, so I installed empty Erlang servers on all PlanetLab machines (about 450 of them).
"I really didn't know what I would do with the machines, so I just set up the server infrastructure to do something later."
Do not use external ports, use internal drivers written in C or C ++ instead.

Answer 3

You will find a lot of information about erlang architecture - this is the answer: How scalable is Erlang?

Short answer: there is a restriction on the restriction of nodes in the cluster, but this restriction can be quite easily violated by the federations.

EDIT 1 / Further, I would recommend reading this book: Designing for Scalability with Erlang / OTP