Let M be a member of Erlang (), which is the message we send in the system. One obvious way to handle M is to create a pipeline of processes and queues. M processed by the first worker in the pipeline and then sent to the next queue. Then it gets to the next workflow, is processed again and placed in the queue. And so on, until the message is completely processed.
Perhaps a less obvious way is to define the process P , and then handwritten M to P We will denote it by P(M) . Now the message itself is a process, not a piece of data. P will do the same work as the workers in solving the queue, but he will not have to pay the overhead by inserting M back into the queue and taking it out again, etc. When processing P(M) is completed, the process simply ends. If another message M' transmitted, we simply create P(M') and allow it to process this message at the same time. If we get a set of processes, we will do [P(M) || M <- Set] [P(M) || M <- Set] , etc.
If P needs to synchronize or exchange messages, he can do this without having to "pretend to be a message" because it is a message. Contrast with the approach of the working order, when the employee must take responsibility for the message that comes through him. If P has an error, only the message P(M) affected by this error will fail. Again, we contrast with the working order approach, when an accident in the conveyor can affect other messages (mainly if the pipeline is poorly designed).
So the trick at the end: turn the message into a process that becomes a message.
The idiom is “One Process Per Communication” and is quite common in Erlang. The price and overhead of creating a new process is low enough for it to work. However, if you use the idea, you may need overload protection. The reason is that you probably want to limit the number of simultaneous requests so that you control the system’s loading, rather than blindly letting it destroy your servers. One such implementation is Job, created by Erlang Solutions, see
https://github.com/esl/jobs
and Ulf Viger represents him:
http://www.erlang-factory.com/conference/ErlangFactoryLiteLA/speakers/UlfWiger
As Ulf prompts in a conversation, we usually do some preprocessing outside of P to parse the message and internalize it in the Erlang system. But as soon as possible we will send the message M to the task, wrapping it in a process ( P(M) ). Thus, we immediately get the benefits of the Erlang Scheduler.
There is another important branch of this choice: if processing takes a long time for the message, then Erlang proactive scheduler ensures that messages with less processing needs will still be processed quickly. If you have a limited number of work queues, you may find that many of them are clogged, which makes system bandwidth difficult.