How to create a memory-bound message queue in Erlang?

Question

I want asynchronous message speed, but still have some flow control. How can I accomplish this in Erlang?

Answer 1

Limiting process memory right now - this is discussed on the mailing list, etc. You can watch these streams.

At the top, when you use an implementation of OTP templates, such as gen_server, you have a lot of freedom in receiving messages from the process queue and measuring the length of the queue.

gen_server2 used in rabbitmq used for optimization by moving messages to the internal data structure. At the same time, you can refuse any new incoming message if the internal queue is too long. You can do this silently or notify the sender that the message has been rejected.

All this is at a very low level.

RabbitMQ will provide this functionality at the AMQP level.

Answer 2

A common and fairly good way to control the flow is to make well-selected messages into calls that limit how much load each client can load the server on one, effectively providing forcible return back in an extremely simple way. The trick, of course, is to choose which messages use synchronous calls :-)