Advantages and disadvantages of brokerage and non-brokerage messaging systems

Question

I am trying to develop real-time monitoring and a control system that is modular, so it can be distributed and expanded / reconfigured for various equipment & networks.

I quickly came to the conclusion that I would need some sort of distributed corporate messaging system. But there are many options, each of which has its advantages and disadvantages, and some of them dictate different architectures. I’m trying to find out if I need a broker or a system without a broker, whether I need the reliability of messages of some systems (for example, RabbitMQ) or a lightweight high-performance system like ZeroMQ, or “arrive in order” Kafka’s high bandwidth.

First, do these architectures make sense?

---

ZeroMQ system type "Without intermediaries":

Notes:

In each “Part B” there can be many “Part A”, and in “Part C” there can be many “Part B”

Benefits:

• High throughput, low latency
• Easily integrates into components, facilitates deployment (no need to deploy a broker).

disadvantages

• . . . GUI, , . ( , , , - ). , A B , . ?
• "". .

---

RabbitMQ:

:

• .
• Discovery .

• ,
• & ( /RabbitMQ , )

  ---

:

. , . , , RabbitMQ, , (, / ), , RabbitMQ. , , .

, . , "", . , - ?

" " ZeroMQ , . , RabbitMQ, , ( ).

, - . : - , ? ?

, #, .

Answer 1

. zmq, , , . ,

Apache Kafka - . . . , "Kafka semantics are straight-forward. When publishing a message we have a notion of the message being "committed" to the log. Once a published message is committed it will not be lost as long as one broker that replicates the partition to which this message was written remains "alive". "

RabbitMQ . Clustering HA. , Apache Kafka ( ) - , , , , .

ZMQ zmq, . , zmq . embeddable networking library, , . , , ( ). Apache Kafka, , - , .

: , . , , . , .

Answer 2

" ZeroMQ , ", . - ? .

"" , . ZMQ /.

Answer 3

. (, ), .

, , :

• ( , )
• ( )

Looking at your diagram, I think you have a good requirement for two message systems

• zeromq for real-time control parts
• Kafka for guaranteed delivery of historical / database.

By the way, zmq discovery is pretty easy to solve with a few redundant zmq proxies and some form of DNS.