The fastest way to share a connection and data from it with multiple processes?

Question

I have several application processes, each of which connects to the servers and receives data from them. Often the servers to which they are connected and the data retrieved overlap between the processes. Thus, there is a lot of unnecessary duplication of data over the network, more connections than necessary (which levy taxes from the servers), and the data ends up being stored excessively in memory in applications.

One solution would be to combine several application processes into one, but for the most part they are really logically different, and this can be many years of experience.

Unfortunately, the delay is critical and the amount of data is huge (any base unit may not be too large, but as soon as the client makes a request, the server will send a quick stream of updates as the data changes, which may be higher than 20 MB / s, and all this must be provided to requesting applications with the maximum possible delay).

The solution that comes to mind is the local daemon process code from which application processes will request data. The daemon should check if a connection exists with the corresponding server, and if it is not created. Then it will retrieve the data and use the shared memory (due to a delay problem, otherwise I would use sockets) to provide the data to the requesting application.

A simpler idea in the short term that would allow only redundant connections would be to use unix domain sockets (this will work on Unix OS, although I prefer to stick to cross-platform libs when I can) to share the socket descriptor between all processes, so they use one connection. The problem with this is the buffer consumption - I want all processes to see everything that happens through the socket, and if I understand this approach correctly, reading in one process on the socket will not allow other processes to see the same data on their next reading (the offset inside the common descriptor will be distorted).

Answer 1

, , , . , , , Unix-, Windows.

UDP, . , UDP .

, - , , , , , (, , , , .) , , , , , - , , ( , , ).

, push, . ( , , ).

, , BNC , , , . , NIC ( ) , ( / TCP), . , , , , .

- , , , 5 - /.

Answer 2

ZeroMQ. . , 20 / ... , TCP ZeroMQ. , OpenPGM. UNIX- .

, , , . , , ... , , , .