Is it useful to use ZeroMQ to create a peer workload scheduler?

Question

I am coding a workload scheduler. I would like my piece of software to be a peer-to-peer scheduler, i.e. a node knows only some neighbors (other nodes) and uses them to access other nodes.

Each node will have its own table of weighted routes for sending messages to other peer nodes (mainly based on the number of hops), i.e. "I want the wizard to give me my graph" or "is there resource A available on node B?": Which neighbor is closer to my target?

For example, I wrote my own routing protocol using XML-RPC ( xmlrpc-c ) and std::multimaps / std::maps .

I am thinking of using ZeroMQ to optimize my data streams:

• priority can reduce the network load between peers;
• Subscriptions can be used to post updates.

Consequently:

• I would need to open as many sockets as I would create new types of connections;
• Each node must be a client, server, publisher, subscriber, broker and directory;
• I'm not sure that my “peer-to-peer architecture” is compatible with the main purpose of ZeroMQ.

Do you think ZeroMQ might be a useful concept?

Answer 1

It would be helpful to know exactly what you mean by “routing protocol”. It sounds as if you mean the business logic of routing for a particular partner. It is also more useful to know what you want to achieve with ZeroMQ.

Have you read the ZeroMQ Guide ? ZeroMQ is a pretty great beast, and without wasting time on it, you will most likely be embarrassed. As a bonus, reading the manual will also help you answer this question for yourself, as you know your requirements better.

ZeroMQ has been designed to create robust distributed and multi-threaded applications. Because distributed applications can often take the form of peer-to-peer, ZeroMQ may indeed be suitable for your needs.