Non-threaded alternative to anticipation provided. (Edit: proactor template with boost.asio?)

Question

I am implementing a message passing algorithm. Messages are transmitted between neighboring nodes when they have enough information on a node to compose a message - information that is transmitted by a node from neighboring nodes. The implementation is trivial if I make each of the messages a stream and use boost :: condition to put the stream to sleep until the required information is available.

Unfortunately - I have 100k nodes in the graph, which would mean 300k threads. When I asked how to make so many threads, the answer was such that I shouldn't - and reverse engineer instead.

My question is: is there a standard design pattern for expecting a condition? Perhaps some kind of asynchronous control pattern?

EDIT: I think I can do this with a proactor template. I edited the tags to enable boost :: asio - to find out if anyone has any suggestions with this.

So, the discussion can be specific, here is how the messages are defined so far:

class Message { public: Message(const Node* from, Node* to) : m_from(from), m_to(to) {} void operator()() { m_to->ReceiveMessage( m_from->ComposeMessage() ); } private: Node *m_from, *m_to; }; 

These message functions are currently running with boost :: thread. Then we have

 class Node { Node(Node* Neighbour1, Node* Neighbour2, Node* Neighbour3); // The messages (currently threads) are created on construction, // The condition locks then sort out when they actually get passed // without me having to think too hard. void ReceiveMessage(const Message&); //set m_message from received messages; //EDIT This looks like an async write - use boost asio here? Message ComposeMessage() { // If possible I want to implement this function without threads // It works great but it if every message is a thread // then I have 300k threads. // EDIT: this looks like an async read (use boost asio here?) boost::mutex::scoped_lock lock(m_mutex); while (!m_message) //lock the thread until parameter is set. m_cond.wait(lock); return *m_message; } private: boost::optional<Message> m_message; boost::mutex m_mutex; boost::condition m_cond; } 

I like the transparency of the code and, if possible, I would like to support the same interfaces, having some alternative to conditional locking?

Answer 1

I assume you are looking for a reactor template . Here, most events do not take too much time, and they perform joint multitasking. See node.js for a JavaScript idea, but in C ++, the ACE library provides a ready-to-use concept that allows multiple threads based on the number of cores in the system.

All these libraries depend on some OS APIs that support non-blocking IO on disks, on the network, etc. When you do not wait for the OS, but another message source in your application, they provide you with tools for this.