What is the best way to transfer data between parallel threads in .NET?

Question

I have two threads, I need to poll a bunch of separate static resources looking for updates. Another should get the data and save it in the database. How does thread 1 tell thread 2 that there is something to process?

Answer 1

If the pieces of data are independent, then treat the pieces of data as work items that should be processed by the thread pool. Use the thread pool and QueueUserWorkItem to send data to the stream (s). You should get better scalability using a symmetric thread pool and limit the amount of synchronization that should happen between producer and consumer.

For example (from MSDN ):

  TaskInfo ti = new TaskInfo("This report displays the number {0}.", 42); // Queue the task and data. if (ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc), ti)) { Console.WriteLine("Main thread does some work, then sleeps."); // If you comment out the Sleep, the main thread exits before // the ThreadPool task has a chance to run. ThreadPool uses // background threads, which do not keep the application // running. (This is a simple example of a race condition.) Thread.Sleep(1000); Console.WriteLine("Main thread exits."); } else { Console.WriteLine("Unable to queue ThreadPool request."); } // The thread procedure performs the independent task, in this case // formatting and printing a very simple report. // static void ThreadProc(Object stateInfo) { TaskInfo ti = (TaskInfo) stateInfo; Console.WriteLine(ti.Boilerplate, ti.Value); } 

Answer 2

I am using Monitor.Wait / Pulse in a work item queue.

Answer 3

Should the stream "store in the DB" be started? It seems that perhaps the best option (if possible) is for the poll thread to scroll through another thread to save. Depending on the number of threads created, it may be that using the first poll thread ThreadPool.QueueUserWorkItem () may be a more efficient route.

For greater efficiency, when saving to the database, I will use asynchronous I / O in the database, rather than synchronization methods.

Anytime you can get away from communicating directly between two threads, you should. To compose some synchronization primitives, your code will not be so easy to debug and may introduce some very subtle race conditions that cause once-in-a-million errors (which are not so fun to find / fix).

If the second thread always needs to be executed, let us know why with additional information, and we can return with a more detailed answer.

Good luck

Answer 4

I personally would have a thread of the 1st event that thread 2 can respond to. Threads can be connected to the corresponding events through a control process that triggers both threads.