Complete a task order using the .NET 4.0 parallel task libraries

Question

I have a program that has a ton of sensors that produce data at a fairly high speed, and consumers who need to consume it. Consumers consume very different tariffs.

Since I use IObserver / IObservable, the trivial solution was to simply create a task for each event and wrap the OnNext () call and the data in lamda. It worked very well, I was surprised how little overhead there was on raw calls.

The problem is that some of these consumers need an order of strict compliance with events and cannot miss any events. "PerferFairness" is not good enough.

The best solution I came across is instead of wrapping an event / OnNext () pair to wrapping Insert in a ParallelQueue, one queue for each consumer and having a thread at the other end of the queue to do OnNext ().

Three immediate problems with this approach. This is much, much slower than the solution for processing tasks / OnNext (). Is there a Dequeue lock (or is there?) For ParallelQueue, so the implementation is a bit complicated. The third is that it seems such a general problem that I cannot imagine that there is no way to enforce the order that I missed, maybe something like several task factories sharing the base pool, each Factory with some a setting that makes them strictly keep order.

Does anyone know the right way to achieve what I'm trying to do?

EDIT: , , . / , .

Answer 1

TPL DataFlow . TPL , .

TPL Dataflow .NET 4 .NET 4.5.

Answer 2

, BlockingCollection<T> ConcurrentQueue<T>, Take . :.

// the default is ConcurrentQueue, so you don't have to specify, but if you
// wanted different behavior you could use e.g. ConcurrentStack

var coll = new BlockingCollection<int>(new ConcurrentQueue<int>());

coll.Add(5); // blocks if the collection is at max capacity

int five = coll.Take(); // blocks if the collection is empty