What is a fast, memory efficient way to transfer data between threads in C #?

Question

I have a uniprocessor dual-threaded application. In stream 1, you will hear a market data feed and update the latest quote on thousands of stocks. In stream 2, a timer with a sampling frequency will start and take a snapshot of the latest quotes for processing. In fact, I need to downsample an extremely fast market data feed.

My first guess is to use a BlockingQueue. To do this, I need to move the timer functions to Thread 1, which I can do by checking the clock every time a quote update arrives and sends a quote snapshot to the queue at the sample rate. My concern is that the queue will consume a lot of memory, and garbage collection will slow down.

My second suggestion is for Thread 1 to copy the data to a locked element at the sampling rate that Thread 2 can access. My concern is that locks will be slow.

My mom guesses this to make quote primitives unstable. Since one thread only writes and only one thread reads, maybe this is suitable?

Is there a better way to transfer data between threads for this delay-sensitive application? This is not an ultra-high frequency application. I can tolerate latencies of the order of tens of ms.

Answer 1

If you have only 2 threads accessing this resource (i.e. simultaneous readings are not required), then the simplest (and one of the fastest) would be to simply use the lock keyword:

 public class QuoteStore { private readonly List<Quote> _quotes = new List<Quote>(); private readonly object _mutex = new object(); public ReadOnlyCollection<Quote> GetQuotes() { lock (_mutex) { return _quotes.ToReadOnly(); } } public void AddQuote() { lock (_mutex) { _quotes.Add(quote); } } } 

If, however, parallel readings are required, this would be well suited for the ReaderWriterLockSlim class . You can get a read lock when copying data and a write lock when writing data, for example:

 public class QuoteStore : IDisposable { private readonly ReaderWriterLockSlim _mutex = new ReaderWriterLockSlim(); private readonly List<Quote> _quotes = new List<Quote>(); public ReadOnlyCollection<Quote> GetQuotes() { _mutex.EnterReadLock(); try { return _quotes.ToReadOnly(); } finally { _mutex.ExitReadLock(); } } public void AddQuote() { _mutex.EnterWriteLock(); try { _quotes.Add(quote); } finally { _mutex.ExitWriteLock(); } } public void Dispose() { _mutex.Dispose(); } } 

Or, if you are using .Net 4 or higher, there are many great, simultaneously mutable collections in the System.Collections.Concurrent namespace that you could probably use without any problems (they are objects that are free from blocking, and, like usually very fast) and some performance improvements are included in .Net 4.5 too !).

Answer 2

Is this not the case in the lineup of consumer producers? The consumer will wait (Monitor.Wait) after Producer starts to pulse when a new channel appears. As soon as new / updated feeds appear, Producer will fill the queue and start Monitor.Pulse.