Monitoring the number of threads to balance CPU and IO is waiting in .NET

Question

Is there a way to find out inside a .NET application whether Im is currently limited by CPU time or if IO is a bottleneck?

I am requesting a bunch of remote network services, the details of which are not significant and can be abstracted as:

// perform a single operation if ( [randomness] ) { sleep(10s of seconds); // DNS/TCP connection timeout } else { sleep(10s of miliseconds); // query a remote server } for x = 1…lots { // Do some CPU intensive work } 

While I have a processor, Id can schedule as many of them as possible, because they will wait a long time for I / O, but as soon as the processor is fully loaded, I do not want to support spawning threads, because this will lead to increased performance collapse. The total number of tasks is "large."

The obvious answer would be to simply select “a reasonable amount of parallel threads as the setting, but this has two problems:

• The length of "sleep" can vary widely between a round trip on a local network and a TCP connection timeout, so the ratio of CPU to I / O latency can vary by 3 orders of magnitude.
• The size of the machine on which it will run can vary from a small single CPU to a machine with a heavy weight.

In an ideal world, I / O operations will all be replaced with asynchronous termination callbacks, but this is not easy / possible in this case, since the network RPC uses the existing blocking code.

Answer 1

I recommend two thread queues with the first processing of I / O calls and the second processing of processing after I / O, because you are addressing two different issues.

For I / O, I will create a thread queue that dumps the I / O results to a secondary queue. You can queue a large number of threads and set the throttle to the number of active ones using a callback that signals when the stream ends and starts the next one. The limiting factor will be more memory than the processor, since the working threads conserve memory even when they are blocked (but not yet running areas of the thread's memory are small).

The second queue can use ThreadPool or a separate cluster of your own threads. With separate chokes, you can adjust how many of both groups of threads are running, based on the average elapsed time or something like that.

If you make the second turn a database table, then you have a simple point with which you can divide the load on several machines (or the cloud) and copy the time statistics.

Answer 2

For this type of work, you'd better use the async i / o template instead of creating multiple threads (or using many threadpool threads).

This is mainly due to the use of BeginXxx methods for all I / O calls that will use the I / O completion port only while waiting for results. When the result returns, it will call your callback in the threadpool thread. The end result is that your code will only work until the start of the call, and when the result returns. You will not have threads waiting for replies.

Answer 3

If you can use .NET 4, you can use a parallel task library.

TPL will control the number of threads for you automatically (for CPU-bound operations) or you can limit the maximum degree of parallelism when you need it (usually for I / O-bound operations where you want to limit many can happen right away).

In your case, you can divide the work into two tasks and use a separate custom TaskScheduler for each, so that you have full control over how many threads are allocated and what CPU priority they start (for example, put all computing tasks at normal priority below).

Schedule the I / O task, then use Task.ContinueWith (see http://msdn.microsoft.com/en-us/library/dd321307.aspx ) to schedule the calculation part on a separate TaskScheduler.

(This may also be of interest: http://mikehadlow.blogspot.com/2010/11/using-task-parallel-library-with-aspnet.html )

Answer 4

In C #, you can use ThreadPool to manage all your threads and set a limit on the number of threads you are currently using. There is an object in .NET called PerformanceCounter in System.Diagnostics that will help you.

 PerformanceCounter cpuCounter; PerformanceCounter ramCounter; 

cpuCounter = new PerformanceCounter();

cpuCounter.CategoryName = "Processor"; cpuCounter.CounterName = "% Processor Time"; cpuCounter.InstanceName = "_Total";

ramCounter = new PerformanceCounter("Memory", "Available MBytes");

public string getCurrentCpuUsage(){ cpuCounter.NextValue()+"%"; }

public string getAvailableRAM(){ ramCounter.NextValue()+"MB"; }