C # Downloader: should I use Threads, BackgroundWorker or ThreadPool?

Question

I write a bootloader in C # and dwell on the following problem: what method should I use to parallelize my downloads and update my GUI?

In my first attempt, I used 4 threads, and upon completion of each of them I started another one: the main problem was that my processor is 100% started every time a new thread starts.

At Google, I discovered the existence of BackgroundWorker and ThreadPool: stating that I want to update my GUI with the progress of each link that I download, what is the best solution?

1) Creating 4 different BackgroundWorker, attaching a delegate to each ProgressChanged event to a function in my GUI to update progress?

2) Use ThreadPool and set the maximum and minimum number of threads for the same value?

If I select # 2 when there are no more threads in the queue, will it stop 4 worker threads? Does he hang them? Since I have to load different link lists (20 links to them) and switch from one to the other at the end, does ThreadPool start and stop threads between each list?

If I want to change the number of worker threads in real time and decide to use ThreadPool, changing them from 10 threads to 6, it throws and excludes and stops 4 random threads?

This is the only part that gives me a headache. I thank each of you in advance for your answers.

Answer 1

I would suggest using WebClient.DownloadFileAsync for this. You can have multiple downloads, each of which raises the DownloadProgressChanged event as it progresses, and DownloadFileCompleted when completed.

You can control concurrency using a semaphore queue or, if you are using .NET 4.0, BlockingCollection . For instance:

 // Information used in callbacks. class DownloadArgs { public readonly string Url; public readonly string Filename; public readonly WebClient Client; public DownloadArgs(string u, string f, WebClient c) { Url = u; Filename = f; Client = c; } } const int MaxClients = 4; // create a queue that allows the max items BlockingCollection<WebClient> ClientQueue = new BlockingCollection<WebClient>(MaxClients); // queue of urls to be downloaded (unbounded) Queue<string> UrlQueue = new Queue<string>(); // create four WebClient instances and put them into the queue for (int i = 0; i < MaxClients; ++i) { var cli = new WebClient(); cli.DownloadProgressChanged += DownloadProgressChanged; cli.DownloadFileCompleted += DownloadFileCompleted; ClientQueue.Add(cli); } // Fill the UrlQueue here // Now go until the UrlQueue is empty while (UrlQueue.Count > 0) { WebClient cli = ClientQueue.Take(); // blocks if there is no client available string url = UrlQueue.Dequeue(); string fname = CreateOutputFilename(url); // or however you get the output file name cli.DownloadFileAsync(new Uri(url), fname, new DownloadArgs(url, fname, cli)); } void DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e) { DownloadArgs args = (DownloadArgs)e.UserState; // Do status updates for this download } void DownloadFileCompleted(object sender, AsyncCompletedEventArgs e) { DownloadArgs args = (DownloadArgs)e.UserState; // do whatever UI updates // now put this client back into the queue ClientQueue.Add(args.Client); } 

There is no need to explicitly manage threads or switch to TPL.

Answer 2

I think you should study the Task Parallel Library , which is new in .NET 4 and is designed to solve these problems.

Answer 3

Having a download of 100% cpu has nothing to do with downloading (since your network is almost always a bottleneck). I would say that you need to check your logic as you wait for the download to complete.

Can you post the code for the thread code that you run several times?

Answer 4

By creating 4 different background jobs, you will create separate threads that will no longer interfere with your GUI. Background workers are easy to implement, and I understand that they will do exactly what you need.

Personally, I would do this and simply let others not start until the previous one is complete. (Or maybe one, and allow it to execute one method at a time in the correct order.)

FYI - Background Worker