Using Task.WhenAll with endless tasks created by BlockingCollection

Question

I am adding background tasks to the lock assembly (added in the background).

I am waiting with Task.WhenAll in the Enumerable returned by GetConsumingEnumerable.

My question is: is Task.WhenAll overloaded, which gets IEnumerable "prepared" to potentially get an infinite number of tasks?

I'm just not sure if I can do it this way or if it is intended to be used that way?

private async Task RunAsync(TimeSpan delay, CancellationToken cancellationToken) { using (BlockingCollection<Task> jobcollection = new BlockingCollection<Task>()) { Task addingTask = Task.Run(async () => { while (true) { DateTime utcNow = DateTime.UtcNow; var jobs = Repository.GetAllJobs(); foreach (var job in GetRootJobsDue(jobs, utcNow)) { jobcollection.Add(Task.Run(() => RunJob(job, jobs, cancellationToken, utcNow), cancellationToken), cancellationToken); } await Task.Delay(delay, cancellationToken); } }, cancellationToken); await Task.WhenAll(jobcollection.GetConsumingEnumerable(cancellationToken)); } } 

Answer 1

Task.WhenAll will not work with an infinite number of tasks. First (synchronously) wait for the listing to complete, and then (asynchronously) wait for them to complete.

If you want to respond to a sequence asynchronously, you need to use IObservable<Task> (Reactive Extensions). You can use the TPL BufferBlock data BufferBlock as a "queue", which can work with both synchronous and asynchronous code and is easily converted to IObservable<Task> .

Answer 2

Since your goal is simply to wait for the cancellation token to be canceled , you must do this. For reasons that others have explained, using WhenAll in an infinite sequence of tasks is not the way to go. There are simpler ways to get a task that will never be completed.

 await new TaskCompletionSource<bool>().Task .ContinueWith(t => { }, cancellationToken); 

Answer 3

I assume that Task.WhenAll will try to enumerate the collection, which means that it itself will be blocked until the collection is completed or canceled. If this is not the case, then the code will theoretically end await before creating the tasks. So there will be an extra block ... it will block waiting for threads to be created, and then block again until the tasks are completed. I do not think this is bad for your code, since it is still blocked until the same point in time.