Output from Parallel.ForEach on LINQ

Question

I iterate over LINQ to SQL binding using Parallel.ForEach . I exit the loop when a certain number of elements are processed. Processing stops, but the loop then freezes for a few seconds before I get the error:

System.Data.SqlClient.SqlErrorCollection: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.

How to make a loop exit normally? Forcing a local collection by adding say ToList() to the query is not an option. I also tried wrapping everything in the using block to no avail. note that

Here is the code:

 var query = SomeDataContext.SomeTableMapping; // All ok, if I append Take(maxRecords) int maxRecords = 1000; Parallel.ForEach(query, (queryResult, pLoopState, idx) => { // Do whatever here on queryResult. if (idx > maxRecords) { Console.WriteLine("Reached maximum number of records: {0}", maxRecords); pLoopState.Break(); } }); 

Thanks,

/ David

Answer 1

You can use an instance of the CancellationTokenSource class to cancel a parallel loop. For more information: How to cancel a Parallel.For or ForEach loop

It is important to note that when the cancel token in a call is inside a parallel loop, the execution of an already running iteration will not be stopped. It’s just that no new iterations will be launched.

There are other ways to interrupt / stop a parallel loop using the Break () and Stop () method of the ParallelLoopState class. When you initialize a Parallel.For / ForEach loop, you can pass an instance of the ParallelLoopState class and use this instance to call the Break / Stop method.

The interrupt method behaves a little differently than Stop. In the case of Stop, the infrastructure asks to stop the iteration as soon as possible. In Break mode, the infrastructure requests a loop to stop iterations outside the current iteration as soon as possible. If you are looking for a specific key / text and want to split it as soon as you find it, you should use the Stop () method.

Answer 2

Although the Break state is one of two ways to stop the parallel loop, exiting the delegate does not affect the execution of the loop, since the delegate body is disconnected from the loop structure at each execution. This explains why you have been observing this for a while (and I agree that at first glance it seems contradictory that Break is different from the semantics we are used to in loops). The loop is still working!

The decision will depend on what you are trying to execute in a loop, but you can experiment with ...

 if (pLoopState.ShouldExitCurrentIteration) { return; } 

instead of Break. This should make you go past the part where it seems to be hanging.

Answer 3

You should probably check the SQL statements that you are executing, not the loop. The loop will process ALL rows returned by the statement in parallel. Exiting the loop will not stop the execution of the instruction; it will just stop processing the results.

LINQ executes a query when you try to list its results, not when you try to access one of them. In your example, this is the moment when you pass the request variable to Parallel.ForEach and convert to IEnumerable. Then Parallel.ForEach takes each row of results and tries to process it in parallel.

I suspect you have a large table that you are querying without any WHERE criteria. As a result, your connection timed out after the default execution timeout (about 60 seconds, I think). If you want to get a certain number of rows, you should use the e TOP command in SQL or the Take () method in LINQ, for example. calling

 Parallel.ForEach(query.Take(10), 

instead of just passing the request.

If you want to limit the rows returned by the SQL or LINQ statement, you must do this in the expression itself, and not try to limit the results after they are returned. Getting unnecessary row results will significantly slow down your database server and lead to possible deadlocks.