How to identify parallel HTTP request bottleneck in .NET?

Question

I am trying to make as many HTTP requests for a URL as possible .

I use this code to allow throttling of the maximum Parallelism Degrees, so I do not overfill the memory, creating multiple sets and Tasks once.

  public static Task ForEachAsync<T>(this IEnumerable<T> source, int dop, Func<T, Task> body) { return Task.WhenAll( from partition in Partitioner.Create(source).GetPartitions(dop) select Task.Run(async delegate { using (partition) while (partition.MoveNext()) await body(partition.Current); })); } 

This seems to work fine.

body() essentially boils down to:

 async Task Body() { var r = WebRequest.Create("// the url"); await r.GetResponseAsync(); } 

However, I seem to have a bottleneck. If I try to iterate 2500 with variable values ​​for dop , I get the following results:

 DOP: 50 Total Time: 00:00:14.4801781 Average (ms): 246.6088 StDev: 84.1327983759009 DOP: 75 Total Time: 00:00:09.8089530 Average (ms): 265.758 StDev: 110.22912244956 DOP: 100 Total Time: 00:00:11.9899793 Average (ms): 344.9168 StDev: 173.281468939295 DOP: 200 Total Time: 00:00:09.1512825 Average (ms): 627.0492 StDev: 572.616238312676 DOP: 500 Total Time: 00:00:09.3556978 Average (ms): 1361.5328 StDev: 1798.70589239157 DOP: 750 Total Time: 00:00:12.6076035 Average (ms): 2009.058 Normal Total: 5022646 StDev: 2348.20874093199 DOP: 1000 Total Time: 00:00:11.4721195 Average (ms): 2453.782 StDev: 2481.56238190299 DOP: 2000 Total: 00:00:11.6039888 Average (ms): 4100.5536 StDev: 2459.36983911063 

dop=50 seems to be less than a bottleneck. If you exceed dop~=100 , you will notice the Average time that each request takes (the average value of the Func<T, Task> body to run 2500 times) increases almost linearly with dop (theres bit noise in these results, but they repeat with a small error).

This suggests that the body has a "queue", right?

I already install

 ServicePointManager.DefaultConnectionLimit = int.MaxValue; 

and if i do

 servicePoint = ServicePointManager.FindServicePoint("// the url", null); 

and monitor

 servicePoint.CurrentConnections 

each time the body executed, it is always dop (with the exception of the built-in rise and return).

I tried this from different networks, so it is unlikely to be hardware based, and it should not be a remote server, since it is designed for heavy incoming loads (not that the numbers I'm talking about are even heavy)

What is the best way to find out what I am doing?

Answer 1

The total runtime of all operating levels is from 9 to 11 seconds. This makes sense, because by increasing DOP (exponentially), you will eventually saturate the underlying resource or network or something else.

I bet if you placed lower DOP check digits, we would see higher overall times.

When you double the number of simultaneous requests at this point, the average completion time doubles.

Look at throughput measured in points per second or total time. This is an interesting metric. The delay between the elements is not.