HttpClient - the task has been canceled - How to get the exact error message?

Question

I have the following test code. I always get the "The task was canceled" error after a cycle of 316934 or 361992 times.

If I am not mistaken, there are two possible reasons why the task was canceled. a) HttpClient received a timeout or b) there are too many tasks in the queue, and some tasks have timed out.

I could not find the documentation about the restriction in the sequence of tasks. And I tried to create more than 500 thousand tasks and without a timeout. I think reason "b" may be wrong.

Q1. Is there another reason I missed?

Q2. If this is due to the HttpClient timeout, how can I get the exact exception message instead of the TaskCancellation exception.

Q3. What would be the best way to fix this? Should I introduce a throttle?

Thank!

var _httpClient = new HttpClient(); _httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept", "text/html,application/xhtml+xml,application/xml"); _httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Encoding", "gzip, deflate"); _httpClient.DefaultRequestHeaders.TryAddWithoutValidation("User-Agent", "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0"); _httpClient.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Charset", "ISO-8859-1"); int[] intArray = Enumerable.Range(0, 600000).ToArray(); var results = intArray .Select(async t => { using (HttpRequestMessage requestMessage = new HttpRequestMessage(HttpMethod.Get, "http://www.google.com")) { log.Info(t); try { var response = await _httpClient.SendAsync(requestMessage); var responseContent = await response.Content.ReadAsStringAsync(); return responseContent; } catch (Exception ex) { log.ErrorException(string.Format("SoeHtike {0}", Task.CurrentId), ex); } return null; } }); Task.WaitAll(results.ToArray()); Console.ReadLine(); 

The following is a step to replicate the problem.

• Create a console project in VS 2012.

• Copy and paste my code into Main.

• Place a breakpoint on this line "log.ErrorException (string.Format (" SoeHtike {0} ", Task.CurrentId), ex);"

Run the program in debug mode. Wait a few minutes. (maybe 5 minutes?) I just checked my code and I got an exception after 3 minutes. If you have a violinist, you can track requests so that you know that the program is still working or not.

Feel free to let me know if you can not reproduce the problem.

Answer 1

The default value of HttpClient.Timeout is 100 seconds (00:01:40). If you make a timestamp in your catch , you will notice that tasks will begin to be canceled at that exact time. There seems to be a limited number of HTTP requests you can make per second, while others are in the queue. Requests in the queue are timed out. Of all 600 thousand tasks, I personally received only 2500 successful ones, others were canceled.

I also find it unlikely that you can run just 600,000 tasks. Many network drivers skip a large number of requests only for a short time, and after a while reduce this number to a very low value. My network card allowed me to send a total of 921 requests in 36 seconds and lowered this speed by only one request per second. At this speed, it takes a week to complete all the tasks.

If you can get around this restriction, make sure that you are creating code for a 64-bit platform, since the application is very memory-hungry.

Answer 2

Do not use the HttpClient instance that you are using. Strange, but fixed this problem for me.

Answer 3

just wanted to share. I had a similar code to upload tests to our servers, and its very likely that your requests will be disconnected. You can set a timeout for your max HTTP request and see if it changes anything for you. I tried to hit our servers by creating various threads. And this has increased the number of views, but they will all eventually run out of time. And also you cannot set a timeout when getting into another thread.