I am loading a datatable that has aprox. 60,000 lines.
I loop them and write them in such a way as to build the file:
HttpContext.Current.Response.Write(lineItem); HttpContext.Current.Response.Write(Environment.NewLine);
When I repeat ~ 15-25,000 lines, it works for 5 seconds and the file is generated perfectly. However, there seems to be a limit when it suddenly takes a lot of time and then expires (?).
I get an error message:
System.Threading.ThreadAbortException: thread terminated.
in System.AppDomain.GetId () in System.Threading.Thread.get_CurrentCulture () in System.String.IndexOf (String value) in Modules.DownloadReports.RemoveTags (String sat Modules.DownloadReports.WriteProductData (DataTable products)
My timeout in web.config is 3600 seconds, for both applications and SQL calls.
My code is:
private void WriteProductData(DataTable products) { StringBuilder bld = new StringBuilder(); try { //Column names const string str = "OrderId; ProductId; ProductName; SeriesName; BrandName; PrimaryCategory; Content; ContentUnit; Quantity; ListPrice; PlacedPrice; Status"; bld.AppendLine(str); bld.AppendLine(Environment.NewLine); //Data string trackingNumber, productId, productName, seriesName, brandName, prodCategory, content, contentUnit, quantity, listPriceUnit, placedPriceUnit, status; string lineItem; string errorString = string.Empty; foreach (DataRow product in products.Rows) { // initialize all the different strings lineItem = string.Format("{0};{1};{2};{3};{4};{5};{6};{7};{8};{9};{10};{11};", trackingNumber, productId, productName, seriesName, brandName, prodCategory, content, contentUnit, quantity, listPriceUnit, placedPriceUnit, status); //bld.AppendLine(lineItem); //bld.AppendLine(Environment.NewLine); HttpContext.Current.Response.Write(lineItem); HttpContext.Current.Response.Write(Environment.NewLine); } } catch (Exception ex) { Logger.ErrorFormat(ex, "An exception occured during writing of product data to BI report"); throw; } finally { //Response.Write(bld.ToString()); } }
What have I tried ...
I optimized the loop as much as possible, so all that was left was to build a string concatenating with string.format.
I also tried using Stringbuilder to create a large string and then write it at the end.
So my questions are ...
- Why am I even getting a Threas exception? It will take about 75 seconds before it gives me an error, not 5 minutes
- ~ 15-25,000 lines take 5 seconds, so my logic says 60,000 should take a maximum of 15 seconds. What could go wrong here?
UPDATE:
I solved the problem by eliminating the cause of the long operation. There was a string operation that was pretty slow. And when she was called 3 times in a row, she did everything very slowly.
However, this is not the root of the problem. This is just a pragmatic fix that will work until the data load becomes much larger.