Why entity + mysql provider enumeration returns partial results without any exceptions

Question

I am trying to understand a situation where I have an entity infrastructure on .net 3.5 sp1 + MySQL 6.1.2.0 as a provider. It includes the following code:

Response.Write("Products: " + plist.Count() + "<br />"); var total = 0; foreach (var p in plist) { //... some actions total++; //... other actions } Response.Write("Total Products Checked: " + total + "<br />"); 

In general, the total production varies in each run, and it does not correspond to the total amount in the plist. It varies widely: from ~ 1/5 to half.

Inside foreach, there is no control flow code, i.e. do not interrupt, continue, try / catch, conditions around total ++, everything that can affect the score. As a confirmation, there are other totals recorded within the cycle associated with the actions, and they correspond to lower and higher total runs.

I cannot find the reason for the above, except for anything in the entity infrastructure or the mysql provider that causes it to complete foreach when the element is retrieved.

The foreach body can have good changes over time, since the actions are related to access to files and the network, my best shot is when the .net code goes beyond a certain threshold, there is some type of timeout in the base structure / provider and instead In order to raise an exception, it silently reports the absence of elements for enumeration .

Can someone give light in the above scenario and / or confirm if the framework / mysql operator entity has the above behavior?

---

Update 1: I cannot reproduce the behavior using Thread.Sleep in a simple foreach in a test project, not sure where else to look for this strange behavior: (.

---

Update 2: in the example above .Count () always returns the same + the correct number of elements. Using ToList or ToArray, as suggested, allows you to bypass the problem as expected (without flow control instructions in the foreach body) and both coincidence + matches do not change with each run.

I am interested in what causes this behavior in the framework + mysql entity. It would really not be necessary to change the code in all projects that use the framework + mysql entity to do .ToArray before listing the results, because I don't know when it will swallow some results. Or, if I do, at least I know what / why it happened.

Answer 1

The problem, reason and workaround are described in this mysql bug .

As expected, this is a timeout error in the provider, but this is not a regular timeout, i.e. net_write_timeout. Therefore, simple playback in the test project did not work, because the timeout applies to all foreach loops, and not just to the very long body between reading two lines.

At the moment, the problem is present in the latest version of the MySql provider and under normal conditions will only affect scripts in which lines are read during a long connection (which may or may not include a slow query), This is great because it does not affect all previous projects in which I used MySql / applying a workaround to sources also means that it does not fail.

Ps. a couple of seemingly related mysql errors: 1 , 2

Answer 2

If the problem is with the supplier or something else, then you can solve / identify this by doing the enumeration before sorting through it:

 var realisedList = plist.ToArray(); foreach(var p in realisedList) { //as per your example } 

If after this the problem still persists, then

a) One of the actions in the enumerator raises an exception that is swallowed somewhere

b) The source data is really different every time.

UPDATE: (as per your comment)

[deleted - several enumeration elements according to your comment]

At the end of the day - I would call ToArray () to fix the problem in this case (if the Count () method is required to get the amount, just change it to .Length to the constructed array).

Perhaps MySql kills the connection during the enumeration and does not throw an EF error when the next MoveNext () is called. EF then simply dutifully replies, saying that the enumerated is simply over. If so, until the error in the provider is fixed, ToArray () is the way forward.

Answer 3

I think that you actually answered the answer in your question, but this may be data that causes the problem not in timeout. Here is a theory:

One (or several) lines in the result set contain some data that cause an exception / problem, when it falls into this line, the system thinks that it has reached the last line.

To check this, you can try:

• Order data and see if each returned number in each of the operators is the same.
• Select only the id column and see if the problem goes away.
• Remove all rows from the table, add them several times at a time to see if a particular row is causing a problem.

If this is a timeout problem, try changing the timeout in the connection string.

Answer 4

I believe this is due to the way EF handles lazy loading. You may need to use either Load() or Include() , and also check using the IsLoaded property in your processing loop. Check out these two links for more information:
http://www.singingeels.com/Articles/Entity_Framework_and_Lazy_Loading.aspx
http://blogs.msdn.com/jkowalski/archive/2008/05/12/transparent-lazy-loading-for-entity-framework-part-1.aspx

Sorry, I don't know more about EF to be more specific. We hope that the links provide enough information to get you started, while others can listen to any of your questions.