How to avoid memory exception when using PLINQ?

Question

Hi and thanks for watching!

Background

I have a computational task that requires either a lot of time or parallel computing.

In particular, I need to scroll through a list of approximately 50 images, Base64 encode them, and then calculate the Levenshtein distance between each newly encoded element and the values ​​in the XML file containing about 2000 Base64 encoded image files to find the line in the XML file that has the most little lion. The distance from the control line.

The regular foreach works, but is too slow, so I decided to use PLINQ to use my multi-core Core i7 processor:

 Parallel.ForEach(candidates, item => findImage(total,currentWinner,benchmark,item)); 

The task starts brilliantly, moving forward at high speed, but then I get the exception "Out of Memory".

I am using C #, .NET 4, Forms App.

Question

How do I configure my PLINQ code so that I don’t have any free memory?

Update / Sample Code

Here is the method that is called to enter PLINQ foreach :

 private void btnGo_Click(object sender, EventArgs e) { XDocument doc = XDocument.Load(@"C:\Foo.xml"); var imagesNode = doc.Element("images").Elements("image"); //Each "image" node contains a Base64 encoded string. string benchmark = tbData.Text; //A Base64 encoded string. IEnumerable<XElement> candidates = imagesNode; currentWinner = 1000000; //Set the "Current" low score to a million and bubble lower scores into it place iteratively. Parallel.ForEach(candidates, i => { dist = Levenshtein(benchmark, i.Element("score").Value); if (dist < currentWinner) { currentWinner = dist; path = i.Element("path").Value; } }); } 

.,. And here is the Levenshtein distance method:

 public static int Levenshtein(string s, string t) { int n = s.Length; int m = t.Length; var d = new int[n + 1, m + 1]; // Step 1 if (n == 0) { return m; } if (m == 0) { return n; } // Step 2 for (int i = 0; i <= n; d[i, 0] = i++) { } for (int j = 0; j <= m; d[0, j] = j++) { } // Step 3 for (int i = 1; i <= n; i++) { //Step 4 for (int j = 1; j <= m; j++) { // Step 5 int cost = (t[j - 1] == s[i - 1]) ? 0 : 1; // Step 6 d[i, j] = Math.Min( Math.Min(d[i - 1, j] + 1, d[i, j - 1] + 1), d[i - 1, j - 1] + cost); } } // Step 7 return d[n, m]; } 

Thanks in advance!

Answer 1

Update

Again, this error appeared today under various circumstances. I have been working on a high demand memory desktop application. Make sure you set up a project for 64-bit architecture to access all available memory. My project was installed on x86 by default, so I kept getting exceptions from memory. Of course, this only works if you can count on 64-bit processors for deployment.

Final update

After fighting this, it looks like an operator error:

I made calls to the user interface thread from parallel threads to update the execution labels, but I did not do this in a thread-safe way.

In addition, I ran the application without a debugger, so every time the code tried to update the UI thread from the parallel thread that caused the overflow, an uncaught exception was thrown.

Not being an expert on PLINQ, I assume that it handles all the low-level distribution resources for you, until you make stupid smelly code like this.

Hope this helps someone else.