C # .NET garbage collection question

Question

I am having a problem in my application with an OutOfMemoryException. My application can search for words in texts. When I start searching for a long time to search for about 2000 different texts for about 2175 different words, the application will end by about 50% with an OutOfMemoryException (after about 6 hours of processing).

I am trying to find a memory leak. I have a graph of objects, for example: (-> - links)

a static global application object (controller) → an algorithm starter object → a start text launch object → an intelligent search algorithm object (this object performs a search).

The text start starter object will launch the run () method, a method for the text search method in a separate thread.

To fix the problem, I edited the code so that the text markup start-up object object divided the texts into search into several groups and initialized one text-mining algorithm object for each group of texts sequentially (therefore, when one text-mining algorithm is created, the object will be created to search for the next text groups). Here I set the previous smart search algorithm object to null. But this does not solve the problem.

When I create a new object of the mining algorithm, I have to give it some parameters. They are taken from the properties of the previous object of the mining algorithm before I set this object to null. Will this prevent garbage collection of the object of the mining algorithm?

:

    private void RunSeveralAlgorithmObjects()
    {

        IEnumerable<ILexiconEntry> currentEntries = allLexiconEntries.GetGroup(intCurrentAlgorithmObject, intNumberOfAlgorithmObjectsToUse);

        algorithm.LexiconEntries = currentEntries;
        algorithm.Run();

        intCurrentAlgorithmObject++;

        for (int i = 0; i < intNumberOfAlgorithmObjectsToUse - 1; i++)
        {
            algorithm = CreateNewAlgorithmObject();
            AddAlgorithmListeners();
            algorithm.Run();
            intCurrentAlgorithmObject++;
        }

    }

    private TextMiningAlgorithm CreateNewAlgorithmObject()
    {
        TextMiningAlgorithm newAlg = new TextMiningAlgorithm();

        newAlg.SortedTermStruct = algorithm.SortedTermStruct;
        newAlg.PreprocessedSynonyms = algorithm.PreprocessedSynonyms;
        newAlg.DistanceMeasure = algorithm.DistanceMeasure;
        newAlg.HitComparerMethod = algorithm.HitComparerMethod;
        newAlg.LexiconEntries = allLexiconEntries.GetGroup(intCurrentAlgorithmObject, intNumberOfAlgorithmObjectsToUse);
        newAlg.MaxTermPercentageDeviation = algorithm.MaxTermPercentageDeviation;
        newAlg.MaxWordPercentageDeviation = algorithm.MaxWordPercentageDeviation;
        newAlg.MinWordsPercentageHit = algorithm.MinWordsPercentageHit;
        newAlg.NumberOfThreads = algorithm.NumberOfThreads;
        newAlg.PermutationType = algorithm.PermutationType;
        newAlg.RemoveStopWords = algorithm.RemoveStopWords;
        newAlg.RestrictPartialTextMatches = algorithm.RestrictPartialTextMatches;
        newAlg.Soundex = algorithm.Soundex;
        newAlg.Stemming = algorithm.Stemming;
        newAlg.StopWords = algorithm.StopWords;
        newAlg.Synonyms = algorithm.Synonyms;
        newAlg.Terms = algorithm.Terms;
        newAlg.UseSynonyms = algorithm.UseSynonyms;

        algorithm = null;

        return newAlg;
    }

, :

            // Run the algorithm in it own thread
            Thread algorithmThread = new Thread(new ThreadStart
                (RunSeveralAlgorithmObjects));
            algorithmThread.Start();

- , ?

Answer 1

, . (, ).

, :

• . Properties -> Debug -> Enable unmanaged code debugging.
• . , , , , , 6 ; , Debug -> Break All.
• Immediate. Debug -> Windows -> Immediate

• , , 32 64 ,.NET 2.0/3.0/3.5 .NET 4.0:

  .load C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll 32- .NET 2.0-3.5

  .load C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\sos.dll 32- .NET 4.0

  .load C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\sos.dll 64- .NET 2.0-3.5

  .load C:\WINDOWS\Microsoft.NET\Framework64\v4.0.30319\sos.dll 64- .NET 4.0

• SoS Immediate. !dumpheap -stat, , !finalizequeue.

:

• (), VS .
• , , ! ( ).

Tess Microsoft, , .NET.

, , . , , .

Answer 2

1) , (AddAlgorithmListeners ), "" . , , - . , , .

---

2) , (, -) :

private void RunSeveralAlgorithmObjects()
{
    ...
    algorithm.LexiconEntries = currentEntries;
    // ^ when/where is algorithm initialized?

    for (...)
    {
        algorithm = CreateNewAlgorithmObject();
        ....
    }
}

Is algoritm ? algorithm.LexiconEntries . , , , .

, , , CreateNewAlgorithmObject algorithm . , algorithm , , . "" , .

PS: , , , CreateNewAlgorithmObject void algorithm .

Answer 3

AddAlgorithmListeners , ? , , .

, , .

for (int i = 0; i < intNumberOfAlgorithmObjectsToUse - 1; i++)
        {
            algorithm = CreateNewAlgorithmObject();
            AddAlgorithmListeners();
            algorithm.Run();
            RemoveAlgoritmListeners();    // See if this fixes your issue.
            intCurrentAlgorithmObject++;
        }

Answer 4

AddAlgorithmListeners();, , ?

Answer 5

IEnumerable GetGroup() ? , , , , , .

Memory profiling is useful, have you tried exploring the application with the profiler? I found Red Gate useful in the past (it's not free, but has an evaluation version, IIRC).