Stack overflow on debug but not release

Question

I have the following code that parses a text file and indexes words and lines:

bool Database::addFromFileToListAndIndex(string path, BSTIndex* & index, list<Line *> & myList) { bool result = false; ifstream txtFile; txtFile.open(path, ifstream::in); char line[200]; Line * ln; //if path is valid AND is not already in the list then add it if(txtFile.is_open() && (find(textFilePaths.begin(), textFilePaths.end(), path) == textFilePaths.end())) //the path is valid { //Add the path to the list of file paths textFilePaths.push_back(path); int lineNumber = 1; while(!txtFile.eof()) { txtFile.getline(line, 200); ln = new Line(line, path, lineNumber); if(ln->getLine() != "") { lineNumber++; myList.push_back(ln); vector<string> words = lineParser(ln); for(unsigned int i = 0; i < words.size(); i++) { index->addWord(words[i], ln); } } } result = true; } return result; } 

My code runs flawlessly and fairly quickly until I give it a HUGE text file. Then I get the error from Visual Studio. When I switch to "Release", the code runs without a hitch. Is there something wrong with my code or are there any limitations when running the Debug configuration? Am I trying to do too much in one function? If so, how can I break it so that it does not break during debugging?

EDIT Upon request, my addWord implementation;

 void BSTIndex::addWord(BSTIndexNode *& pCurrentRoot, string word, Line * pLine) { if(pCurrentRoot == NULL) //BST is empty { BSTIndexNode * nodeToAdd = new BSTIndexNode(); nodeToAdd->word = word; nodeToAdd->pData = pLine; pCurrentRoot = nodeToAdd; return; } //BST not empty if (word < (pCurrentRoot->word)) //Go left { addWord(pCurrentRoot->pLeft, word, pLine); } else //Go right { addWord(pCurrentRoot->pRight, word, pLine); } } 

And lineParser:

 vector<string> Database::lineParser(Line * ln) //Parses a line and returns a vector of the words it contains { vector<string> result; string word; string line = ln->getLine(); //Regular Expression, matches anything that is not a letter, number, whitespace, or apostrophe tr1::regex regEx("[^A-Za-z0-9\\s\\']"); //Using regEx above, replaces all non matching characters with nothing, essentially removing them. line = tr1::regex_replace(line, regEx, std::string("")); istringstream iss(line); while(iss >> word) { word = getLowercaseWord(word); result.push_back(word); } return result; } 

Answer 1

Stack overflow indicates that you have run out of stack space (probably obviously, but just in case). Typical reasons are non-limiting or excessive recursion, or very large duplication of the stack object. Oddly enough, this may be the case in this case.

It is likely that in Release, your compiler performs tail call optimization, which prevents stack overflows due to excessive recursion.

It is also likely that in Release, your compiler will optimize the inverse of the vector from lineParser.

So, you need to find out which condition is overcrowded in Debug, I would start with recursion as the most likely offender, trying to change the type of string parameter to a link, i.e.

 void BSTIndex::addWord(BSTIndexNode *& pCurrentRoot, string & word, Line * pLine) 

This should prevent you from duplicating the word object with each call to addWord.

Also consider adding std :: cout <"recursing addWord" <std :: css; type in addWord so you can see how deep it goes, and if its completion is correct.

Answer 2

The problem is almost certainly a recursive call in addWord - in a non-optimized assembly this will consume a lot of stack space, while in an optimized assembly the compiler will turn it into a tail call that repeats the same stack frame.

You can easily convert a recursive call in a loop:

 void BSTIndex::addWord(BSTIndexNode ** pCurrentRoot, string word, Line * pLine) { while (*pCurrentRoot != NULL) { //BST not empty if (word < (*pCurrentRoot)->word) //Go left { pCurrentRoot = &(*pCurrentRoot)->pLeft; } else //Go right { pCurrentRoot = &(*pCurrentRoot)->pRight; } } //BST is empty BSTIndexNode * nodeToAdd = new BSTIndexNode(); nodeToAdd->word = word; nodeToAdd->pData = pLine; *pCurrentRoot = nodeToAdd; } 

Answer 3

You should also post your stack, which will actually show what caused the overflow. Obviously, recursion in addWord consumes a lot of memory.

If you just want it to work, go to your compiler / linker settings and increase the size reserved for your stack. By default, it is only 1 MB, roll it up to 32 MB or something else, and you will see that any additional counter or trial version of the debug assembly, you will have enough stack to process it.

Answer 4

You can increase the size of the stack to the appropriate number of bytes.

 #pragma comment(linker, "/STACK:1000000000")