StringBuilder to concatenate strings throws an OutOfMemoryException

Question

We mainly adhere to the above best practices.

Take a look at String vs StringBuilder

But StringBuilder may throw an OutOfMemoryException, even if there is enough memory . It throws an OOM exception because it needs a "contiguous block of memory."

Some reference links StringBuilder OutOfMemoryException

and there are many more ...

How many of you have encountered or realized this problem, and what have you done to solve it?

Is there something I am missing?

PS: I did not know about this.

I rephrased the question.

*** Same thing with manual concatenation (I will check this and update SO). Another thing that bothered me was that the system had enough memory. This is the reason I put this question here to check if anyone has come across this problem or something is wrong with the code.

Answer 1

The underyling string you create also needs a continuous block of memory because it is represented as an array of characters (arrays require continuous memory). If StringBuilder throws an OOM exception, you cannot create base ones without it.

If creating a string causes OOM, most likely your application will have a more serious problem.

Edit in response to clarification:

There is a small subset of cases where building a string using StringBuilder fails, when manual concatenation succeeds. Manual concatenation will use the exact length needed to concatenate the two strings, while StringBuilder has a different algorithm for allocating memory. It is more aggressive and most likely allocates more memory than is really necessary for the string.

Using StringBuilder will also temporarily double the required memory, as the string will be present in the form of System.String and StringBuilder at the same time for a short time.

But if one of the methods calls OOM and the other does not, this probably indicates a more serious problem in your program.

Answer 2

If a StringBuilder is about to throw an OutOfMemoryException in your specific situation, then doing a manual string concatenation is NOT the best solution; it is much worse. This is exactly the case (creating an extremely long string) where you intend to use StringBuilder. Manually concatenating a string of this large amount will take up memory many times, creating a string using StringBuilder.

However, on a modern computer, if your line starts the computer from contiguous memory, your design is deeply, deeply corrupted. I cannot imagine what you could do to create a large string.

Answer 3

How much memory are we talking about? I'm not talking about free or full memory in the system, but how long do you concatenate a string?

The memory overflow exception is almost always a very bad sign of your code, even if it fails long before the memory actually runs out, as you survived due to lack of read-only memory.

At this point you should really rebuild the code.

For example, here are some ways to deal with the problem:

• Do not store so much memory in memory at a time by putting it on a disk or something like that
• Separate it, save the list of string / string constructions and add them to a certain length before switching to a new one, while preserving the "continuous memory" problem.
• Rebuild the algorithm to avoid creating gigabytes of data in memory at all

Answer 4

If you use soclose for your memory limitations, which is even a problem, then you probably should consider a different architecture or get a more rigid machine.

Answer 5

If you look at how StringBuilder implemented, you will see that it actually uses String to store data ( String has internal methods that allow StringBuilder to change in place).

those. they both use the same amount of memory. However, since StringBuilder automatically expands the base array and if necessary copies (but doubles the capacity), which is most likely the cause of an error from memory. But, as already noted, both of them require a continuous block of memory,

Answer 6

Well, actually the question is, why do you need to work with strings for a long time? If you stumble upon this problem, more than likely you should change your concept.

These problems even affect the System.String class, so you should rather add your list to the List <string> and process the data in parallel, which should improve overall performance when spelled correctly.

Answer 7

I came across this exception with very large strings that were built using various string constructions (which should not have caused problems, as they were declared in anonymous functions), and finally solved this by reusing one StringBuilder declared outside of anonymous functions.

Answer 8

I had a very similar experience when I added strings, but forgot to add String.Format. In this way:

 myStringBuilder.Append("1,""{0}""", someVeryLargeIntVariable) 

should have been:

 myStringBuilder.Append(String.Format("1,""{0}""", someVeryLargeIntVariable)) 

Please note that this is my vb.net code that failed. I repeated a similar test in C # with:

 myStringBuilder.Append('a', 1564544656); 

against.

 myStringBuilder.Append(string.Format("1,\"{0}\"", 1564544656)); 

But in my case, vb.net caused me b / c implicit conversion problems (I could not parallel with the same problem in C #).

I hope this helps someone.