Generator by memory string with freeing the string ad byte [] in C #: memory exception

Question

I am working on C #. I want to know how I can free stringbuilder n byte [] .... Because I get an exception from memory when using the string builder .... Another thing is that String.Replace () also gives an exception from memory or is there another way to do the same .... please tell me how can i overcome these problems ... thanks in advance

Answer 1

How big is the line you are building? Do you know how big it will be ahead? If so, install Capacity StringBuilder.

If you do not, it will start at 16 and then double it every time you break through that limit. The capacity will go 16, 32, 64, 128.

Each time he does this, you will most likely need to reallocate memory.

If the data is so large that it is better to create it in a file stream rather than in StringBuilder?

Answer 2

If you get OOM, you store too much data. It seems to me that you are probably using TextWriter (to replace StringBuilder ) or BinaryWriter / Stream (to replace byte[] ) - in particular StreamWriter and FileStream (which write to the file).

For example:

 using (TextWriter dest = File.CreateText("out.txt")) { for (int i = 0; i < 100000; i++) { dest.Write(i); dest.Write(": "); dest.WriteLine(i % 5000); // why not } } 

The advantage of thread-based APIs like this is that after you write a data block, it can collect garbage, etc.

Answer 3

Here is the code for the experiment:

 using System; using System.Text; class Program { static void Main(string[] args) { StringBuilder sb = new StringBuilder(); try { //sb.Capacity = 690 * 1024 * 1024; while (true) sb.Append("asdf"); } catch (OutOfMemoryException) { Console.WriteLine("Died at: {0:N0} characters", sb.Capacity); Console.WriteLine("Memory used: {0:N0} bytes", GC.GetTotalMemory(false)); Console.ReadLine(); } } } 

The output on my computer (32-bit Win7):

 Died at: 268,435,456 characters Memory used: 537,091,632 bytes 

The 32-bit operating system provides a process with almost 2 gigabytes of virtual memory. As you can see, the sample program is significantly different from this, consuming only 25%. The problem is fragmentation of the virtual memory address space. Available 2 gigabytes should store both code and data. You can get an idea of ​​how a VM is cut using the SysInternal VMMap utility .

There is something you can do with this to understand how StringBuilder works. It uses an internal array to store the string. This array is redistributed as necessary to store the growing string, doubling the size each time. The Capacity property indicates the size of this array.

This causes a different kind of fragmentation, fragmentation of the heap. After the array doubles in size, the old array can be assembled, but it creates a “free block” that cannot be combined to become available for large contiguous distribution. This is a detail of the implementation of the heap of large objects and the Windows heap manager, Google "look-cache cache", to learn more about this.

One way is to quickly gobble up virtual memory space before it can get fragmentation. Delete the comment to see it at work. On my car:

 Died at: 723,517,440 characters Memory used: 1,447,256,944 bytes 

I fine-tuned the value of Capacity through experimentation, you may have to do the same to make it work on your computer. The difference is quite dramatic, almost 3 times more. Remember that it is very unlikely to play in a real program, time is critical. Or on another machine, for that matter, be sure to stay well below the cut-off point. In fact, you risk OOM after the buffer you use receives more than 134 million characters.