How to get around an OutOfMemory exception in C #?

Question

I have some huge xml files, 1+ gb. I need to do some filtering operations with them. The easiest idea that I came up with is to save them as txt and ReadAllText from them, and start doing some operations like

var a = File.ReadAllText("file path"); a = a.Replace("<", "\r\n<"); 

The moment I try to do this, the program crashes from memory. I looked at my task manager while I run it, and RAM usage increases to 50%, and as soon as it reaches this, the program dies.

Does anyone have any ideas on how I work with this file, avoiding the OutOfMemory exception or letting the program extract most of the memory.

Answer 1

If you can do this line-by-line, instead of "Reading everything into memory" with File.ReadAllText , you can say "Let me have one line in time" with File.ReadLines .

This will return an IEnumerable that uses deferred execution. You can do it as follows:

 using(StreamWriter sw = new StreamWriter(newFilePath)) foreach(var line in File.ReadLines(path)) { sw.WriteLine(line.Replace("<", "\r\n<")); sw.Flush(); } 

If you want to know more about deferred execution, you can check out this github page.