How to get specific lines from a text file in C #?

Question

I am working in C # and I have a large text file (75 MB) I want to keep the lines matching the regular expression

I tried reading the file using streamreader and ReadToEnd, but it takes 400 MB of RAM

and when reused, throws an exception to the memory.

Then I tried using File.ReadAllLines ():

string[] lines = File.ReadAllLines("file");

StringBuilder specialLines = new StringBuilder();


foreach (string line in lines)

 if (match reg exp)

  specialLines.append(line);

This is all great, but when my function ends, the memory is not cleared, and I have 300 MB of used memory left, only when the function is called and the line is executed: string [] lines = File.ReadAllLines ("file"); I see that clearing memory up to 50 MB gives or takes and then redistributes back up to 200 MB.

How can I clear this memory or get the lines I need differently?

Answer 1

        var file = File.OpenRead("myfile.txt");
        var reader = new StreamReader(file);
        while (!reader.EndOfStream)
        {
            string line = reader.ReadLine();
            //evaluate the line here.
        }
        reader.Dispose();
        file.Dispose();

Answer 2

. , Linq:

static class ExtensionMethods
{
    public static IEnumerable<string> EnumerateLines(this TextReader reader)
    {
        string line;
        while((line = reader.ReadLine()) != null)
        {
            yield return line;
        }
    }
}

...

var regex = new Regex(..., RegexOptions.Compiled);
using (var reader = new StreamReader(fileName))
{
    var specialLines =
        reader.EnumerateLines()
              .Where(line => regex.IsMatch(line))
              .Aggregate(new StringBuilder(),
                         (sb, line) => sb.AppendLine(line));
}

Answer 3

You can use StreamReader # ReadLine to read the file in turn and to save the lines you need.

Answer 4

You should use the Enumerator template to keep memory low in case your file can be huge.