Exception while reading millions of characters in a JSON file [OutOfMemoryException]

I uploaded a JSON file recorded from Azure Blob repository. File size - 137 MB.

Character and string properties when opened with Notepad ++, as shown below: enter image description here

It takes about 1-2 seconds when I select "Edit with Notepad ++" from the file context menu. So, I decided to create a program to convert JSON to CSV format. But it looks like I came across some kind of exception. Currently, for viewing JSON content, I will show in RichTextBox, since it can view the content before I decide to convert it to CSV.

Event to load: -

private async void txtjsonname_DoubleClick(object sender, EventArgs e)
{
    OpenFileDialog ofd = new OpenFileDialog();
    ofd.Filter = "JSON Files (*.json)|*.json";
    ofd.InitialDirectory = @"C:\";
    ofd.Title = "Select single json file to be converted";
    ofd.Multiselect = false;
    if (ofd.ShowDialog() == DialogResult.OK)
    {
        rtbstat.Text = null;
        txtcsvname.Text = null;
        txtjsonname.Text = null;
        lblcsvpath.Text = null;
        lbljsonpath.Text = null;
        rtbjson.Clear();
        txtjsonname.Text = Path.GetFileName(ofd.FileName);
        lbljsonpath.Text = Path.GetDirectoryName(ofd.FileName);

        if (await LoadJSONtoRTB(ofd.FileName))
        {
            rtbjson.WordWrap = false;
            rtbstat.Text = "Load file finished! " + (rtbjson.Lines.Count()).ToString() + " line(s) detected | " + rtbjson.Text.Length.ToString() + " character(s) detected";
            txtcsvname.Text = Path.GetFileNameWithoutExtension(ofd.FileName) + ".csv";
        }
    }
    await Task.Delay(1000);
}

The code I'm trying to exclude from view: -

First approach: First code:

private async Task<bool> LoadJSONtoRTB(string path)
    {
        try
        {
            foreach (var line in File.ReadLines(path))
            {
                rtbjson.Text = line;
            }
            await Task.Delay(10);
            return true;
        }
        catch (Exception)
        {
            return false;
        }
    }

Second code:

private async Task<bool> LoadJSONtoRTB(string path)
    {
        try
        {
            using (FileStream fs = File.Open(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            using (BufferedStream bs = new BufferedStream(fs))
            using (StreamReader sr = new StreamReader(bs))
            {
                string line;
                while ((line = sr.ReadLine()) != null)
                {
                    rtbjson.AppendText(line);
                }
            }
            await Task.Delay(10);
            return true;
        }
        catch (Exception)
        {
            return false;
        }
    }

enter image description here : "System.AccessViolationException" System.Windows.Forms.dll

: . , .

: -

private async Task<bool> LoadJSONtoRTB(string path)
{
    try
    {
        StreamReader sr = new StreamReader(@path);
        while (!sr.EndOfStream)
            rtbjson.Text += sr.ReadLine();
        await Task.Delay(10);
        return true;
    }
    catch (Exception)
    {
        return false;
    }
}

, 12 , , . enter image description here

12 6 .

(json/txt) 64 , , ++ 1-2 , ?

+4
2

, Notepad ++ - System.IO.File.ReadAllText. , , . RichTextBox :

richTextBox1.Text = System.IO.File.ReadAllText(filePath);

Notepad ++ Scintilla, , RichTextBox.

ScintillaNET, Scintilla.

, RichTextBox:

scintilla1.Text = System.IO.File.ReadAllText(filePath);
+2

LoadJSONtoRTB . , gui ( ) . gui :

this.Invoke(new Action(() => { rtbjson.Text += sr.ReadLine(); }));

, , , StringBuilder. gui gui. , Form.Invoke

+1

Source: https://habr.com/ru/post/1667339/


All Articles