How to parse a huge JSON file as a stream in Json.NET?

Question

I have a very, very large JSON file (1000+ MB) of identical JSON objects. For instance:

[
    {
        "id": 1,
        "value": "hello",
        "another_value": "world",
        "value_obj": {
            "name": "obj1"
        },
        "value_list": [
            1,
            2,
            3
        ]
    },
    {
        "id": 2,
        "value": "foo",
        "another_value": "bar",
        "value_obj": {
            "name": "obj2"
        },
        "value_list": [
            4,
            5,
            6
        ]
    },
    {
        "id": 3,
        "value": "a",
        "another_value": "b",
        "value_obj": {
            "name": "obj3"
        },
        "value_list": [
            7,
            8,
            9
        ]

    },
    ...
]

Each individual item in the JSON root list follows the same structure and, therefore, will be individually deserializable. I already have C # classes written to get this data, and deserializing a JSON file containing a single object without a list works as expected.

At first, I tried to just deserialize my objects in a loop:

JsonSerializer serializer = new JsonSerializer();
MyObject o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
    while (!sr.EndOfStream)
    {
        o = serializer.Deserialize<MyObject>(reader);
    }
}

This did not work, it clearly indicated the exception that an object was expected, not a list. I understand that this command would just read one object contained at the root level of the JSON file, but since we have a list of objects, this is an invalid request.

My next idea was to deserialize as a C # List of objects:

JsonSerializer serializer = new JsonSerializer();
List<MyObject> o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
    while (!sr.EndOfStream)
    {
        o = serializer.Deserialize<List<MyObject>>(reader);
    }
}

. . , , JSON , , # JSON . .

( [), sr.Read() , . , , " ". , , .

, , . }, , , , .

, . , - , . JSON, #.

?

Answer 1

. , , , , { , , .

JsonSerializer serializer = new JsonSerializer();
MyObject o;
using (FileStream s = File.Open("bigfile.json", FileMode.Open))
using (StreamReader sr = new StreamReader(s))
using (JsonReader reader = new JsonTextReader(sr))
{
    while (reader.Read())
    {
        // deserialize only when there "{" character in the stream
        if (reader.TokenType == JsonToken.StartObject)
        {
            o = serializer.Deserialize<MyObject>(reader);
        }
    }
}

Answer 2

nuget, , . JStreamAsyncNet

Answer 3

, , , - ... 14 3,5 . 1/4 JSON 0,5 . ( 4-7 ), ...

( ) JSON /

- Gason 2 #, , , 2 C++ (Debug/1/2 4 Release builds), 2 : https://github.com/eltomjan/Gason

Answer 4

Is this what you are looking for? Found on previous question

The current version of Json.net does not allow the use of the accepted response code. Current Alternative:

public static object DeserializeFromStream(Stream stream)
{
    var serializer = new JsonSerializer();

    using (var sr = new StreamReader(stream))
    using (var jsonTextReader = new JsonTextReader(sr))
    {
        return serializer.Deserialize(jsonTextReader);
    }
}

Documentation: Deserialize JSON from a file stream