I am creating a json deserializer. I am deserializing a rather large json file (25mb) that contains a lot of information. This is an array for words with lots of duplicates. With help, NewtonSoft.JsonI can deserialize the input as a stream:
using (var fs = new FileStream(@"myfile.json", FileMode.Open, FileAccess.Read))
using (var sr = new StreamReader(fs))
using (var reader = new JsonTextReader(sr))
{
while (reader.Read())
{
var obj = JObject.Load(reader);
}
}
This allows me to read small parts of json and parse it, check for duplicates, etc.
If I want to do the same with ServiceStack.Text. I am doing something like:
using (var fs = new FileStream(@"myfile.json", FileMode.Open, FileAccess.Read))
using (var sr = new StreamReader(fs))
{
var result = ServiceStack.Text.JsonSerializer.DeserializeFromReader<MyObject>(sr);
}
MyObject only contains the subset of json that interests me, but this creates significant overhead since I will get a large array containing many duplicates.
In the first method, I can filter them out immediately and therefore not store them in memory.
A trace of memory between the two (this includes the console utilities):
- NewtonSoft: 30mb
- ServiceStack.Text: 215mb
:
- NewtonSoft: 2.5s
- ServiceStack.Text: 1.5s
, .
, ServiceStack TypeSafety, .
, ServiceStack.Text , , NewtonSoft, ServiceStack.Text?
( , ):
public class MyObject
{
public List<List<Word>> Words { get; set; }
}
public class Word
{
public string B { get; set; }
public string W { get; set; }
public string E { get; set; }
public string P { get; set; }
}
( ) 29000 , 8500 . , . , .