Streaming a large list of data in JSON format using Json.net

Question

Using the MVC model, I would like to write a JsonResult that will pass the Json string to the client, and not convert all the data to a Json string right away, and then pass it back to the client. I have actions that require sending very large (over 300,000 entries) as Json transfers, and I think the main implementation of JsonResult is not scalable.

I am using Json.net, I am wondering if there is a way to pass pieces of a Json string when it is being converted.

//Current implementation: response.Write(Newtonsoft.Json.JsonConvert.SerializeObject(Data, formatting)); response.End(); //I know I can use the JsonSerializer instead Newtonsoft.Json.JsonSerializer serializer = new Newtonsoft.Json.JsonSerializer(); serializer.Serialize(textWriter, Data); 

However, I'm not sure how I can get the pieces written in textWriter and write back and answer the call to .Flush () until all 300,000 entries are converted to Json.

Is this even possible?

Answer 1

Assuming your end result is a JSON array, and each “piece” is one element in this array, you can try something like the following JsonStreamingResult class. It uses a JsonTextWriter to write JSON to the output stream and uses JObject as a means of serializing each element separately before writing it to the record. You can pass an implementation of JsonStreamingResult a IEnumerable , which can read elements separately from your data source so that you do not immediately have them all in memory. I have not tested this extensively, but it should make you go in the right direction.

 public class JsonStreamingResult : ActionResult { private IEnumerable itemsToSerialize; public JsonStreamingResult(IEnumerable itemsToSerialize) { this.itemsToSerialize = itemsToSerialize; } public override void ExecuteResult(ControllerContext context) { var response = context.HttpContext.Response; response.ContentType = "application/json"; response.ContentEncoding = Encoding.UTF8; JsonSerializer serializer = new JsonSerializer(); using (StreamWriter sw = new StreamWriter(response.OutputStream)) using (JsonTextWriter writer = new JsonTextWriter(sw)) { writer.WriteStartArray(); foreach (object item in itemsToSerialize) { JObject obj = JObject.FromObject(item, serializer); obj.WriteTo(writer); writer.Flush(); } writer.WriteEndArray(); } } }