How to partially serialize objects by providing "paths" using Newtonsoft JSON.NET

Question

I have a situation where I have a very large C # object, however I only need to return a few properties (which may be on nested objects), allow the JavaScript client to change these properties, and then send the resulting object back to the server to do partial de-serialization in place.

The idea is to reuse some very large existing business objects, but be smart only to serialize and send only those properties back to the client application for modification (to save the amount of data transferred at least).

I basically have an XML file in which I pre-define all the bindings using a path syntax that only specifies the properties that I need to serialize. So I could use something like "WorkOrder.UserField1" or "WorkOrder.Client.Name".

I tried using a custom contract resolver to determine if a property should be serialized; however, it seems that I do not have information about the “path” (in other words, about other properties of the object model up the chain) to determine whether this property should or should not be serialized.

I also tried using a custom JsonTextWriter, but it doesn't seem like I can override the methods needed to track the path, even though the Path property is present. Is there something, possibly simple, that I am losing sight of in order to be able to view the hierarchy of paths of a serializable property and determine whether it should be serialized by finding the path in the table and making a decision?

Answer 1

The main difficulty here is that Json.NET is a contract based serializer that creates a contract for each serializable type and then (de) serializes according to the contract. If the type is displayed in several places in the object hierarchy, the same contract applies. But you want to selectively include properties for this type depending on its location in the hierarchy, which contradicts the basic design of "one type one contract".

One quick way around this is to serialize in JObject and then use JToken.SelectTokens() to select only the JSON data that you want to return, deleting everything else. Because SelectTokens fully supports JSONPath query syntax , you can selectively enable the use of array wildcards and properties or other filters, for example:

 "$.FirstLevel[*].Bar" 

includes all properties with the name "Bar" in all elements of the property array named "FirstLevel" root object.

This should reduce the use of your network at will, but will not save processing time on the server.

Removal can be performed using the following extension methods:

 public static partial class JsonExtensions { public static TJToken RemoveAllExcept<TJToken>(this TJToken obj, IEnumerable<string> paths) where TJToken : JToken { if (obj == null || paths == null) throw new NullReferenceException(); var keepers = new HashSet<JToken>(paths.SelectMany(path => obj.SelectTokens(path)), ObjectReferenceEqualityComparer<JToken>.Default); var keepersAndParents = new HashSet<JToken>(keepers.SelectMany(t => t.AncestorsAndSelf()), ObjectReferenceEqualityComparer<JToken>.Default); // Keep any token that is a keeper, or a child of a keeper, or a parent of a keeper // Ie if you have a path ""$.AB" and it turns out that B is an object, then everything // under B should be kept. foreach (var token in obj.DescendantsAndSelfReversed().Where(t => !keepersAndParents.Contains(t) && !t.AncestorsAndSelf().Any(p => keepers.Contains(p)))) token.RemoveFromLowestPossibleParent(); // Return the object itself for fluent style programming. return obj; } public static string SerializeAndSelectTokens<T>(T root, string[] paths, Formatting formatting = Formatting.None, JsonSerializerSettings settings = null) { var obj = JObject.FromObject(root, JsonSerializer.CreateDefault(settings)); obj.RemoveAllExcept(paths); var json = obj.ToString(formatting); return json; } public static void RemoveFromLowestPossibleParent(this JToken node) { if (node == null) throw new ArgumentNullException(); var contained = node.AncestorsAndSelf().Where(t => t.Parent is JArray || t.Parent is JObject).FirstOrDefault(); if (contained != null) contained.Remove(); } public static IEnumerable<JToken> DescendantsAndSelfReversed(this JToken node) { if (node == null) throw new ArgumentNullException(); return RecursiveEnumerableExtensions.Traverse(node, t => ListReversed(t as JContainer)); } // Iterate backwards through a list without throwing an exception if the list is modified. static IEnumerable<T> ListReversed<T>(this IList<T> list) { if (list == null) yield break; for (int i = list.Count - 1; i >= 0; i--) yield return list[i]; } } public static partial class RecursiveEnumerableExtensions { // Rewritten from the answer by Eric Lippert https://stackoverflow.com/users/88656/eric-lippert // to "Efficient graph traversal with LINQ - eliminating recursion" http://stackoverflow.com/questions/10253161/efficient-graph-traversal-with-linq-eliminating-recursion // to ensure items are returned in the order they are encountered. public static IEnumerable<T> Traverse<T>( T root, Func<T, IEnumerable<T>> children) { yield return root; var stack = new Stack<IEnumerator<T>>(); try { stack.Push((children(root) ?? Enumerable.Empty<T>()).GetEnumerator()); while (stack.Count != 0) { var enumerator = stack.Peek(); if (!enumerator.MoveNext()) { stack.Pop(); enumerator.Dispose(); } else { yield return enumerator.Current; stack.Push((children(enumerator.Current) ?? Enumerable.Empty<T>()).GetEnumerator()); } } } finally { foreach (var enumerator in stack) enumerator.Dispose(); } } } /// <summary> /// A generic object comparerer that would only use object reference, /// ignoring any <see cref="IEquatable{T}"/> or <see cref="object.Equals(object)"/> overrides. /// </summary> public class ObjectReferenceEqualityComparer<T> : IEqualityComparer<T> where T : class { // Adapted from this answer https://stackoverflow.com/a/1890230 // to https://stackoverflow.com/questions/1890058/iequalitycomparert-that-uses-referenceequals // By https://stackoverflow.com/users/177275/yurik private static readonly IEqualityComparer<T> _defaultComparer; static ObjectReferenceEqualityComparer() { _defaultComparer = new ObjectReferenceEqualityComparer<T>(); } public static IEqualityComparer<T> Default { get { return _defaultComparer; } } #region IEqualityComparer<T> Members public bool Equals(T x, T y) { return ReferenceEquals(x, y); } public int GetHashCode(T obj) { return System.Runtime.CompilerServices.RuntimeHelpers.GetHashCode(obj); } #endregion } 

And then use them like:

 public class TestClass { public static void Test() { var root = new RootObject { FirstLevel1 = new FirstLevel { SecondLevel1 = new List<SecondLevel> { new SecondLevel { A = "a11", B = "b11", Third1 = new ThirdLevel { Foo = "Foos11", Bar = "Bars11" }, Third2 = new List<ThirdLevel> { new ThirdLevel { Foo = "FooList11", Bar = "BarList11" } } } }, SecondLevel2 = new List<SecondLevel> { new SecondLevel { A = "a12", B = "b12", Third1 = new ThirdLevel { Foo = "Foos12", Bar = "Bars12" }, Third2 = new List<ThirdLevel> { new ThirdLevel { Foo = "FooList12", Bar = "BarList12" } } } }, }, FirstLevel2 = new FirstLevel { SecondLevel1 = new List<SecondLevel> { new SecondLevel { A = "a21", B = "b21", Third1 = new ThirdLevel { Foo = "Foos21", Bar = "Bars21" }, Third2 = new List<ThirdLevel> { new ThirdLevel { Foo = "FooList21", Bar = "BarList21" } } } }, SecondLevel2 = new List<SecondLevel> { new SecondLevel { A = "a22", B = "b22", Third1 = new ThirdLevel { Foo = "Foos22", Bar = "Bars22" }, Third2 = new List<ThirdLevel> { new ThirdLevel { Foo = "FooList22", Bar = "BarList22" } } } }, } }; Assert.IsTrue(JObject.FromObject(root).DescendantsAndSelf().OfType<JValue>().Count() == 24); // No assert var paths1 = new string[] { "$.FirstLevel2.SecondLevel1[*].A", "$.FirstLevel1.SecondLevel2[*].Third2[*].Bar", }; Test(root, paths1, 2); var paths3 = new string[] { "$.FirstLevel1.SecondLevel2[*].Third2[*].Bar", }; Test(root, paths3, 1); var paths4 = new string[] { "$.*.SecondLevel2[*].Third2[*].Bar", }; Test(root, paths4, 2); } static void Test<T>(T root, string [] paths, int expectedCount) { var json = JsonExtensions.SerializeAndSelectTokens(root, paths, Formatting.Indented); Console.WriteLine("Result using paths: {0}", JsonConvert.SerializeObject(paths)); Console.WriteLine(json); Assert.IsTrue(JObject.Parse(json).DescendantsAndSelf().OfType<JValue>().Count() == expectedCount); // No assert } } public class ThirdLevel { public string Foo { get; set; } public string Bar { get; set; } } public class SecondLevel { public ThirdLevel Third1 { get; set; } public List<ThirdLevel> Third2 { get; set; } public string A { get; set; } public string B { get; set; } } public class FirstLevel { public List<SecondLevel> SecondLevel1 { get; set; } public List<SecondLevel> SecondLevel2 { get; set; } } public class RootObject { public FirstLevel FirstLevel1 { get; set; } public FirstLevel FirstLevel2 { get; set; } } 

Please note that there is a request to expand Feature Feature: ADD JsonProperty.ShouldSerialize (object target, line path) # 1857, which would simplify such functionality.

Demo violin here and here .