.NET4 ExpandoObject memory leak usage

Question

I have a legacy .NET 4.0 application that runs as a Windows service. I'm not a .NET expert, but after writing code for 30 years, I know how to find my way.

When the service is first launched, the watch runs about 70 MB of private working set. The longer the service runs, the more memory is required. This is not so impressive that you notice, just sit and watch, but we have seen cases where the application has been running for a long time (more than 100 days) - up to several GB (5 GB - current recording). I connected the ANTS Memory Profiler to a running instance and found that using ExpandoObject seems to take into account several megabytes of strings that are not cleared by the GC. Most likely, there are other leaks, but this was most noticeable, so they attacked first.

I learned from other SO posts that the “normal” use of ExpandoObject throws an internal RuntimeBinderException when reading (but not writing) dynamically assigned attributes.

dynamic foo = new ExpandoObject(); var s; foo.NewProp = "bar"; // no exception s = foo.NewProp; // RuntimeBinderException, but handled by .NET, s now == "bar" 

You can see that the exception occurs in VisualStudio, but it is ultimately handled internally by .NET, and all you return is the value you need.

Except ... The line in the Message property of the property seems to remain on the heap and never gets Garbage Collected, even long after the ExpandoObject generated has gone out of scope.

A simple example:

 using System; using System.Dynamic; namespace ConsoleApplication2 { class Program { public static string foocall() { string str = "", str2 = "", str3 = ""; object bar = new ExpandoObject(); dynamic foo = bar; foo.SomePropName = "a test value"; // each of the following references to SomePropName causes a RuntimeBinderException - caught and handled by .NET // Attach an ANTS Memory profiler here and look at string instances Console.Write("step 1?"); var s2 = Console.ReadLine(); str = foo.SomePropName; // Take another snapshot here and you'll see an instance of the string: // 'System.Dynamic.ExpandoObject' does not contain a definition for 'SomePropName' Console.Write("step 2?"); s2 = Console.ReadLine(); str2 = foo.SomePropName; // Take another snapshot here and you'll see 2nd instance of the identical string Console.Write("step 3?"); s2 = Console.ReadLine(); str3 = foo.SomePropName; return str; } static void Main(string[] args) { var s = foocall(); Console.Write("Post call, pre-GC prompt?"); var s2 = Console.ReadLine(); // At this point, ANTS Memory Profiler shows 3 identical strings in memory // generated by the RuntimeBinderExceptions in foocall. Even though the variable // that caused them is no longer in scope the strings are still present. // Force a GC, just for S&G GC.Collect(); GC.WaitForPendingFinalizers(); GC.Collect(); Console.Write("Post GC prompt?"); s2 = Console.ReadLine(); // Look again in ANTS. Strings still there. Console.WriteLine("foocall=" + s); } } } 

"mistake" is in the eye of the contemplators, I suppose (my eyes speak of error). Am I missing something? Is this normal and expected for .NET masters in a group? Is there any way to talk about this? Is the best way to simply not use dynamic / ExpandoObject in the first place?

Answer 1

This is due to caching performed by the code generated by the compiler to access the dynamic property. (The analysis is performed with the outputs of VS2015 and .NET 4.6; other versions of the compiler may produce different output.)

Call str = foo.SomePropName; is rewritten by the compiler into something like this (according to dotPeek, note that <>o__0 , etc. are tokens that are not legal C #, but created by the C # compiler):

 if (Program.<>o__0.<>p__2 == null) { Program.<>o__0.<>p__2 = CallSite<Func<CallSite, object, string>>.Create(Binder.Convert(CSharpBinderFlags.None, typeof (string), typeof (Program))); } Func<CallSite, object, string> target1 = Program.<>o__0.<>p__2.Target; CallSite<Func<CallSite, object, string>> p2 = Program.<>o__0.<>p__2; if (Program.<>o__0.<>p__1 == null) { Program.<>o__0.<>p__1 = CallSite<Func<CallSite, object, object>>.Create(Binder.GetMember(CSharpBinderFlags.None, "SomePropName", typeof (Program), (IEnumerable<CSharpArgumentInfo>) new CSharpArgumentInfo[1] { CSharpArgumentInfo.Create(CSharpArgumentInfoFlags.None, (string) null) })); } object obj3 = Program.<>o__0.<>p__1.Target((CallSite) Program.<>o__0.<>p__1, obj1); string str1 = target1((CallSite) p2, obj3); 

Program.<>o__0.<>p__1 is a static field (for a private nested class) of type CallSite<Func<CallSite,object,object>> . It contains a link to a dynamic method that was compiled on demand the first time it accessed foo.SomePropName . (Presumably this is due to the fact that the creation of the binding is slow, so caching provides a significant increase in speed on subsequent accesses.)

This DynamicMethod contains a link to DynamicILGenerator , which refers to DynamicScope , which eventually contains a list of tokens. One of these tokens is the dynamically generated string 'System.Dynamic.ExpandoObject' does not contain a definition for 'SomePropName' . This line exists in memory, so that dynamically generated code can raise (and catch) a RuntimeBinderException using the "correct" message.

In general, the <>p__1 contains about 2K data (including 172 bytes for this string). There is no supported way to free this data, since it is rooted in a static field in the type generated by the compiler. (Of course, you could use reflection to set this static field to null , but it will be extremely dependent on the implementation details of the current compiler and will most likely be broken in the future.)

From what I have seen so far, it seems that using dynamic allocates about 2 Kbytes of memory to access properties in C # code; you probably should just consider this at the cost of using dynamic code. However (at least in this simplified example) this memory is allocated only when the code is first run, so it should not continue to use more memory the longer the program runs; another leak may occur that pushes the working set up to 5 GB. (There are three line instances because there are three separate lines of code that execute foo.SomePropName , however there will still be only three instances if you call foocall 100 times.)

To improve performance and reduce memory usage, you may need to use Dictionary<string, string> or Dictionary<string, object> as a simpler storage of keys / values ​​(if possible using a way to write code). Note that ExpandoObject implements IDictionary<string, object> , so the following small one overwrites the same output, but avoids the overhead of dynamic code:

 public static string foocall() { string str = "", str2 = "", str3 = ""; // use IDictionary instead of dynamic to access properties by name IDictionary<string, object> foo = new ExpandoObject(); foo["SomePropName"] = "a test value"; Console.Write("step 1?"); var s2 = Console.ReadLine(); // have to explicitly cast the result here instead of having the compiler do it for you (with dynamic) str = (string) foo["SomePropName"]; Console.Write("step 2?"); s2 = Console.ReadLine(); str2 = (string) foo["SomePropName"]; Console.Write("step 3?"); s2 = Console.ReadLine(); str3 = (string) foo["SomePropName"]; return str; }