ASP.NET Web Application Does Not Unload AppDomains After Deployment

Question

When we deploy our web application, we copy all the code into a new directory and then point iis to this new directory. When we do this, the number of updates increases, but never decreases. In addition, the Application_End event never fires.

For some unknown time, while both sets of AppDomains are still being reported by perfmon, the system works very poorly, and% of the time in GC buses is up to 100%. In the end, I recycle the application pool so that the application runs smoothly again.

Additional information: the list of applications shows 2 on my dev machine, but 4 runs on a real server ... we just have one application running in the pool, so this means that some kind of library that we use is creating application domains.

What should I do to try to debug what is happening? What can prevent the unloading of the application domain?

Update 9/3/2014

After receiving more detailed information about the log, it seems that the problem is not that the old application domains remain, the new applications are saved during the reboot. Instead of an application starting one new instance, it starts with two. Sometimes we get application_end from the old instance, sometimes we do not.

Update 9/4/2014

Both things are happening. Using the process explorer, I see on one of the machines that the old application domain still exists, and the new one. There were only 2 application domains on the other machine, but there was a space in their sequential identifiers. So, two instances began (we also received a log message from the start application), one of them died almost instantly, leaving 2 application domains.

Answer 1

Although I'm not sure if this is the only reason the app does not unload appdomains, this is certainly one of the reasons. The actual answer is much less interesting than the steps I used to determine it.

• I wrote a small ruby ​​script to just make 100 requests in 10 threads against our application (setting up jmeter will take longer than that).
• Run the script and make changes to the application configuration file many times during its launch. It took a couple of minutes.
• I used a process handler to confirm that 4 application domains were still loaded in this process.
• I ran procdump and created a dump file.
• I uploaded the procdump file to the visual studio and clicked "Debug with managed only" (although I had no idea if there was an error in the managed code or not)
• The debugger stopped at a line of code where the thread that called Application_End was waiting for the queue to complete. By looking at the values ​​of the variables, I was able to say that the queue no longer processes the elements, but we will wait until the queue is empty.
• Changed the code again and started the process, this time all the application domains started by my changes in the web configuration were unloaded.

Answer 2

One thing you can check to help troubleshoot:

Process Explorer has a .NET Assemblies tab that lists all the AppDomains loaded by the process. NOTE. The tab is displayed only for processes using the .NET Framework.