What makes hot deployment a "difficult problem"?

Question

At work, we had a problem with the " PermGen out of memory " exception, and the team management decided that this was a bug in the JVM - something related to the hot deployment of the code. Without explaining a lot of details, he noted that hot deployment is a "difficult problem", so complex that even .NET does not do it yet.

I have found many articles explaining hot deployment from a bird's eye view, but always lacking technical details. Can someone provide me with a technical explanation and explain why hot deployment is a "difficult problem"?

Answer 1

When loading a class, various static class data is stored in PermGen. As long as there is a live link to this class instance, the class instance cannot be garbage collected.

I believe that part of the problem is whether the GC should remove old instances of the class from perm gen or not. As a rule, every time you deploy hotly, instances of a new class are added to the PermGen memory pool, and old ones that are no longer in use are usually not deleted. By default, Sun JVMs will not start garbage collection in PermGen, but this can be enabled with the optional java command arguments.

Therefore, if you quickly deploy enough time, you will eventually run out of PermGen space.

If your web application does not shut down completely when it is not deployed - if it, for example, leaves the thread that is running, then all instances of the class used by this web application will be pinned to Permgen. You are relocating and now have another whole copy of all these class instances loaded into PermGen. You refuse to deploy, and the thread continues to move, fixing ANOTHER set of class instances in PermGen. You relocate and download a whole set of copies ... and eventually your PermGen fills up.

You can sometimes fix this:

• Providing team arguments to the recent Sun JVM to include GC in PermGen and classes. That is: -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
• Using another JVM that does not use PermGen fixed size or what GC on loaded classes

But this will only help if your web application shuts down completely and cleanly, without leaving direct links to any of the class instances of any class loaded by the Class loaders for this web application.

Even this will not necessarily fix the problem due to leaks of the class loader. (In some cases, too many interned strings.)

Check out the following links for more information (two bold ones have good diagrams to illustrate part of the problem).

• Class loader leaks: terrible exception "java.lang.OutOfMemoryError: PermGen space"
• Unknown generation: Perm
• Continuous generation performance
• Tomcat Wiki: how to deal with memory errors

Answer 2

The problem in general is the Java security model, which is actually trying to prevent the reloading of an already loaded class.

Of course, Java from the very beginning supported dynamic loading of classes, which is difficult - reloading the class.

It was considered harmful (and for good reason) that a running Java application was introduced by a new class with malicious code. For example, java.lang.String hacked an implementation coming from the Internet, instead of creating a line, it deletes some random hile file that calls the length () method.

So, they somehow conceived Java (and I suppose the .NET CLR because it was very "inspired" in the JVM) was to prevent the already loaded class from loading again into the same virtual machine.

They proposed a mechanism to override this "function." Classloaders, but again there were rules for class loaders, they should ask the permission of the "parent" class loader before trying to load a new class, if the parent has already loaded the class, the new class is ignored.

For example, I used class loaders that load classes from LDAP or RDBMS

Hot deployment becomes a necessity in the Java world when the application server becomes the core for Java EE (and also creates the need for micro-containers such as spring to avoid this kind of burden).

Restarting the entire application server after each compiler causes anyone to be crazy.

So, application server provider, offer this “custom” class loader to help with a hot deployment and using the configuration file, this behavior MUST be disabled during installation during production. But the trade-off is that you have to use tons of memory during the development process. Thus, a good way to do this is to restart every 3 to 4 deployments.

This does not happen with other languages ​​that were designed from the very beginning to load their classes.

In Ruby, for example, you can even add methods to a running class, override a method at runtime, or even add a single method to a unique, specific object.

The trade-off in such environments is, of course, memory and speed.

Hope this helps.

EDIT

I found this product some time ago that promises that rebooting makes it as simple as possible. I do not remember the link when I first wrote this answer, and I do it.

This is JavaRebel from ZeroTurnaround

Answer 3

The Sun JVM has a fixed PermGen space, and ultimately it is all consumed (yes, apparently due to a bug in the code related to loading classes) => OOM.

If you can use the JVM of another provider (e.g. Weblogic one), it dynamically expands the PermGen space, so you will never get a migration-bound OOM.

Answer 4

What version of java are you using? There were bugs in early Sun 1.4.2, but it worked for a long time.
By the way, how can you share the news with the team? Do you lead a team?