How to resolve OutOfMemoryError with ImageIO plugins as a reason?

Question

At work, we have several tomcat servers running several webapps, about half of which must perform some image processing.

Before performing image processing, these web applications execute ImageIO.scanForPlugins() to retrieve the corresponding image readers and records in memory. Although before it was just launched at any time, the image should be processed, now we start scanning only when we initialize webapps (since after launch we do not add jars, why start scanning more than once?)

A few days later, the tomcat instance crashed due to an OutOfMemoryError . Fortunately, we had the HeapDumpOnOutOfMemoryError option, so I looked at a bunch of heaps. In the dump, I found that 97% of the memory was taken by an instance of javax.imageio.spi.PartialOrderIterator . Most of this space was occupied by him, supporting a java.util.LinkedList , which had 18 million elements. The linked list consists of javax.imageio.spi.DigraphNode , which contains image readers and records loaded by ImageIO.scanForPlugins() .

"Yeah," I thought, "we have to run the scan in a loop somewhere, and we just add the same elements over and over." But, I decided that I should double check this assumption, so I wrote the following test class:

 import javax.imageio.ImageIO; public class ImageIOTesting { public static void main(String[] args) { for (int i = 0; i < 100000; i++) { ImageIO.scanForPlugins(); if (i % 1000 == 0) { System.out.println(Runtime.getRuntime().totalMemory() / 1024); } } } } 

However, when I run this class in a server environment, the amount of memory used never changes!

A quick search of the package source javax.imageio shows that the check checks if the service provider is already registered and if it unregisters the old provider before registering the new one. So now the question is: why do I have this giant list of service providers? Why are they stored as a directed graph? And more importantly, how can I prevent this?

Answer 1

Late answer to the old question, but anyway:

Because the ImageIO plug-in ImageIO ( IIORegistry ) is "global global", it does not work with servlet contexts by default. This is especially noticeable if you load plugins from the WEB-INF/lib or classes folder, as the OP does.

Servlet contexts dynamically load and unload classes (using the new class loader for each context). If you restart the application, the old classes will by default remain in memory forever (because the next time scanForPlugins is called, and another ClassLoader , which scans / loads the classes, and therefore they will be new instances in the registry. The test code cycle is the same ClassLoader used all the time, so instances are replaced, and the real problem never shows up).

To get around this resource leak, I recommend using ContextListener to make sure that these “contextual local” plugins are explicitly removed. Here is an example of the IIOProviderContextListener I used in several projects that implement the dynamic loading and unloading of ImageIO plugins.

Answer 2

Looks like an ugly memory leak, I can't figure out where it is without looking at all the code, but I know that you should check very carefully.

Possible reasons:

_Global Variable List, adding items, never deleting them.

_Infinite / big loops, loading many elements into lists, never deleting them.

It will also help if you use a Java profiler like VisualVM

If you provide more information in order to better understand how this works, I can improve my answer, I can not do anything without additional information =)

Hope this helps!