Is it possible to create a memory leak in Tomcat?

Question

I configured tomcat to work with other external open source.

However, after tomcat has been working for several minutes, I get:

SEVERE: the web application [/ MyProject] created ThreadLocal with a key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b3f02f]) and a value of type [org.apache.axis.MessageContext] (value [org. apache.axis.MessageContext@5dbd4e]), but could not delete it when the web application was stopped. This is likely to create a memory leak.

What can cause this?

Where do i search? Could this be datapooling on Tomcat?

And what does "Tomcat Themes" mean?

EDITED

Here is my complete footprint. The application seems to reload its context while it is running - and I don't know why!

Mar 13, 2011 10:56:12 PM org.apache.catalina.core.StandardContext reload INFO: Reloading this Context has started Mar 13, 2011 10:56:12 PM org.apache.catalina.core.StandardWrapper unload INFO: Waiting for 1 instance(s) to be deallocated Mar 13, 2011 10:56:13 PM org.apache.catalina.core.StandardWrapper unload INFO: Waiting for 1 instance(s) to be deallocated Mar 13, 2011 10:56:14 PM org.apache.catalina.core.StandardWrapper unload INFO: Waiting for 1 instance(s) to be deallocated Mar 13, 2011 10:56:14 PM org.apache.catalina.core.ApplicationContext log INFO: Closing Spring root WebApplicationContext Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc SEVERE: The web application [/MyProject] registered the JBDC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc SEVERE: The web application [/MyProject] registered the JBDC driver [oracle.jdbc.driver.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [NioSocketAcceptor-1] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-1] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-4] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [bitronix-disk-force-batcher] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [bitronix-scheduler] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] is still processing a request that has yet to finish. This is very likely to create a memory leak. You can control the time allowed for requests to finish by using the unloadDelay attribute of the standard Context implementation. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-7] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/MyProject] appears to have started a thread named [NioProcessor-2] but has failed to stop it. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b5a8e1]) and a value of type [org.mvel2.debug.DebuggerContext] (value [org.mvel2.debug.DebuggerContext@16259fd]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.apache.axis.utils.XMLUtils.ThreadLocalDocumentBuilder] (value [org.apache.axis.utils.XMLUtils$ThreadLocalDocumentBuilder@84b0b4]) and a value of type [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl] (value [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl@16d2cfa]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [null] (value [com.sun.faces.util.Util$1@16bbac9]) and a value of type [java.util.HashMap] (value [{com.sun.faces.patternCache={ = }}]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [java.lang.ThreadLocal] (value [java.lang.ThreadLocal@1b3f02f]) and a value of type [org.apache.axis.MessageContext] (value [org.apache.axis.MessageContext@5dbd4e]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.apache.axis.utils.XMLUtils.ThreadLocalDocumentBuilder] (value [org.apache.axis.utils.XMLUtils$ThreadLocalDocumentBuilder@84b0b4]) and a value of type [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl] (value [com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl@378584]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [org.springframework.core.NamedThreadLocal] (value [Transactional resources]) and a value of type [java.util.HashMap] (value [{org.hibernate.impl.SessionFactoryImpl@ccc27b=org.springframework.orm.hibernate3.SessionHolder@4f6ada}]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. Mar 13, 2011 10:56:15 PM org.apache.catalina.loader.WebappClassLoader clearThreadLocalMap SEVERE: The web application [/MyProject] created a ThreadLocal with key of type [null] (value [com.sun.faces.application.ApplicationAssociate$1@1f01fcf]) and a value of type [com.sun.faces.application.ApplicationAssociate] (value [com.sun.faces.application.ApplicationAssociate@1b85528]) but failed to remove it when the web application was stopped. This is very likely to create a memory leak. 2011-03-13 22:57:27,734 ERROR ( ContextLoader.java:220) - Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'transactionManager' defined in class path resource [applicationContext-hibernate.xml]: Cannot resolve reference to bean 'sessionFactory' while setting bean property 'sessionFactory'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'sessionFactory' defined in class path resource [applicationContext-hibernate.xml]: Invocation of init method failed; nested exception is java.lang.OutOfMemoryError: Java heap space at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:328) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:106) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1325) 

Answer 1

The message is actually pretty clear: something creates a ThreadLocal with a value of type org.apache.axis.MessageContext - this is a great hint. Most likely, this means that the Apache Axis infrastructure has forgotten / failed to clear after itself. The same problem arose, for example, in a magazine. You shouldn't worry too much, but an error message for the Axis team might be a good idea.

Tomcat reports this error because ThreadLocal is created for HTTP worker streams. Your application is not deployed, but HTTP streams remain - and these ThreadLocal . This can lead to a memory leak ( org.apache.axis.MessageContext cannot be unloaded) and some problems when these threads will be reused in the future.

For more details see: http://wiki.apache.org/tomcat/MemoryLeakProtection

Answer 2

I added the following @PreDestroy method to my CDI @ApplicationScoped bean, and when I complete TomEE 1.6.0 (tomcat7.0.39, today), it clears the stream locators.

 /* * To change this template, choose Tools | Templates * and open the template in the editor. */ package pf; import java.lang.ref.WeakReference; import java.lang.reflect.Array; import java.lang.reflect.Field; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * * @author Administrator * * google-gson issue # 402: Memory Leak in web application; comment # 25 * https://code.google.com/p/google-gson/issues/detail?id=402 */ public class ThreadLocalImmolater { final Logger logger = LoggerFactory.getLogger(ThreadLocalImmolater.class); Boolean debug; public ThreadLocalImmolater() { debug = true; } public Integer immolate() { int count = 0; try { final Field threadLocalsField = Thread.class.getDeclaredField("threadLocals"); threadLocalsField.setAccessible(true); final Field inheritableThreadLocalsField = Thread.class.getDeclaredField("inheritableThreadLocals"); inheritableThreadLocalsField.setAccessible(true); for (final Thread thread : Thread.getAllStackTraces().keySet()) { count += clear(threadLocalsField.get(thread)); count += clear(inheritableThreadLocalsField.get(thread)); } logger.info("immolated " + count + " values in ThreadLocals"); } catch (Exception e) { throw new Error("ThreadLocalImmolater.immolate()", e); } return count; } private int clear(final Object threadLocalMap) throws Exception { if (threadLocalMap == null) return 0; int count = 0; final Field tableField = threadLocalMap.getClass().getDeclaredField("table"); tableField.setAccessible(true); final Object table = tableField.get(threadLocalMap); for (int i = 0, length = Array.getLength(table); i < length; ++i) { final Object entry = Array.get(table, i); if (entry != null) { final Object threadLocal = ((WeakReference)entry).get(); if (threadLocal != null) { log(i, threadLocal); Array.set(table, i, null); ++count; } } } return count; } private void log(int i, final Object threadLocal) { if (!debug) { return; } if (threadLocal.getClass() != null && threadLocal.getClass().getEnclosingClass() != null && threadLocal.getClass().getEnclosingClass().getName() != null) { logger.info("threadLocalMap(" + i + "): " + threadLocal.getClass().getEnclosingClass().getName()); } else if (threadLocal.getClass() != null && threadLocal.getClass().getName() != null) { logger.info("threadLocalMap(" + i + "): " + threadLocal.getClass().getName()); } else { logger.info("threadLocalMap(" + i + "): cannot identify threadlocal class name"); } } } 

Answer 3

The Transactional Resources key looks like you are talking to a database without a proper transaction. Make sure that transaction management is configured correctly and that there is no way to access a DAO that does not start under the @Transactional annotation. This can happen when you set up transaction management at the controller level, but call the DAO in a timer or use @PostConstruct annotations. I wrote it here http://georgovassilis.blogspot.nl/2014/01/tomcat-spring-and-memory-leaks-when.html

Edit: looks like this is (also?) A bug with spring -data-jpa fixed with v1.4.3. I looked at it in the sources spring -data-jpa LockModeRepositoryPostProcessor, which sets the Transactional Resources key. In 1.4.3, it clears the key again.

Answer 4

Sometimes this is due to configuration changes. When we upgraded from Tomncat 6.0.14 to 6.0.26, we saw something similar. here is the solution http://www.skill-guru.com/blog/2010/08/22/tomcat-6-0-26-shutdown-reports-a-web-application-created-a-threadlocal-threadlocal-has- been-forcibly-removed /

Answer 5

This problem arises when we use any third solution, without using handlers to activate the cleanup. For me, this was happening for EhCache. We used EhCache in our project for caching. And often we saw the following error in the logs

  SEVERE: The web application [/products] appears to have started a thread named [products_default_cache_configuration] but has failed to stop it. This is very likely to create a memory leak. Aug 07, 2017 11:08:36 AM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads SEVERE: The web application [/products] appears to have started a thread named [Statistics Thread-products_default_cache_configuration-1] but has failed to stop it. This is very likely to create a memory leak. 

And we often notice that tomcat was unable to execute the OutOfMemory error during development, where we made backend changes and deployed the application several times to reflect our changes.

This is the correction we made

 <listener> <listener-class> net.sf.ehcache.constructs.web.ShutdownListener </listener-class> </listener> 

So what I'm trying to do is check the documentation of the third-party libraries that you use. They should provide some mechanisms for cleaning threads during shutdown. What you need to use in your application. No need to reinvent the wheel if it is not provided by them. The worst case scenario is to provide you with your own implementation.

Link to complete EHCache http://www.ehcache.org/documentation/2.8/operations/shutdown.html