All geek questions in one place

Why does the New Relic eat a lot of cat memory?

We recently started using New Relic to monitor our web server, which was hosted on the tomcat 7.0.6 server, but we noticed that the cat’s memory is constantly increasing and within a week it eats up the entire server (AWS High-Memory Double Extra Large Instance) and become immune, the only way to get it back is to restart it. We provide Xms and Xmx arguments when starting tomcat, but for several hours using the memory using the Xmx value of the tomcat processing process, and it continues to increase until all server memory runs out. Here is the process command:

/usr/java/jdk1.6.0_24//bin/java -Djava.util.logging.config.file=/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/conf/logging.properties -Xms8192m -Xmx8192m -javaagent:/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/newrelic/newrelic.jar -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Duser.timezone=Asia/Calcutta -Djava.endorsed.dirs=/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/endorsed -classpath /xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/bin/bootstrap.jar:/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/bin/tomcat-juli.jar -Dcatalina.base=/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6 -Dcatalina.home=/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6 -Djava.io.tmpdir=/xxx/xxx/xxx/xxx/apache-tomcat-7.0.6/temp org.apache.catalina.startup.Bootstrap start"

Ideally, I would expect this process to not use more than 8 GB of memory, but in a few hours it will exceed 10 GB and in a few days it will exceed 20 GB, and everything else on this server will suffer because of this ( I use "top" to view memory usage). How is this possible?

+6
source share
3 answers

There is a problem that affects any Sun / Oracle JVM and manifests itself as unlimited growth in non-heap (native) memory. There is a workaround for the new Relic Java agent version 2.16+, adding a shutdown delay to converting the class to the newrelic.yml file in the general section.

  class_transformer: shutdown_delay: 3600

From change log

Workaround for Oracle JVM error, which in rare cases causes memory leak

In rare cases, the Oracle JVM can leak native OS memory (not a lot of space) when agents intercept classes. This setting rotates intercepting classes that load after a given number of seconds. The agent will continue to monitor classes loaded before this time.

+4
source

I give more detailed information about the above incident. memory leak is not on the Java heap. An application never reaches any OUT OF MEMORY error (8 gb is the limit of the maximum Java heap we set). However, virtual and resident memory continue to grow until RAM runs out of memory. We have confirmed that this leak occurs when a relic agent is used. Version: New Relic Agent v2.1.2

+1
source

Excuse for troubling. We (The New Relic) are investigating the problem, but the first suggestion is to try the latest version of Java Agent 2.2.1, which has made significant changes to the way we deal with tools.

I will follow us when we have additional information.

+1
source

Source: https://habr.com/ru/post/904986/

More articles:


All Articles