Top -H shows several stream identifiers that are not displayed in the jstack file

I am trying to find out why my jboss occupies 600% of the CPU (works on CentOS 5).

I run top -H and try to compare the result with jstack output. Most thread identifiers that appear on top are displayed on jstack's output, but unfortunately the threads that occupy most of the processor are not listed there.

Since the conclusion is very long, I list it only here.

 $top -b -n1 -H -p <jboss pid> top - 08:04:58 up 73 days, 19:50, 1 user, load average: 5.72, 5.97, 5.95 Tasks: 128 total, 6 running, 122 sleeping, 0 stopped, 0 zombie Cpu(s): 63.6%us, 1.7%sy, 0.0%ni, 34.2%id, 0.3%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 32946160k total, 32545228k used, 400932k free, 86776k buffers Swap: 34996216k total, 136k used, 34996080k free, 21577176k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12908 jboss 16 0 9488m 8.6g 18m S 71.5 27.4 146:45.68 java 12903 jboss 16 0 9488m 8.6g 18m S 69.5 27.4 146:36.08 java 12904 jboss 16 0 9488m 8.6g 18m S 69.5 27.4 146:39.63 java 12905 jboss 16 0 9488m 8.6g 18m S 69.5 27.4 146:44.46 java 12906 jboss 16 0 9488m 8.6g 18m S 69.5 27.4 146:53.94 java 12907 jboss 16 0 9488m 8.6g 18m S 69.5 27.4 146:39.18 java 12914 jboss 16 0 9488m 8.6g 18m R 27.8 27.4 36:26.64 java 12901 jboss 25 0 9488m 8.6g 18m S 0.0 27.4 0:00.00 java 12902 jboss 25 0 9488m 8.6g 18m S 0.0 27.4 0:00.05 java 12909 jboss 23 0 9488m 8.6g 18m S 0.0 27.4 76:34.09 java 12910 jboss 23 0 9488m 8.6g 18m S 0.0 27.4 76:33.49 java 12911 jboss 15 0 9488m 8.6g 18m S 0.0 27.4 257:10.35 java 12915 jboss 15 0 9488m 8.6g 18m S 0.0 27.4 0:06.63 java 12916 jboss 15 0 9488m 8.6g 18m S 0.0 27.4 0:19.61 java more lines.... 

The jstack bend is as follows

 # /usr/java/jdk1.6.0_33/bin/jstack -F 3362 Deadlock Detection: No deadlocks found. Thread 13055: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=415 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11, line=441 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame) Thread 12996: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=415 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11, line=441 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame) Thread 12994: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=485 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.await() @bci=8, line=415 (Interpreted frame) - org.apache.tomcat.util.net.JIoEndpoint$Worker.run() @bci=11, line=441 (Interpreted frame) - java.lang.Thread.run() @bci=11, line=662 (Interpreted frame) 

more stacks ....

Thread IDs on jstack output jstack also displayed on top output, but thread IDs on top, which work most intensively with the CPU (1903-1908), are not displayed on jstack output.

I thought it could be GC streams, but I cannot find anything to prove it.

Any idea why I can't see all the threads in jstack or what can I do to see the missing threads?

+4
source share
1 answer

Try running jstack under the owner of jboss. This way you will get more threads as output. Giving out

 jstack -l 12345 

I got

  "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x00007f976c0d3000 nid=0x1f56a runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE Locked ownable synchronizers: - None 

Note that nid is the hexadecimal representation of the stream identifier received from the vertex -H

+1
source

Source: https://habr.com/ru/post/1483870/


All Articles