JVM Fatal Error Guessing Line Number

Question

I have a fatal error log without a core dump and you need to determine the cause. This is the stack found in the .log file:

# Problematic frame: # C [libc.so.6+0x7b4bb] memcpy+0x15b 

{...}

 Stack: [0x00002ac8c4d2c000,0x00002ac8c4e2d000], sp=0x00002ac8c4e28ef8, free space=1011k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) C [libc.so.6+0x7b4bb] memcpy+0x15b C [libzip.so+0x50b0] ZIP_GetEntry+0xd0 C [libzip.so+0x3eed] Java_java_util_zip_ZipFile_getEntry+0xad J 28 java.util.zip.ZipFile.getEntry(J[BZ)J (0 bytes) @ 0x00002ac8acf404ee [0x00002ac8acf40420+0xce] Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) J 28 java.util.zip.ZipFile.getEntry(J[BZ)J (0 bytes) @ 0x00002ac8acf40478 [0x00002ac8acf40420+0x58] J 33 C2 java.util.zip.ZipFile.getEntry(Ljava/lang/String;)Ljava/util/zip/ZipEntry; (86 bytes) @ 0x00002ac8acf45548 [0x00002ac8acf45480+0xc8] J 58 C2 java.util.jar.JarFile.getJarEntry(Ljava/lang/String;)Ljava/util/jar/JarEntry; (9 bytes) @ 0x00002ac8acf4e828 [0x00002ac8acf4e7e0+0x48] J 44 C2 sun.misc.URLClassPath$JarLoader.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; (91 bytes) @ 0x00002ac8acf47168 [0x00002ac8acf47100+0x68] J 34 C2 sun.misc.URLClassPath.getResource(Ljava/lang/String;Z)Lsun/misc/Resource; (74 bytes) @ 0x00002ac8acf41f70 [0x00002ac8acf41f00+0x70] j java.net.URLClassLoader$1.run()Ljava/lang/Class;+26 j java.net.URLClassLoader$1.run()Ljava/lang/Object;+1 v ~StubRoutines::call_stub j java.security.AccessController.doPrivileged(Ljava/security/PrivilegedExceptionAction;Ljava/security/AccessControlContext;)Ljava/lang/Object;+0 j java.net.URLClassLoader.findClass(Ljava/lang/String;)Ljava/lang/Class;+13 j java.lang.ClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+70 j sun.misc.Launcher$AppClassLoader.loadClass(Ljava/lang/String;Z)Ljava/lang/Class;+36 j java.lang.ClassLoader.loadClass(Ljava/lang/String;)Ljava/lang/Class;+3 v ~StubRoutines::call_stub j com.smackdapp.SmackHandler.handleRequest(Ljava/lang/String;Lorg/eclipse/jetty/server/Request;Ljavax/servlet/http/HttpServletRequest;Ljavax/servlet/http/HttpServletResponse;)V+72 

As I read in other StackOverflow answers, this can be a common mistake when replacing any .jar that should load jvm (or a hardware memory error).

What I'm trying to guess is the .jar file that caused this fatal error. Is there a way to identify the line number in my source code with information in the fatal error log?

I was hoping that this “V + 72” at the end of the line should do something, but cannot understand.

Answer 1

This may not help solve the problem in the end, but it answers part of the question:

As Hot Licks already guessed in his comment, +72 simply a bytecode offset. I checked this with a small program:

^{(otherwise it was not connected, it was just that he was quickly available here from another question)}

 class TestNativeArray3D { public static void main(String args[]) { System.loadLibrary("TestNativeArray3D"); int terrain[][][] = genTerrain(123, 8, 6); } private native static int[][][] genTerrain(int seed, int x, int y); } 

The native genTerrain function explicitly creates an arbitrary error, causing

 jclass errorClass = env->FindClass("XXX"); env->NewObjectArray(10, errorClass, NULL); 

calling core dump.

The stack is as follows:

 Stack: [0x0000000002500000,0x0000000002600000], sp=0x00000000025ff4e0, free space=1021k Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) V [jvm.dll+0x14ebb6] C [TestNativeArray3D.dll+0x397d] JNIEnv_::NewObjectArray+0x4d C [TestNativeArray3D.dll+0x3ae8] Java_TestNativeArray3D_genTerrain+0x98 C 0x0000000002715534 Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j TestNativeArray3D.genTerrain(III)[[[I+0 j TestNativeArray3D.main([Ljava/lang/String;)V+11 v ~StubRoutines::call_stub 

Here is a line similarly containing the (possible) offset,

j TestNativeArray3D.main([Ljava/lang/String;)V +11

When decompiling a class file with

 javap -c -l TestNativeArray3D 

(pay attention to -l (small "L") to get line numbers!), output

 class TestNativeArray3D { TestNativeArray3D(); Code: 0: aload_0 1: invokespecial #1 // Method java/lang/Object."<init>":()V 4: return LineNumberTable: line 3: 0 public static void main(java.lang.String[]); Code: 0: ldc #2 // String TestNativeArray3D 2: invokestatic #3 // Method java/lang/System.loadLibrary:(Ljava/lang/String;)V 5: bipush 123 7: bipush 8 9: bipush 6 11: invokestatic #4 // Method genTerrain:(III)[[[I 14: astore_1 15: return LineNumberTable: line 7: 0 line 8: 5 line 9: 15 } 

indeed, the native call occurs at offset 11 . (And yes, I checked this by adding another code before this call: the offset in the kernel dump changes accordingly).

"LineNumberTable" allows you to display bytecode and line offsets. The table in the example means that

• line of source code 7 corresponds to the offset of byte code 0,
• line of source code 8 corresponds to the offset of bytecode 5,
• source code line 9 corresponds to bytecode offset 15

therefore, the critical instruction here is related to code line 8. (This was obvious in this case, because it was an invokestatic call invokestatic . I just wanted to indicate that you can search for LineNumberTable to display the offset of the bytecode to the actual line number in the source code)

---

Unfortunately, this is unlikely to help you really solve this error, because it obviously caused some levels deeper, in the native libc.so code, for some memcpy - which explicitly received invalid pointers from libzip.so ...