JVM crashes without a specific frame, only "timer expired",

Question

I am running Java work under Hadoop, which breaks down the JVM. I suspect this is due to some JNI code (it uses JBLAS with a multi-threaded embedded BLAS implementation). However, while I expect the crash log to provide a “problem frame” for debugging, the log instead looks like this:

# # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00007f204dd6fb27, pid=19570, tid=139776470402816 # # JRE version: 6.0_38-b05 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.13-b02 mixed mode linux-amd64 compressed oops) # Problematic frame: # # [ timer expired, abort... ] 

Does the JVM have some kind of timer, how long will it wait for this emergency exit to be generated? If so, is there a way to increase the time so that I can get more useful information? I don't think the timer in question comes from Hadoop, as I see (useless) links to this error in many places that do not mention Hadoop.

Google shows that the line "timer expired, abort" appears only in these JVM error messages, so it is unlikely to come from the OS.

Edit: It seems like I'm probably out of luck. From ./hotspot/src/share/vm/runtime/thread.cpp in the OpenJDK version of the JVM source:

  if (is_error_reported()) { // A fatal error has happened, the error handler(VMError::report_and_die) // should abort JVM after creating an error log file. However in some // rare cases, the error handler itself might deadlock. Here we try to // kill JVM if the fatal error handler fails to abort in 2 minutes. // // This code is in WatcherThread because WatcherThread wakes up // periodically so the fatal error handler doesn't need to do anything; // also because the WatcherThread is less likely to crash than other // threads. for (;;) { if (!ShowMessageBoxOnError && (OnError == NULL || OnError[0] == '\0') && Arguments::abort_hook() == NULL) { os::sleep(this, 2 * 60 * 1000, false); fdStream err(defaultStream::output_fd()); err.print_raw_cr("# [ timer expired, abort... ]"); // skip atexit/vm_exit/vm_abort hooks os::die(); } // Wake up 5 seconds later, the fatal handler may reset OnError or // ShowMessageBoxOnError when it is ready to abort. os::sleep(this, 5 * 1000, false); } } 

It seems that it is hard-coded to wait two minutes. Why the accident report takes more time for my work, I don’t know, but I think that at least this question has been answered.

Answer 1

It seems like I'm probably out of luck. From. /hotspot/src/share/vm/runtime/thread.cpp in the OpenJDK JVM source version:

  if (is_error_reported()) { // A fatal error has happened, the error handler(VMError::report_and_die) // should abort JVM after creating an error log file. However in some // rare cases, the error handler itself might deadlock. Here we try to // kill JVM if the fatal error handler fails to abort in 2 minutes. // // This code is in WatcherThread because WatcherThread wakes up // periodically so the fatal error handler doesn't need to do anything; // also because the WatcherThread is less likely to crash than other // threads. for (;;) { if (!ShowMessageBoxOnError && (OnError == NULL || OnError[0] == '\0') && Arguments::abort_hook() == NULL) { os::sleep(this, 2 * 60 * 1000, false); fdStream err(defaultStream::output_fd()); err.print_raw_cr("# [ timer expired, abort... ]"); // skip atexit/vm_exit/vm_abort hooks os::die(); } // Wake up 5 seconds later, the fatal handler may reset OnError or // ShowMessageBoxOnError when it is ready to abort. os::sleep(this, 5 * 1000, false); } } 

It seems that it is hard-coded to wait two minutes. Why the accident report takes more time for my work, I don’t know, but I think that at least this question has been answered.

Answer 2

The way to do this is to specify -XX:ShowMessageBoxOnError on the command line and join the process using a debugger from a different term.