We have a mongodb replica set on which a member crashed with a segmentation error. What could be causing this problem? We are launching version 2.2.2.
Thanks. Here is a log from the accident.
Mon Sep 2 03:37:26 Invalid access at address: 0xfffffd7d00680038 from thread: conn2014070 Mon Sep 2 03:37:26 Got signal: 11 (Segmentation Fault). Mon Sep 2 03:37:26 Backtrace: 0xb331b8 0x7bd48b 0x7bd695 0xfffffd7fff1d7666 0xfffffd7fff1ca35c 0x9ff980 0x873f13 0x873fcb 0x981331 0x982af2 0x92d2da 0x93183b 0x7cead0 0xb2539a 0xfffffd7ff95f364c 0xfffffd7fff1d72d4 0xfffffd7fff1d75a0 /opt/local/bin/mongod'_ZN5mongo15printStackTraceERSo+0x28 [0xb331b8] /opt/local/bin/mongod'_ZN5mongo10abruptQuitEi+0x11b [0x7bd48b] /opt/local/bin/mongod'_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x125 [0x7bd695] /lib/amd64/libc.so.1'__sighndlr+0x6 [0xfffffd7fff1d7666] /lib/amd64/libc.so.1'call_user_handler+0x2a4 [0xfffffd7fff1ca35c] /opt/local/bin/mongod'_ZNK5mongo6Record5touchEb+0x0 [0x9ff980] /opt/local/bin/mongod'_ZN5mongo12ClientCursor5yieldEiPNS_6RecordE+0x63 [0x873f13] /opt/local/bin/mongod'_ZN5mongo12ClientCursor14yieldSometimesENS0_11RecordNeedsEPb+0x6b [0x873fcb] /opt/local/bin/mongod'_ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEPNS_11RemoveSaverEbRKNS_24QueryPlanSelectionPolicyEb+0x9a1 [0x981331] /opt/local/bin/mongod'_ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES4_bbbRNS_7OpDebugEbRKNS_24QueryPlanSelectionPolicyE+0xa2 [0x982af2] /opt/local/bin/mongod'_ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE+0x27a [0x92d2da] /opt/local/bin/mongod'_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0xe9b [0x93183b] /opt/local/bin/mongod'_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x90 [0x7cead0] /opt/local/bin/mongod'_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x32a [0xb2539a] /opt/local/lib/libboost_thread.so.1.49.0'thread_proxy+0x6c [0xfffffd7ff95f364c] /lib/amd64/libc.so.1'_thrp_setup+0xbc [0xfffffd7fff1d72d4] /lib/amd64/libc.so.1'_lwp_start+0x0 [0xfffffd7fff1d75a0]
Also, I see some assertion failures before the failure, I'm not sure if they are related. Otherwise, as far as I know, nothing unusual.
Wed Sep 4 02:19:04 [conn988803] cratefm Assertion failure !e.eoo() src/mongo/db/../bson/bsonobjbuilder.h 131 0xb331b8 0xb01e70 0x7cbe04 0x88e7ec 0x8b5f18 0x8b6b66 0x8b714a 0x978044 0x97ab32 0x931065 0x7cead0 0xb2539a 0xfffffd7fdd1a364c 0xfffffd7fff1d72d4 0xfffffd7fff1d75a0 /opt/local/bin/mongod'_ZN5mongo15printStackTraceERSo+0x28 [0xb331b8] /opt/local/bin/mongod'_ZN5mongo12verifyFailedEPKcS1_j+0xc0 [0xb01e70] /opt/local/bin/mongod'0x3cbe04 [0x7cbe04] /opt/local/bin/mongod'_ZN5mongo16CmdFindAndModify3runERKSsRNS_7BSONObjEiRSsRNS_14BSONObjBuilderEb+0x65c [0x88e7ec] /opt/local/bin/mongod'_ZN5mongo12_execCommandEPNS_7CommandERKSsRNS_7BSONObjEiRNS_14BSONObjBuilderEb+0x48 [0x8b5f18] /opt/local/bin/mongod'_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0xa26 [0x8b6b66] /opt/local/bin/mongod'_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x37a [0x8b714a] /opt/local/bin/mongod'_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_11_BufBuilderINS_16TrivialAllocatorEEERNS_14BSONObjBuilderEbi+0x34 [0x978044] /opt/local/bin/mongod'_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x6c2 [0x97ab32] /opt/local/bin/mongod'_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x6c5 [0x931065] /opt/local/bin/mongod'_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0x90 [0x7cead0] /opt/local/bin/mongod'_ZN5mongo3pms9threadRunEPNS_13MessagingPortE+0x32a [0xb2539a] /opt/local/lib/libboost_thread.so.1.49.0'thread_proxy+0x6c [0xfffffd7fdd1a364c] /lib/amd64/libc.so.1'_thrp_setup+0xbc [0xfffffd7fff1d72d4] /lib/amd64/libc.so.1'_lwp_start+0x0 [0xfffffd7fff1d75a0]
source share