All geek questions in one place

Lost communication with mongoDB when running too many tests

I have a problem with MongoDb when running too many JUnit tests that use my local test database. When I run all the tests one by one, everything is fine, and each test passes. But when I try to "Run all tests", somewhere after the 80th test, I lose contact with Mongo, and all subsequent tests fail. This is strange because, as I understand it, I see that “run all tests” just makes all tests run in a row.

I drop my db at @ After each test, so I thought that maybe creating my own unique database for each test could solve the problem (before I used one). But this is not so. I still lose contact after 80+ tests (each time each time).

Maybe the driver does not have enough time to get a response from Mongo at some point, and this leads to a failure?

I am using the 2.11.3 Mongo Java driver. Need help. Thanks.

Here is the error stack:

Sep 13, 2013 5:32:07 PM com.mongodb.DBTCPConnector initDirectConnection WARNING: Exception executing isMaster command on /127.0.0.1:27017 java.io.EOFException at org.bson.io.Bits.readFully(Bits.java:48) at org.bson.io.Bits.readFully(Bits.java:33) at org.bson.io.Bits.readFully(Bits.java:28) at com.mongodb.Response.<init>(Response.java:40) at com.mongodb.DBPort.go(DBPort.java:142) at com.mongodb.DBPort.go(DBPort.java:106) at com.mongodb.DBPort.findOne(DBPort.java:162) at com.mongodb.DBPort.runCommand(DBPort.java:170) at com.mongodb.DBTCPConnector.initDirectConnection(DBTCPConnector.java:547) at com.mongodb.DBTCPConnector.isMongosConnection(DBTCPConnector.java:334) at com.mongodb.Mongo.isMongosConnection(Mongo.java:618) at com.mongodb.DB.wrapCommand(DB.java:282) at com.mongodb.DB.command(DB.java:260) at com.mongodb.DB.command(DB.java:244) at com.mongodb.DB.command(DB.java:301) at com.mongodb.DB.command(DB.java:199) at com.mongodb.DB.dropDatabase(DB.java:557) at com.*******.dbconnection.mongodb.BaseMongodbTest.tearDown(BaseMongodbTest.java:102) at com.*******.rpcserver.methods.BaseTestClient.tearDown(BaseTestClient.java:57) at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) com.mongodb.MongoException$Network: Read operation to server /127.0.0.1:27017 failed on database test_mydb_fb19fd07-6e4f-4 at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:253) at com.mongodb.DBTCPConnector.call(DBTCPConnector.java:216) at com.mongodb.DBApiLayer$MyCollection.__find(DBApiLayer.java:288) at com.mongodb.DB.command(DB.java:262) at com.mongodb.DB.command(DB.java:244) at com.mongodb.DB.command(DB.java:301) at com.mongodb.DB.command(DB.java:199) at com.mongodb.DB.dropDatabase(DB.java:557) at com.*******.dbconnection.mongodb.BaseMongodbTest.tearDown(BaseMongodbTest.java:102) at com.*******.rpcserver.methods.BaseTestClient.tearDown(BaseTestClient.java:57) at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runners.Suite.runChild(Suite.java:128) at org.junit.runners.Suite.runChild(Suite.java:24) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.junit.runner.JUnitCore.run(JUnitCore.java:157) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:77) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:195) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:63) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120) Caused by: java.io.EOFException at org.bson.io.Bits.readFully(Bits.java:48) at org.bson.io.Bits.readFully(Bits.java:33) at org.bson.io.Bits.readFully(Bits.java:28) at com.mongodb.Response.<init>(Response.java:40) at com.mongodb.DBPort.go(DBPort.java:142) at com.mongodb.DBPort.call(DBPort.java:92) at com.mongodb.DBTCPConnector.innerCall(DBTCPConnector.java:244) ... 42 more
+4
source share
1 answer

I seem to have figured out where the problem is. There was a stupid mistake in our test client: a new Mongo object was created for each test case! Team

db.serverStatus (). Connections

shows that the number of available connections is about 200, but each new instance of Mongo creates 10 new (default) connections. So I just exceeded this limit, as was shown in the logs (you can read them, for example, by doing mongo --eval = "printjson (db.adminCommand ({getLog: 'global'}))" )

The solution is to create a factory class that stores the only instance of the Mongo object that is used in each test case (I also initialize the DB and MongoClient instances here and recommend that you do the same if you need them too).

I highly recommend reading this topic, it solves a similar problem (and it helped me a lot!): Mongo opens too many connections

Thanks to @RobMoore and @interlude for quick answers and a willingness to help!

+2
source

Source: https://habr.com/ru/post/1502060/

More articles:


All Articles