Is it a good idea to open several Mongolian connections from a single Node.js application?

Question

Background . I am trying to optimize the speed of my Node.js API created using Express and Mongoose in the Amazon cloud. I have one API call that takes a long time to start (my / stats API call compiles data from a large number of sources and therefore makes hundreds of mongo requests and therefore takes about 20 seconds to start). I noticed that while this API call has been launched, other API calls that also fall into the Mongo replica set are slowly returning. My first thought was that the statistics queries were slow, therefore blocking, but according to my statistics panel, I have no queries requiring> 100 ms to run, and also my statistics in Mongo DB are in pretty healthy ranges (20 + requests for the second, <1% btree miss, <5% lock).

Question Currently, my Node.js application establishes a single connection with Mongo at startup, and all requests use this connection. Would it be better to establish multiple connections? For example, should you instead establish a new Mongo connection for each incoming API request? Or maybe I should have a static number of connections to the replica set and a load balance between these connections when making queries?

Or maybe I'm completely out of base?

Answer 1

You absolutely must use more than one connection, as otherwise only one request can be executed at a time. The cleanest way to do this is simply to include pooling in your mongodb.Server object when creating the Db object. For instance:

 var serverOptions = { 'auto_reconnect': true, 'poolSize': 5 }; var mdb = new mongodb.Db('test', new mongodb.Server('127.0.0.1', 27017, serverOptions)); mdb.open(function (err, mdb) { ... } 

The reason you don't see the slow query time in statistics is because your concurrent requests just come to life on the client side, waiting for the connection to become available. This timeout will not appear in server-side time requests.