How to increase throughput on a NodeJS server using a cluster?

Question

I have a NodeJS server (Express), and I distribute requests to several processors using an example cluster module on a nodeJs node.

if (cluster.isMaster) { for (var i = 0; i < numCPUs; i++) { cluster.fork(); }; cluster.on('exit', function(worker, code, signal) { console.log('worker ' + worker.process.pid + ' died'); cluster.fork(); }); } else { server.listen(app.get('port'), function(){ console.log('HTTP server on port ' + app.get('port') + ' - running as ' + app.settings.env); }); // setup socket.io communication io.sockets.on('connection', require('./app/sockets')); io.sockets.on('connection', require('./app/downloadSockets')); } 

The problem is that the siege benchmark shows me that the number of hits is not increasing. This is the result of a siege:

 $ siege -c100 192.168.111.1:42424 -t10S ** SIEGE 3.0.5 ** Preparing 100 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 1892 hits Availability: 100.00 % Elapsed time: 10.01 secs Data transferred: 9.36 MB Response time: 0.01 secs Transaction rate: 189.01 trans/sec Throughput: 0.93 MB/sec Concurrency: 1.58 Successful transactions: 1892 Failed transactions: 0 Longest transaction: 0.05 Shortest transaction: 0.00 

After clustering:

 $ siege -c100 192.168.111.1:42424 -t10S ** SIEGE 3.0.5 ** Preparing 100 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 1884 hits Availability: 100.00 % Elapsed time: 9.52 secs Data transferred: 9.32 MB Response time: 0.01 secs Transaction rate: 197.90 trans/sec Throughput: 0.98 MB/sec Concurrency: 1.72 Successful transactions: 1884 Failed transactions: 0 Longest transaction: 0.07 Shortest transaction: 0.00 

Does this mean that my server is already getting the maximum throughput with one server, probably because its local machine or maybe it can’t get 4 processors, because too many processes are working, I'm not sure.

How to use the cluster module to increase performance and why is my current code not doing well? I also verified that it really creates 4 server instances, and cluster.fork works. Any advice would be very helpful.

Answer 1

The effect is achieved using parallel clustering or growth queries (try increasing the number of concurrent users to 300-400). Or tasks that bring a serious load. Let’s do a more interesting test: upload a file size of about 1 MB, plus we will delay 5 ms and 50 ms to emulate complex operations. For a quad-core processor during local testing, there will be the following (for conventional and cluster ones, respectively):

 $ siege -c100 http://localhost/images/image.jpg -t10S 

Normal mode (5 ms delay):

 Lifting the server siege... done. Transactions: 1170 hits Availability: 100.00 % Elapsed time: 9.10 secs Data transferred: 800.79 MB Response time: 0.27 secs Transaction rate: 128.57 trans/sec Throughput: 88.00 MB/sec Concurrency: 34.84 Successful transactions: 1170 Failed transactions: 0 Longest transaction: 0.95 Shortest transaction: 0.01 

Cluster mode (5 ms delay):

 Lifting the server siege... done. Transactions: 1596 hits Availability: 100.00 % Elapsed time: 9.04 secs Data transferred: 1092.36 MB Response time: 0.06 secs Transaction rate: 176.55 trans/sec Throughput: 120.84 MB/sec Concurrency: 9.81 Successful transactions: 1596 Failed transactions: 0 Longest transaction: 0.33 Shortest transaction: 0.00 

Normal mode (50 ms delay):

 Lifting the server siege... done. Transactions: 100 hits Availability: 100.00 % Elapsed time: 9.63 secs Data transferred: 68.44 MB Response time: 5.51 secs Transaction rate: 10.38 trans/sec Throughput: 7.11 MB/sec Concurrency: 57.18 Successful transactions: 100 Failed transactions: 0 Longest transaction: 7.77 Shortest transaction: 5.14 

Cluster mode (delay 50 ms):

 Lifting the server siege... done. Transactions: 614 hits Availability: 100.00 % Elapsed time: 9.24 secs Data transferred: 420.25 MB Response time: 0.90 secs Transaction rate: 66.45 trans/sec Throughput: 45.48 MB/sec Concurrency: 59.59 Successful transactions: 614 Failed transactions: 0 Longest transaction: 1.50 Shortest transaction: 0.50 

Answer 2

you are not doing anything in your example. Connect to mysql and run a heavy request or make an HTTP request that takes a few seconds. You noticed that in the end you will code something (or use a third-party library) that blocks the event loop. This happens when clustering is important because an event loop will exist for each processor. If one request is slow and the event loop needs to wait for it, it will not stop new requests that fall into your API / application.

In addition, you can read information about the connection pool and the specific generic pool on npm if you plan to use or connect to the database or receive external resources.