When using cluster and winston in nodejs, log maxsize does not work

Question

When I use cluster and winston in nodejs, log maxsize does not work. The entire log records one file, although the size depends on maxsize. See the same issue: https://github.com/flatiron/winston/issues/275

Answer 1

The obvious solution: let only the main log file.

var winston = require('winston'); var cluster = require('cluster'); if(cluster.isMaster) { cluster.setupMaster({ silent: true }); // Keep cluster from automatically grabbing stdin/out/err for(var i = 0; i < 4; i++) { cluster.fork(); } winston.add(winston.transport.File, { filename: 'log.txt' }); cluster.workers.forEach(function(worker, i) { worker.process.stdout.on('data', function(chunk) { winston.info('worker ' + i + ': ' + chunk); }); worker.process.stderr.on('data', function(chunk) { winston.warn('worker ' + i + ': ' + chunk); }); }); } else { // Leave winston alone and only log to stdout/err in the workers } 

Then only one process has a file descriptor hold, so your log rotation should work as usual.

Answer 2

Maybe you should use the DailyFileRotation. To date, I do not see any problems. I use winston every time in master and forked processes, and they have the same file.

Here is what I found from testing. You can see that all logs are evenly distributed before and after the turn. You can also clearly see circumcision in magazines. Old magazines stopped growing after the switch.

 joephone@Fengs-MacBook-Pro-2 :logs$ ls total 3184 -rw-r--r-- 1 joephone staff 985111 Aug 24 20:01 info.log.2016-08-25-03-00 -rw-r--r-- 1 joephone staff 641506 Aug 24 20:01 info.log.2016-08-25-03-01 joephone@Fengs-MacBook-Pro-2 :logs$ cut -f8 -d"\"" info.log.2016-08-25-03-00 | sort | uniq -c 3299 65151 3355 65152 3307 65153 3345 65154 1 cpucount:4 1 worker 65151 starts working. 1 worker 65152 starts working. 1 worker 65153 starts working. 1 worker 65154 starts working. joephone@Fengs-MacBook-Pro-2 :logs$ cut -f8 -d"\"" info.log.2016-08-25-03-01 | sort | uniq -c 2193 65151 2207 65152 2122 65153 2147 65154 joephone@Fengs-MacBook-Pro-2 :logs$ tail -3 info.log.2016-08-25-03-00 {"level":"info","message":"65154","timestamp":"2016-08-25T03:00:59.997Z"} {"level":"info","message":"65153","timestamp":"2016-08-25T03:00:59.997Z"} {"level":"info","message":"65152","timestamp":"2016-08-25T03:00:59.997Z"} joephone@Fengs-MacBook-Pro-2 :logs$ head -3 info.log.2016-08-25-03-01 {"level":"info","message":"65151","timestamp":"2016-08-25T03:01:00.000Z"} {"level":"info","message":"65151","timestamp":"2016-08-25T03:01:00.012Z"} {"level":"info","message":"65152","timestamp":"2016-08-25T03:01:00.006Z"} 

For my curiosity, I also checked by putting all the code in each forked process. I still do not see any problems.

 joephone@Fengs-MacBook-Pro-2 :logs$ ls total 3216 -rw-r--r-- 1 joephone staff 993838 Aug 24 20:14 info.log.2016-08-25-03-13 -rw-r--r-- 1 joephone staff 650312 Aug 24 20:14 info.log.2016-08-25-03-14 joephone@Fengs-MacBook-Pro-2 :logs$ cut -f8 -d"\"" info.log.2016-08-25-03-13 | sort | uniq -c 3402 65755 3342 65756 3344 65757 3337 65758 1 worker 65755 starts working. 1 worker 65756 starts working. 1 worker 65757 starts working. 1 worker 65758 starts working. joephone@Fengs-MacBook-Pro-2 :logs$ cut -f8 -d"\"" info.log.2016-08-25-03-14 | sort | uniq -c 2183 65755 2202 65756 2230 65757 2173 65758 joephone@Fengs-MacBook-Pro-2 :logs$ tail -3 info.log.2016-08-25-03-13 {"level":"info","message":"65757","timestamp":"2016-08-25T03:13:59.986Z"} {"level":"info","message":"65755","timestamp":"2016-08-25T03:13:59.986Z"} {"level":"info","message":"65756","timestamp":"2016-08-25T03:13:59.994Z"} joephone@Fengs-MacBook-Pro-2 :logs$ head -3 info.log.2016-08-25-03-14 {"level":"info","message":"65758","timestamp":"2016-08-25T03:14:00.005Z"} {"level":"info","message":"65757","timestamp":"2016-08-25T03:14:00.005Z"} {"level":"info","message":"65758","timestamp":"2016-08-25T03:14:00.024Z"} 

For my curiosity, I even checked it with an error, knocking down a forked process, and the master brought a new one. The magazine is still working.

In fact, the two methods may be different: if you use ps for grep script, you are working, you will see the master process and forked processes. I think they just look like you are running them on multiple terminals.