Error handling in asynchronous calls to node.js

Question

I'm new to node.js, although I am pretty familiar with JavaScript in general. My question is about "best practices" on how to handle errors in node.js.

Usually when programming web servers, FastCGI servers or web pages in different languages, I use exceptions with blocking handlers in a multi-threaded environment. When the request comes in, I usually do something like this:

function handleRequest(request, response) { try { if (request.url=="whatever") handleWhateverRequest(request, response); else throw new Error("404 not found"); } catch (e) { response.writeHead(500, {'Content-Type': 'text/plain'}); response.end("Server error: "+e.message); } } function handleWhateverRequest(request, response) { if (something) throw new Error("something bad happened"); Response.end("OK"); } 

That way, I can always handle internal errors and send a valid response to the user.

I understand that with node.js it is supposed to use non-blocking calls, which obviously leads to a different number of callbacks, as in this example:

 var sys = require('sys'), fs = require('fs'); require("http").createServer(handleRequest).listen(8124); function handleRequest(request, response) { fs.open("/proc/cpuinfo", "r", function(error, fd) { if (error) throw new Error("fs.open error: "+error.message); console.log("File open."); var buffer = new require('buffer').Buffer(10); fs.read(fd, buffer, 0, 10, null, function(error, bytesRead, buffer) { buffer.dontTryThisAtHome(); // causes exception response.end(buffer); }); //fs.read }); //fs.open } 

This example will completely destroy the server because exceptions are not caught. My problem is that I can no longer use a single try / catch and thus will not catch any errors that may occur when processing the request.

Of course, I could add try / catch in each callback, but I do not like this approach, because then it is up to the programmer that he does not forget to try / catch. For a complex server with many different and complex handlers, this is unacceptable.

I could use a global exception handler (preventing a server from crashing completely), but then I cannot send a response to the user, since I do not know which request will result in the exception. It also means that the request remains unprocessed / open, and the browser waits forever for a response.

Does anyone have a good solid solution?

Answer 1

Node 0.8 introduces a new concept called Domains. They are very similar to AppDomains in .net and provide a way to encapsulate a group of I / O operations. They basically allow you to wrap request processing calls in a context-specific group. If this group throws any uncaught exceptions, they can be processed and processed in such a way that you can access all the required area and specific specific information for successful error recovery (if possible).

This feature is new and has just been introduced, so use it with caution, but from what I can say, it was specifically introduced to solve the problem that the OP is trying to solve.

The documentation can be found at: http://nodejs.org/api/domain.html

Answer 2

To issue an uncaughtException handler in node.js. It captures discarded errors that go beyond the scope of the event loop.

http://nodejs.org/docs/v0.4.7/api/process.html#event_uncaughtException_

But not throwing mistakes is always better. You can simply do return res.end('Unabled to load file xxx');

Answer 3

This is one of the problems with Node right now. It is almost impossible to track which request caused the error caused by the callback.

You will have to handle your errors in the callbacks themselves (where you still have a link to the request and response objects), if possible. The uncaughtException handler will stop the Node process from exiting, but the request that threw the exception in the first place will just hang there from the user's point of view.

Answer 4

I give an answer to my question ... :)

It seems that there is no way to manually spoof errors. Now I use a helper function, which itself returns a function containing a try / catch block. In addition, my own web server class checks to see if the request processing function calls response.end() or the try / catch helper function waitfor() (otherwise, an exception is thrown). This avoids the request being mistakenly left by an insecure developer. This is not a 100% error solution, but good enough for me.

 handler.waitfor = function(callback) { var me=this; // avoid exception because response.end() won't be called immediately: this.waiting=true; return function() { me.waiting=false; try { callback.apply(this, arguments); if (!me.waiting && !me.finished) throw new Error("Response handler returned and did neither send a "+ "response nor did it call waitfor()"); } catch (e) { me.handleException(e); } } } 

Thus, I just need to add the waitfor() built-in call to be safe.

 function handleRequest(request, response, handler) { fs.read(fd, buffer, 0, 10, null, handler.waitfor( function(error, bytesRead, buffer) { buffer.unknownFunction(); // causes exception response.end(buffer); } )); //fs.read } 

The actual verification mechanism is a bit more complicated, but it should be clear how this works. If anyone is interested, I can post the full code here.

Answer 5

Very good question. I am dealing with the same problem now. Probably the best way would be to use uncaughtException . Linking to respone and request objects is not a problem, because you can wrap them in your exception object, which is passed to the uncaughtException event. Something like that:

 var HttpException = function (request, response, message, code) { this.request = request; this.response = response; this.message = message; this.code = code || 500; } 

Drop it:

 throw new HttpException(request, response, 'File not found', 404); 

And process the answer:

 process.on('uncaughtException', function (exception) { exception.response.writeHead(exception.code, {'Content-Type': 'text/html'}); exception.response.end('Error ' + exception.code + ' - ' + exception.message); }); 

I have not tested this solution yet, but I see no reason why this did not work.

Answer 6

One idea: you can simply use a helper method to create callbacks and use it in your standard practice. This puts a strain on the developer, but at least you might have a “standard” way of handling your callbacks, so there’s a low chance of forgetting it:

 var callWithHttpCatch = function(response, fn) { try { fn && fn(); } catch { response.writeHead(500, {'Content-Type': 'text/plain'}); //No } } <snipped> var buffer = new require('buffer').Buffer(10); fs.read(fd, buffer, 0, 10, null, function(error, bytesRead, buffer) { callWithHttpCatch(response, buffer.dontTryThisAtHome()); // causes exception response.end(buffer); }); //fs.read }); //fs.open 

I know that this is probably not the answer you were looking for, but one of the nice things about ECMAScript (or functional programming in general) is how easily you can use your own tool for such things.

Answer 7

While writing this approach, I see this using "Promises".

http://howtonode.org/promises
https://www.promisejs.org/

They allow you to structure your code and callbacks well for error management, as well as make it more readable. It mainly uses the .then () function.

 someFunction().then(success_callback_func, failed_callback_func); 

Here is a basic example:

  var SomeModule = require('someModule'); var success = function (ret) { console.log('>>>>>>>> Success!'); } var failed = function (err) { if (err instanceof SomeModule.errorName) { // Note: I've often seen the error definitions in SomeModule.errors.ErrorName console.log("FOUND SPECIFIC ERROR"); } console.log('>>>>>>>> FAILED!'); } someFunction().then(success, failed); console.log("This line with appear instantly, since the last function was asynchronous."); 

Answer 8

Two things really helped me solve this problem in my code.

• The 'longjohn' module, which allows you to see the full stack trace (via several asynchronous callbacks).

• A simple closing technique for storing exceptions in the standard callback(err, data) idiom callback(err, data) (shown here in CoffeeScript).

   ferry_errors = (callback, f) -> return (a...) -> try f(a...) catch err callback(err) 

Now you can wrap insecure code, and your callbacks handle errors the same way: checking the error argument.

Answer 9

I recently created a simple abstraction called WaitFor to call asynchronous functions in synchronization mode (based on Fibers): https://github.com/luciotato/waitfor

This is too new to be hard.

using wait.for , you can use the async function as if they were synchronized, without blocking the node event loop. This is almost the same as you are used to:

 var wait=require('wait.for'); function handleRequest(request, response) { //launch fiber, keep node spinning wait.launchFiber(handleinFiber,request, response); } function handleInFiber(request, response) { try { if (request.url=="whatever") handleWhateverRequest(request, response); else throw new Error("404 not found"); } catch (e) { response.writeHead(500, {'Content-Type': 'text/plain'}); response.end("Server error: "+e.message); } } function handleWhateverRequest(request, response, callback) { if (something) throw new Error("something bad happened"); Response.end("OK"); } 

Since you are in a fiber, you can sequentially program a "fiber lock", but not a node event loop.

Another example:

 var sys = require('sys'), fs = require('fs'), wait = require('wait.for'); require("http").createServer( function(req,res){ wait.launchFiber(handleRequest,req,res) //handle in a fiber ).listen(8124); function handleRequest(request, response) { try { var fd=wait.for(fs.open,"/proc/cpuinfo", "r"); console.log("File open."); var buffer = new require('buffer').Buffer(10); var bytesRead=wait.for(fs.read,fd, buffer, 0, 10, null); buffer.dontTryThisAtHome(); // causes exception response.end(buffer); } catch(err) { response.end('ERROR: '+err.message); } } 

As you can see, I used wait.for to call the asynchronous node functions in synchronization mode, without visible callbacks, so I can have all the code inside a single try-catch block.

wait.for will throw an exception if any of the async functions returns err! == null

Additional information at https://github.com/luciotato/waitfor

Answer 10

Also in synchronous multi-threaded programming (for example, .NET, Java, PHP) you cannot return any meaningful information to the client when a custom Unkown exception is detected. You can simply return HTTP 500 when you do not have exception information.

So the “secret” is to populate the Error descriptive object, so your error handler can display a significant error in the correct HTTP + state, optionally a descriptive result. However, you must also catch the exception before it arrives at process.on ('uncaughtException'):

Step1: determining the meaningful error object

 function appError(errorCode, description, isOperational) { Error.call(this); Error.captureStackTrace(this); this.errorCode = errorCode; //...other properties assigned here }; appError.prototype.__proto__ = Error.prototype; module.exports.appError = appError; 

Step 2: when throwing an exception, fill it with properties (see step 1), which allows the handler to convert it to the meannigul HTTP result:

 throw new appError(errorManagement.commonErrors.resourceNotFound, "further explanation", true) 

Step 3: When calling some potentially dangerous code, catch errors and rethrow this error when filling in additional context properties in the Error object

Step 4: You must catch the exception while processing the request. This is easier if you use some kind of leading promises library (BlueBird is excellent), which allows you to catch asynchronous errors. If you cannot use promises than any NODE built-in library will return errors in the callback.

Step 5. Now that your error is caught and contains descriptive information about what is happening, you only need to match it with a meaningful HTTP response. The nice part here is that you can have a centralized, single error handler that receives all errors and matches them with the HTTP response:

  //this specific example is using Express framework res.status(getErrorHTTPCode(error)) function getErrorHTTPCode(error) { if(error.errorCode == commonErrors.InvalidInput) return 400; else if... } 

You can use other relevant best practices here.