Why doesn't this HTTP request work on AWS Lambda?

Question

I'm starting with AWS Lambda, and I'm trying to request an external service from my handler function. According to this answer , the HTTP requests should work fine, and I did not find any documentation that says otherwise. (In fact, people sent a code that uses the Twilio API to send SMS .)

My handler code:

var http = require('http'); exports.handler = function(event, context) { console.log('start request to ' + event.url) http.get(event.url, function(res) { console.log("Got response: " + res.statusCode); }).on('error', function(e) { console.log("Got error: " + e.message); }); console.log('end request to ' + event.url) context.done(null); } 

and I see the following 4 lines in my CloudWatch logs:

 2015-02-11 07:38:06 UTC START RequestId: eb19c89d-b1c0-11e4-bceb-d310b88d37e2 2015-02-11 07:38:06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 start request to http://www.google.com 2015-02-11 07:38:06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 end request to http://www.google.com 2015-02-11 07:38:06 UTC END RequestId: eb19c89d-b1c0-11e4-bceb-d310b88d37e2 

I would expect another line:

 2015-02-11 07:38:06 UTC eb19c89d-b1c0-11e4-bceb-d310b88d37e2 Got response: 302 

but it is absent. If I use the main part without a shell handler in node on my local machine, the code works as expected.

inputfile.txt I use the following to invoke invoke-async :

 { "url":"http://www.google.com" } 

It seems that the part of the handler code that executes the request is completely skipped. I started with request lib and reverted to using simple http to create a minimal example. I also tried to request the URL of the service I control to check the logs and there are no requests coming in there.

I am completely at a dead end. Is there any reason node and / or AWS Lambda will not make an HTTP request?

Answer 1

Of course, I did not understand this problem. As AWS put it :

For those who first encounter nodejs in Lambda, the error forgets that callbacks are executed asynchronously and call context.done() in the original handler when you really wanted to wait for another callback (for example, S3.PUT operation) to complete. The forced function ends incompletely with its work.

I called context.done path before any callbacks to start the request, which led to the termination of my function ahead of time.

Work code:

 var http = require('http'); exports.handler = function(event, context) { console.log('start request to ' + event.url) http.get(event.url, function(res) { console.log("Got response: " + res.statusCode); context.succeed(); }).on('error', function(e) { console.log("Got error: " + e.message); context.done(null, 'FAILURE'); }); console.log('end request to ' + event.url); } 

Update: starting in 2017. AWS is deprecated from the old Nodejs 0.10, and now only the newer version 4.3 is available (old features need to be updated). This runtime has made some changes to the handler function. The new handler now has 3 parameters.

 function(event, context, callback) 

Although you still find succeed , done and fail in the context parameter, AWS will suggest using the callback or null function instead.

 callback(new Error('failure')) // to return error callback(null, 'success msg') // to return ok 

Full documentation can be found at http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-handler.html

Answer 2

Yes, awendt's answer is perfect. I will just show my working code ... I had a string context.succeed ('Blah'); immediately after the line reqPost.end (); . Moving it to where I show below solves everything.

 console.log('GW1'); var https = require('https'); exports.handler = function(event, context) { var body=''; var jsonObject = JSON.stringify(event); // the post options var optionspost = { host: 'the_host', path: '/the_path', method: 'POST', headers: { 'Content-Type': 'application/json', } }; var reqPost = https.request(optionspost, function(res) { console.log("statusCode: ", res.statusCode); res.on('data', function (chunk) { body += chunk; }); context.succeed('Blah'); }); reqPost.write(jsonObject); reqPost.end(); }; 

Answer 3

I had the same problem and realized that programming in NodeJS is actually different from Python or Java as JavaScript based. I will try to use simple concepts, as there may be several new people who would be interested or might come to this question.

Take a look at the following code:

 var http = require('http'); // (1) exports.handler = function(event, context) { console.log('start request to ' + event.url) http.get(event.url, // (2) function(res) { //(3) console.log("Got response: " + res.statusCode); context.succeed(); }).on('error', function(e) { console.log("Got error: " + e.message); context.done(null, 'FAILURE'); }); console.log('end request to ' + event.url); //(4) } 

Whenever you call a method in an http packet (1), it is created as an event, and this event receives a separate event. The get (2) function is actually the starting point of this separate event.

Now the function in (3) will be executed in a separate event, and your code will continue to execute the path and will automatically go to (4) and end it, because there is nothing more to do.

But the event released in (2) is still being performed somewhere, and it will take its sweet time to complete it. Pretty strange, right ?. Well no, this is not so. This is how NodeJS works, and it's pretty important that you envelop this concept. This is where JavaScript Promises comes to the rescue.

Learn more about JavaScript Promises here . In a nutshell you will need JavaScript Promise to keep the execution of the embedded code and will not create new / additional threads.

Most common NodeJS packages have an accessible version of their Promised API, but there are other approaches, such as BlueBirdJS, that address a similar issue.

The code you wrote above can be easily rewritten as follows.

 'use strict'; console.log('Loading function'); var rp = require('request-promise'); exports.handler = (event, context, callback) => { var options = { uri: 'https://httpbin.org/ip', method: 'POST', body: { }, json: true }; rp(options).then(function (parsedBody) { console.log(parsedBody); }) .catch(function (err) { // POST failed... console.log(err); }); context.done(null); }; 

Please note that the above code will not work directly if you import it into AWS Lambda. For Lambda, you will also need to package modules with a code base.

Answer 4

I found a lot of posts on the Internet about different ways of executing a request, but none of them showed how to handle a response synchronously in AWS Lambda.

Here the Node 6.10.3 lambda function, which uses an https request, collects and returns the full response body and transfers control to the unregistered processBody function with the results. I believe that http and https are interchangeable in this code.

I use the async helper module , which is easier for beginners to understand. You will need to port this to your AWS stack in order to use it (I recommend a serverless framework ).

Note that the data is returned in chunks that are collected in the global variable, and finally, the callback is called when the data is end .

 'use strict'; const async = require('async'); const https = require('https'); module.exports.handler = function (event, context, callback) { let body = ""; let countChunks = 0; async.waterfall([ requestDataFromFeed, // processBody, ], (err, result) => { if (err) { console.log(err); callback(err); } else { const message = "Success"; console.log(result.body); callback(null, message); } }); function requestDataFromFeed(callback) { const url = 'https://put-your-feed-here.com'; console.log('Sending GET request to ${url}'); https.get(url, (response) => { console.log('statusCode:', response.statusCode); response.on('data', (chunk) => { countChunks++; body += chunk; }); response.on('end', () => { const result = { countChunks: countChunks, body: body }; callback(null, result); }); }).on('error', (err) => { console.log(err); callback(err); }); } }; 

Answer 5

If you don’t care about the result of your http call (perhaps because it is a reliable service, such as the API endpoint for the firebase cloud functions, and a random failure will not matter, since this is a periodic cleaning work), then your code may be greatly simplified:

 const https = require('https'); exports.handler = (event, context, callback) => { console.log("Invoking GCP."); https.get('https://us-central1-your-app.cloudfunctions.net/runPeriodicCleanup'); console.log("GCP invoked."); }; 

Answer 6

A simple working example of an Http request using a node.

 const http = require('https') exports.handler = async (event) => { return httprequest().then((data) => { const response = { statusCode: 200, body: JSON.stringify(data), }; return response; }); }; function httprequest() { return new Promise((resolve, reject) => { const options = { host: 'jsonplaceholder.typicode.com', path: '/todos', port: 443, method: 'GET' }; const req = http.request(options, (res) => { if (res.statusCode < 200 || res.statusCode >= 300) { return reject(new Error('statusCode=' + res.statusCode)); } var body = []; res.on('data', function(chunk) { body.push(chunk); }); res.on('end', function() { try { body = JSON.parse(Buffer.concat(body).toString()); } catch(e) { reject(e); } resolve(body); }); }); req.on('error', (e) => { reject(e.message); }); // send the request req.end(); }); } 

Answer 7

Yes, there are actually many reasons why you can access AWS Lambda like and HTTP Endpoint.

AWS Lambda Architecture

This is a microservice. Running inside EC2 with Amazon Linux AMI (version 3.14.26-24.46.amzn1.x86_64) and works with Node.js. The memory can be equal to 128 MB and 1 GB. When the data source fires the event, the details are passed to the Lambda function as parameters.

What will happen?

AWS Lambda runs inside the container, and the code is directly loaded into this container with packages or modules. For example, we NEVER can do SSH for a Linux machine with your lambda function. The only things we can track are the logs, with CloudWatchLogs and the exception that came from the runtime.

AWS will take care of the launch and completion of the containers for us, and just run the code. Thus, even if you use require ("http"), it will not work because the place where this code is executed has not been done for this.