Node.js / Awssum for S3

Question

Hi: I am wondering if anyone knows a method for listing all the files inside an S3 bucket with node.js and Awssum, starting with the most recent file.

By default, my code accepts the first file created.

function getFileList(callback){ s3.ListObjects(coldBucket, function(err, data) { var fileList = getFilenames(data, coldBucketPath); callback(fileList); }); }; 

Any help is much appreciated! Thank you

Answer 1

This is Andy, creator of AwsSum.

That's right, there is no way to query S3 for file names in nested order. However, you can get it using LastModified.

If you want to see an example to get all the keys, see this example here:

• https://github.com/appsattic/node-awssum-scripts/blob/master/bin/amazon-s3-list.js

You can then save each item in an array, and then sort it using LastModified. The underline library contains a function that would be useful for this:

• http://documentcloud.github.com/underscore/#sortBy

Answer 2

I look at http://docs.amazonwebservices.com/AmazonS3/latest/API/RESTBucketGET.html , I do not believe s3 allows you to specify the sort order of the response.

However, you can work around this by changing the structure of your file to include folders with a name after the date they were created, then you can make a couple of ListObjects requests with prefixes for the last days / months / etc.

For example, you could have year-month folder names so that your file structure looks like this:

 bucket_name/ - 2012-3/ - file2.jpg - file3.jpg - 2012-2/ - file1.jpg 

and then before you make your request do it

 var now = new Date(); coldBucket.Prefix = now.getFullYear() + "-" + now.getMonth(); 

And then, if you want older things, you would need to complete a subsequent request for the previous months.

(If you have too many files even for this, you can try year-month-day )