Initialize an ec2 instance using data in s3

Question

When starting AMI for Linux, I would like to load some data (for example, solar java libraries and some proprietary programs) from the S3 bucket when initializing my EC2 connection. I notice that one way to do this is to use s3cmd. BUT, how to get the configuration file for s3cmd.s3cfg from S3?

Also, let me also know if there is a better way to connect to the S3 repository during init.

Thanks!

Answer 1

If this is something living java libraries, I suppose you can make them public in your S3 bucket, why not use HTTP requests and curl or wget ?

EDIT:
Personally, I would use the Python S3 libraries (choose a different language if you do not like python) and write a custom script that includes my passkey (if you distribute your code, you need to make sure how you handle the security keys) and extract the ones I need files.

I use this: http://aws.amazon.com/code/134

Answer 2

Obtaining AWS credentials (and other secrets) on an EC2 instance and storing them securely on the instance is a complex problem. The best solution depends on the application and the project environment.

Shlomo Svidler wrote an extensive article studying this problem:

http://shlomoswidler.com/2009/08/how-to-keep-your-aws-credentials-on-ec2.html

A year later, he took a different approach, which has other advantages:

http://shlomoswidler.com/2010/07/storing-aws-credentials-on-an-ebs-snapshot-securely.html