Download to Amazon S3 with tinys3

Question

I use Python and tinys3 to write files to S3, but it does not work. Here is my code:

import tinys3 conn = tinys3.Connection('xxxxxxx','xxxxxxxx',tls=True) f = open('testing_s3.txt','rb') print conn.upload('testing_data/testing_s3.txt',f,'testing-bucket') print conn.get('testing_data/testing_s3.txt','testing-bucket') 

This gives the result:

 <Response [301]> <Response [301]> 

When I try to specify the endpoint, I get:

 requests.exceptions.HTTPError: 403 Client Error: Forbidden 

Any idea what I'm doing wrong?

Edit: When I try to use boto, it works, so the problem is not with the access key or secret key.

Answer 1

I finally figured it out. Here is the correct code:

 import tinys3 conn = tinys3.Connection('xxxxxxx','xxxxxxxx',tls=True,endpoint='s3-us-west-1.amazonaws.com') f = open('testing_s3.txt','rb') print conn.upload('testing_data/testing_s3.txt',f,'testing-bucket') print conn.get('testing_data/testing_s3.txt','testing-bucket') 

You should use the endpoint of the region, not s3.amazonaws.com. You can see the endpoint of the region here: http://docs.aws.amazon.com/general/latest/gr/rande.html . Look under the heading "Amazon Simple Storage Service (S3)."

I got an idea from this thread: https://github.com/smore-inc/tinys3/issues/5

Answer 2

When using the IAM user, you must enable the "s3: PutObjectAcl" action.

Answer 3

I don't know why, but this code never worked for me. I switched to boto and it just downloaded the file from 1 time.

  AWS_ACCESS_KEY_ID = 'XXXXXXXXXXXXXXXXXXXXX' AWS_SECRET_ACCESS_KEY = 'XXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXX' bucket_name = 'my-bucket' conn = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) bucket = conn.get_bucket('my-bucket') print 'Uploading %s to Amazon S3 bucket %s' % \ (filename, bucket_name) k = Key(bucket) k.key = filename k.set_contents_from_filename(filename, cb=percent_cb, num_cb=10)