Script to download a file from Amazon S3 trash

Question

Trying to write a script to download a file from Amazon S3 Recycle Bin.

There were problems with the example on the cURL website. The following is the script:

The request signature we signed does not match your subscription provided. Check your key and signature method.

Appreciate any help.

#!/bin/sh file="filename.php" bucket="my-bucket" resource="/${bucket}/${file}" contentType="text/html" dateValue="`date +'%a, %d %b %Y %H:%M:%S %z'`" stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}" s3Key='ABCABCABCABCABCABCAB' s3Secret='xyzxyzyxyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyzx' signature=`/bin/echo -en "$stringToSign" | openssl sha1 -hmac ${s3Secret} - binary | base64` curl -v -H "Host:lssngen-updates-east.s3.amazonaws.com" \ -H "Date:${dateValue}" \ -H "Content-Type:${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://${bucket}.s3.amazonaws.com/${file} 

Answer 1

Try not to sign the request yourself, a lot can go wrong or difficult to do. For example, you should check that the date is set to GMT or use the x-amz-date headers.

Another approach is to use the AWS command line interface , so use $ aws s3 cp or $ aws s3 sync .

Answer 2

I am writing this bash script to download a file from s3 (load a compressed file, you can change the contentType to load other types of files)

 #!/bin/sh outputFile="Your_PATH" amzFile="AMAZON_FILE_PATH" bucket="YOUR_BUCKET" resource="/${bucket}/${amzFile}" contentType="application/x-compressed-tar" dateValue=`date -R` stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}" s3Key="YOUR_S3_KEY" s3Secret="YOUR_S3SECRET" signature=`echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64` curl -H "Host: ${bucket}.s3.amazonaws.com" \ -H "Date: ${dateValue}" \ -H "Content-Type: ${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://${bucket}.s3.amazonaws.com/${amzFile} -o $outputFile 

Answer 3

 #!/bin/sh # This works for cross region outputFile="/PATH/TO/FILE" awsFile="BUCKETPATH/TO/FILE" bucket="SOME-BUCKET" resource="/${bucket}/${awsFile}" contentType="application/x-compressed-tar" # Change the content type as desired dateValue=`TZ=GMT date -R` #Use dateValue=`date -R` if your TZ is already GMT stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}" s3Key="ACCESS_KEY_ID" s3Secret="SECRET_ACCESS_KEY" signature=`echo -n ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64` curl -H "Host: ${bucket}.s3.amazonaws.com" \ -H "Date: ${dateValue}" \ -H "Content-Type: ${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://${bucket}.s3.amazonaws.com/${awsFile} -o $outputFile 

Answer 4

As of August 2019, I have found that this works. Added a region, and the URL format has changed.

 #!/bin/sh outputFile="/PATH/TO/LOCALLY/SAVED/FILE" amzFile="BUCKETPATH/TO/FILE" region="YOUR-REGION" bucket="SOME-BUCKET" resource="/${bucket}/${amzFile}" contentType="binary/octet-stream" dateValue='TZ=GMT date -R' # You can leave our "TZ=GMT" if your system is already GMT (but don't have to) stringToSign="GET\n\n${contentType}\n${dateValue}\n${resource}" s3Key="ACCESS_KEY_ID" s3Secret="SECRET_ACCESS_KEY" signature='echo -en ${stringToSign} | openssl sha1 -hmac ${s3Secret} -binary | base64' curl -H "Host: s3-${region}.amazonaws.com" \ -H "Date: ${dateValue}" \ -H "Content-Type: ${contentType}" \ -H "Authorization: AWS ${s3Key}:${signature}" \ https://s3-${region}.amazonaws.com/${bucket}/${amzFile} -o $outputFile