How to use shell script to check if bucket exists?

Question

I have aws cli installed. I just don't know how to do this in a shell script.

when i run aws s3 ls s3://bucket , it will give me something like this

When a ListObjects operation was called, a client error (NoSuchBucket) occurred: the specified bucket does not exist

This means that the bucket does not exist. So I want to run this from a shell script and check if grep found. But my team is not working.

 if [ $(aws s3 ls "s3://$S3_BUCKET" | grep 'NoSuchBucket' &> /dev/null) == 0 ] then echo "$S3_BUCKET doesn\'t exist please check again" exit fi 

He just gave it to me

backup.sh: 20: [: 0: unexpected statement

Update

I changed the script to

 echo "S3_BUCKET=$S3_BUCKET" if aws s3 ls "s3://$S3_BUCKET" | grep -q 'AllAccessDisabled' then echo "$S3_BUCKET doesn\'t exist please check again" exit fi 

And this is the result that I got

 A client error (AllAccessDisabled) occurred when calling the ListObjects operation: All access to this object has been disabled 

So the text contains AllAccessDisabled , but I'm still not echo in the next line.

Answer 1

The code you provided would not give you this error.

If you wrote a script without a space between the leading [ and $( which will have.

Also grep is not going to print 0 in this case, so the test will not work the way you want.

If you want to check that grep has found something, you want to use the -q argument to grep as follows:

 if aws s3 ls "s3://$S3_BUCKET" 2>&1 | grep -q 'NoSuchBucket' then 

Answer 2

Head-bucket s3api is more direct and does not bear the cost of listing a bucket with a large number of files.

http://docs.aws.amazon.com/cli/latest/reference/s3api/head-bucket.html

 if aws s3api head-bucket --bucket "$S3_BUCKET" 2>/dev/null; then 

Answer 3

Here is how I did it, another answer will say that there is a bucket if there is an AWS error that does not contain "NoSuchBucket", for example, the token expiration date

 echo "Checking S3 bucket exists..." BUCKET_EXISTS=true S3_CHECK=$(aws s3 ls "s3://${BUCKET_NAME}" 2>&1) #Some sort of error happened with s3 check if [ $? != 0 ] then NO_BUCKET_CHECK=$(echo $S3_CHECK | grep -c 'NoSuchBucket') if [ $NO_BUCKET_CHECK = 1 ]; then echo "Bucket does not exist" BUCKET_EXISTS=false else echo "Error checking S3 Bucket" echo "$S3_CHECK" exit 1 fi else echo "Bucket exists" fi 

Answer 4

The recommended method for aws is aws s3api head-bucket --bucket $S3_BUCKET See https://docs.aws.amazon.com/cli/latest/reference/s3api/head-bucket.html.

Answer 5

it can give me the simplest approach

  if aws s3 ls "s3://$S3_BUCKET" 2>&1 | grep -q 'An error occurred' then echo "bucket does not exit or permission is not there to view it." else echo "bucket exit" fi 

Answer 6

I know this is an old question, but I came here for some answers and used some existing answers, and some of my own experiments came up with a script that processes various return values:

 bucketstatus=$(aws s3api head-bucket --bucket "${s3_bucket}" 2>&1) if echo "${bucketstatus}" | grep 'Not Found'; then echo "bucket does not exist"; elif echo "${bucketstatus}" | grep 'Forbidden'; then echo "Bucket exists but not owned" elif echo "${bucketstatus}" | grep 'Bad Request'; then echo "Bucket name specified is less than 3 or greater than 63 characters" else echo "Bucket owned and exists"; fi 

Answer 7

 #!/bin/bash if [[ -z $(aws s3api head-bucket --bucket my-bucket) ]]; then echo "bucket exists" else echo "bucket does not exist or permission is not there to view it." fi