How to check if a specified key exists in a given S3 bucket using Java

Question

I would like to check if a key exists in this bucket using Java. I looked at the API, but there are no useful methods. I tried using getObject , but he made an exception.

Answer 1

Use the jets3t library. It is much simpler and more reliable than AWS sdk. Using this library, you can call s3service.getObjectDetails (). This will check and retrieve only the details of the object (and not the contents) of the object. It will call 404 if the object is missing. That way you can catch this exception and deal with it in your application.

But for this to work, you will need to have access to the ListBucket for the user in this bucket. Just accessing GetObject will not work. The reason is that Amazon will not allow you to check for a key if you do not have access to ListBucket. In some cases, it will be enough to know if there is a key or not, for attackers. Therefore, if they do not have access to ListBucket, they will not be able to do this.

Answer 2

Now, in the official Java API, doesObjectExist exists.

Enjoy it!

Answer 3

Update:

There seems to be a new API to check this out. See another answer on this page: stackoverflow.com/a/36653034/435605

Original message:

Use errorCode.equals("NoSuchKey")

 try { AmazonS3 s3 = new AmazonS3Client(new ClasspathPropertiesFileCredentialsProvider()); String bucketName = getBucketName(); s3.createBucket(bucketName); S3Object object = s3.getObject(bucketName, getKey()); } catch (AmazonServiceException e) { String errorCode = e.getErrorCode(); if (!errorCode.equals("NoSuchKey")) { throw e; } Logger.getLogger(getClass()).debug("No such key!!!", e); } 

Exception note: I know that exceptions should not be used to control flow. The problem is that Amazon did not provide an API to validate this stream - just the exception documentation.

Answer 4

Using the AWS SDK, use the getObjectMetadata method. The method will throw an AmazonServiceException if the key does not exist.

 private AmazonS3 s3; ... public boolean exists(String path, String name) { try { s3.getObjectMetadata(bucket, getS3Path(path) + name); } catch(AmazonServiceException e) { return false; } return true; } 

Answer 5

In the Amazon Java SDK 1.10+, you can use getStatusCode() to get the HTTP response status code, which will be 404 if the object does not exist.

 import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.model.AmazonS3Exception; import org.apache.http.HttpStatus; try { AmazonS3 s3 = new AmazonS3Client(); S3Object object = s3.getObjectMetadata("my-bucket", "my-client"); } catch (AmazonS3Exception e) { if (e.getStatusCode() == HttpStatus.SC_NOT_FOUND) { // bucket/key does not exist } else { throw e; } } 

getObjectMetadata() consumes less resources, and the answer does not need to be closed, like getObject() .

---

In previous versions, you can use getErrorCode() and check the corresponding line (depending on version).

Answer 6

Use the ListObjectsRequest Prefix parameter as the key.

.NET Code:

  public bool Exists(string key) { using (Amazon.S3.AmazonS3Client client = (Amazon.S3.AmazonS3Client)Amazon.AWSClientFactory.CreateAmazonS3Client(m_accessKey, m_accessSecret)) { ListObjectsRequest request = new ListObjectsRequest(); request.BucketName = m_bucketName; request.Prefix = key; using (ListObjectsResponse response = client.ListObjects(request)) { foreach (S3Object o in response.S3Objects) { if( o.Key == key ) return true; } return false; } } }. 

Answer 7

For PHP (I know the question is Java, but Google brought me here), you can use stream wrappers and file_exces

 $bucket = "MyBucket"; $key = "MyKey"; $s3 = Aws\S3\S3Client->factory([...]); $s3->registerStreamWrapper(); $keyExists = file_exists("s3://$bucket/$key"); 

Answer 8

This Java code checks to see if a key (file) exists in the s3 bucket.

 public static boolean isExistS3(String accessKey, String secretKey, String bucketName, String file) { // Amazon-s3 credentials AWSCredentials myCredentials = new BasicAWSCredentials(accessKey, secretKey); AmazonS3Client s3Client = new AmazonS3Client(myCredentials); ObjectListing objects = s3Client.listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix(file)); for (S3ObjectSummary objectSummary: objects.getObjectSummaries()) { if (objectSummary.getKey().equals(file)) { return true; } } return false; } 

Answer 9

Place your path in the bucket and object. Testing the bucket using the doesBucketExist method. Testing the object using the listing size (0 if absent). So this code will do:

 String bucket = ...; String objectInBucket = ...; AmazonS3 s3 = new AmazonS3Client(...); return s3.doesBucketExist(bucket) && !s3.listObjects(bucket, objectInBucket).getObjectSummaries().isEmpty(); 

Answer 10

Using Object isting. A Java function to check for the specified key in AWS S3.

 boolean isExist(String key) { ObjectListing objects = amazonS3.listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix(key)); for (S3ObjectSummary objectSummary : objects.getObjectSummaries()) { if (objectSummary.getKey().equals(key)) { return true; } } return false; } 

Answer 11

There is an easy way to do this using the jetS3t API isObjectInBucket () method.

Code example:

 ProviderCredentials awsCredentials = new AWSCredentials( awsaccessKey, awsSecretAcessKey); // REST implementation of S3Service RestS3Service restService = new RestS3Service(awsCredentials); // check whether file exists in bucket if (restService.isObjectInBucket(bucket, objectKey)) { //your logic } 

Answer 12

Alternatively, you can use Minio-Java , your Open Source, and AWS S3 compatible API.

You can use the Minio-Java StatObject.java examples for the same.

 import io.minio.MinioClient;
 import io.minio.errors.MinioException;

 import java.io.InputStream;
 import java.io.IOException;
 import java.security.NoSuchAlgorithmException;
 import java.security.InvalidKeyException;

 import org.xmlpull.v1.XmlPullParserException;


 public class GetObject {
   public static void main (String [] args)
     throws NoSuchAlgorithmException, IOException, InvalidKeyException, XmlPullParserException, MinioException {
     // Note: YOUR-ACCESSKEYID, YOUR-SECRETACCESSKEY and my-bucketname are
     // dummy values, please replace them with original values.
     // Set s3 endpoint, region is calculated automatically
     MinioClient s3Client = new MinioClient ("https://s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY");
     InputStream stream = s3Client.getObject ("my-bucketname", "my-objectname");

     byte [] buf = new byte [16384];
     int bytesRead;
     while ((bytesRead = stream.read (buf, 0, buf.length))> = 0) {
       System.out.println (new String (buf, 0, bytesRead));
     }

     stream.close ();
   }
 }

Hope this helps.

Disclaimer: I work for Minio

Answer 13

Use doBucketExist method for S3Client in AWS SDK

  static final AmazonS3 S3_CLIENT = AmazonS3ClientBuilder.defaultClient(); if (!S3_CLIENT.doesBucketExist(bucketName)) { CreateBucketRequest createBucketRequest = new CreateBucketRequest(bucketName, region); Bucket bucket = S3_CLIENT.createBucket(createBucketRequest); logger.log("Created Bucket Name" + bucket.getName() + " | Creation :" + bucket.getCreationDate()); }
 static final AmazonS3 S3_CLIENT = AmazonS3ClientBuilder.defaultClient(); if (!S3_CLIENT.doesBucketExist(bucketName)) { CreateBucketRequest createBucketRequest = new CreateBucketRequest(bucketName, region); Bucket bucket = S3_CLIENT.createBucket(createBucketRequest); logger.log("Created Bucket Name" + bucket.getName() + " | Creation :" + bucket.getCreationDate()); } 

Hope this helps.

Answer 14

Other answers for AWS SDK v1. Here is the method for AWS SDK v2 (currently 2.3.9).

Please note that getObjectMetadata and doesObjectExist are currently missing in the SDK v2! So these are no longer options. We are forced to use either getObject or listObjects .

listObjects is currently 12.5 times more expensive than getObject . But AWS also charges for any uploaded data, which increases the cost of getObject if the file exists. While the file is unlikely to exist (for example, you generated a new UUID key randomly, and you just need to double check that it was not taken), then calling getObject much cheaper by my calculations.

However, to be safe, I added the range() specification to ask AWS to send only a few bytes of the file. As far as I know, the SDK will always respect this and will not charge a fee for downloading the entire file. But I did not test this, so rely on this behavior at your own peril and risk! (Also, I'm not sure how range behaves if the S3 object is 0 bytes long.)

  private boolean sanityCheckNewS3Key(String bucket, String key) { ResponseInputStream<GetObjectResponse> resp = null; try { resp = s3client.getObject(GetObjectRequest.builder() .bucket(bucket) .key(key) .range("bytes=0-3") .build()); } catch (NoSuchKeyException e) { return false; } catch (AwsServiceException se) { throw se; } finally { if (resp != null) { try { resp.close(); } catch (IOException e) { log.warn("Exception while attempting to close S3 input stream", e); } } } return true; } } 

Note: this code assumes that s3Client and log declared and initialized elsewhere. The method returns a boolean, but may throw exceptions.

Answer 15

The correct way to do this in the V2 SDK without overloading the actual receipt of the object is to use S3Client.headObject . Officially supported by AWS Change Log .