Open S3 object as a string with Boto3

Question

I know that with Boto 2 you can open an S3 object as a string with:

get_contents_as_string () http://boto.readthedocs.org/en/latest/ref/file.html?highlight=contents%20string#boto.file.key.Key.get_contents_as_string

Is there an equivalent function in boto3?

Answer 1

read will return bytes. At least for Python 3, if you want to return a string, you must decode using the correct encoding:

 import boto3 s3 = boto3.resource('s3') obj = s3.Object(bucket, key) obj.get()['Body'].read().decode('utf-8') 

Answer 2

I had a problem reading the / .get() object from S3 due to .get() using Python 2.7 inside AWS Lambda.

I added json as an example to show that it became available for analysis :)

 import boto3 import json s3 = boto3.client('s3') obj = s3.get_object(Bucket=bucket, Key=key) j = json.loads(obj['Body'].read()) 

NOTE (for python 2.7): My object is completely ascii, so I don't need .decode('utf-8')

NOTE (for python 3. 6+): we switched to python 3.6 and found that read() now returns bytes so if you want to extract a string from it, you should use:

j = json.loads(obj['Body'].read().decode('utf-8'))

Answer 3

This is not in the boto3 documentation. This worked for me:

 object.get()["Body"].read() 

the object is an s3 object: http://boto3.readthedocs.org/en/latest/reference/services/s3.html#object

Answer 4

Python3 + Using the boto3 API approach.

Using the S3.Client.download_fileobj API and a file-like Python object , the contents of the S3 object can be retrieved into memory.

Since the content being retrieved is bytes, it needs to be decoded to convert to str .

 import io import boto3 client = boto3.client('s3') bytes_buffer = io.BytesIO() client.download_fileobj(Bucket=bucket_name, Key=object_key, Fileobj=bytes_buffer) byte_value = bytes_buffer.getvalue() str_value = byte_value.decode() #python3, default decoding is utf-8 

Answer 5

If the body contains io.StringIO, you must do the following:

 object.get()['Body'].getvalue()