Python - loading plot from memory into s3 using matplotlib and boto

Question

This is my working script that generates the graph, saves it locally to disk, uploads to S3 and deletes the file:

plt.figure(figsize=(6,6))
plt.plot(x, y, 'bo')
plt.savefig('file_location')

conn = boto.s3.connect_to_region(
    region_name=AWS_REGION,
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    calling_format=boto.s3.connection.OrdinaryCallingFormat()
    )
bucket = conn.get_bucket('bucket_name')
k = Key(bucket)
k.key = 'file_name'
k.set_contents_from_filename('file_location')

os.remove(file_location)

I want to skip writing to disk and load the graph directly from memory.

Any suggestions for achieving it?

Answer 1

Putting it all together:

img_data = io.BytesIO()
plt.savefig(img_data, format='png')
img_data.seek(0)

s3 = boto3.resource('s3')
bucket = s3.Bucket(BUCKET_NAME)
bucket.put_object(Body=img_data, ContentType='image/png', Key=KEY)

Thanks @ padraic-cunningham and @ guyb7 for the tips!

Answer 2

import matplotlib.pyplot as plt
import numpy as np
from matplotlib.backends.backend_agg import FigureCanvasAgg
import boto3
import io

# some random plotting. We need the figure object later
fig, ax = plt.subplots(1,1,figsize=(6,6))
ax.plot(np.linspace(0,1,50),
        np.random.normal(0.5,0.5,50))


canvas = FigureCanvas(fig) # renders figure onto canvas
imdata = io.BytesIO() # prepares in-memory binary stream buffer (think of this as a txt file but purely in memory)
canvas.print_png(imdata) # writes canvas object as a png file to the buffer. You can also use print_jpg, alternatively

s3 = boto3.resource('s3',
                    aws_access_key_id='your access key id',
                    aws_secret_access_key='your secret access key',
                    region_name='us-east-1') # or whatever region your s3 is in

s3.Object('yourbucket','picture.png').put(Body=imdata.getvalue(),
                                          ContentType='image/png') 
# this makes a new object in the bucket and puts the file in the bucket
# ContentType parameter makes sure resulting object is of a 'image/png' type and not a downloadable 'binary/octet-stream'

s3.ObjectAcl('yourbucket','picture.png').put(ACL='public-read')
# include this last line if you find the url for the image to be inaccessible