Download media from Heroku to Amazon S3

Question

New to Heroku and Amazon S3, so carry me. We downloaded the Django application on Heroku and had problems loading custom media. Model below:

#models.py class Movie(models.Model): title = models.CharField(max_length = 500) poster = models.ImageField(upload_to = 'storages.backends.s3boto') pub_date = models.DateTimeField(auto_now_add = True) author = models.ForeignKey(User) 

The poster attribute is the one where the image is uploaded. It worked fine for me on the spot, and now an error occurred on Heroku. So I added "storages.backends.s3boto" as many other posts told me. (not sure what is right).

My Settings.py file looks like this right now, sort of a mess:

 #settings.py PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__)) PROJECT_DIR = os.path.join(PROJECT_ROOT, '../qanda') DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage' STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage' AWS_ACCESS_KEY_ID = '****************' AWS_SECRET_ACCESS_KEY = '************' AWS_STORAGE_BUCKET_NAME = 'mrt-assets' AWS_PRELOAD_METADATA = True MEDIA_ROOT = os.path.join(PROJECT_ROOT, 'qanda/media/movie_posters/) MEDIA_URL = '/media' STATIC_ROOT = os.path.join(PROJECT_ROOT, 'staticfiles') STATIC_URL = 'https://mrt-assets.s3.amazonaws.com/static/' STATICFILES_DIRS = (os.path.join(PROJECT_DIR, 'static'),) 

My bucket is called mrt-assets, and there are two folders there: static (css, js, images and media). At the moment, I'm not too worried about static files, since I hard-coded CSS / JS files to my HTML files *, but how can I get user-uploaded media (images of any kind) in mrt-assets / media?

*, although if someone wanted to help with STATIC files, that would be great. But the files uploaded by the user are more relevant.

EDIT (per Yuji comment):

We tried several options, and none of them worked. I came back and deleted a lot of changes, and now these are my settings

 #settings.py PROJECT_ROOT = os.path.abspath(os.path.dirname(__file__)) MEDIA_ROOT = 'http://s3.amazonaws.com/mrt-assets/media/' MEDIA_URL = '/media/' STATIC_ROOT = 'http://s3.amazonaws.com/mrt-assets/static/' STATIC_URL = '/static/' ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/' TEMPLATE_DIRS = (os.path.join(PROJECT_ROOT, "templates"),) #models.py #same as before, but now have changed the poster directory poster = models.ImageField(). 

I'm not sure what to do, I need to connect my Heroku application to S3 so that user downloads are saved in it.

Now I changed my S3 Bucket to this

 mrt-assets static css js images media (empty) 

Answer 1

The trick to loading your media into <bucket>/media and your static assets in <bucket>/static is to create two different default storage backends for two types of assets or to explicitly instantiate model fields with a storage object with location .

Creating an instance of a model field with custom storage

 from storages.backends.s3boto import S3BotoStorage class Movie(models.Model): title = models.CharField(max_length=500) poster = models.ImageField(storage=S3BotoStorage(location='media')) pub_date = models.DateTimeField(auto_now_add=True) author = models.ForeignKey(User) 

Providing S3BotoStorage a location will prefix all downloads with its path.

Create your own backups for media and static assets

This is almost the same as explicit storage definition with, but instead we will use settings.MEDIA_ROOT and settings.STATIC_ROOT apply the path prefix around the world.

 # settings.py STATIC_ROOT = '/static/' MEDIA_ROOT = '/media/' DEFAULT_FILE_STORAGE = 'app.storage.S3MediaStorage' STATICFILES_STORAGE = 'app.storage.S3StaticStorage' # app/storage.py from django.conf import settings from storages.backends.s3boto import S3BotoStorage class S3MediaStorage(S3BotoStorage): def __init__(self, **kwargs): kwargs['location'] = kwargs.get('location', settings.MEDIA_ROOT.replace('/', '')) super(S3MediaStorage, self).__init__(**kwargs) class S3StaticStorage(S3BotoStorage): def __init__(self, **kwargs): kwargs['location'] = kwargs.get('location', settings.STATIC_ROOT.replace('/', '')) super(S3StaticStorage, self).__init__(**kwargs) 

Cleaning

You can refine the code above to take advantage of the Heroku Configuration Repository to make it more portable:

 # settings.py import os STATIC_ROOT = os.environ.get('STATIC_ROOT', os.path.join(os.path.dirname(__file__), 'static')) MEDIA_ROOT = os.environ.get('MEDIA_ROOT', os.path.join(os.path.dirname(__file__), 'media')) DEFAULT_FILE_STORAGE = os.environ.get('DEFAULT_FILE_STORAGE', 'django.core.files.storage.FileSystemStorage') STATICFILES_STORAGE = os.environ.get('STATICFILES_STORAGE', 'django.contrib.staticfiles.storage.StaticFilesStorage') 

.env above settings with the .env file, and you can use the default storage repositories locally for development and testing, and when deploying to Heroku, you will automatically switch to app.storage.S3MediaStorage and app.storage.S3StaticStorage respectively:

 # .env STATIC_ROOT=static MEDIA_ROOT=media DEFAULT_FILE_STORAGE=app.storage.S3MediaStorage STATICFILES_STORAGE=app.storage.S3StaticStorage