Django as an S3 proxy

Question

I expanded ModelAdmin with a custom "Upload file" field, which is a link to a URL in my Django project, for example:

http://www.myproject.com/downloads/1 

There I want to serve a file that is stored in an S3 bucket. Files in the bucket are not publicly accessible, and the user may not have direct access to it. Now I want

• avoid downloading the file to the server’s memory (these are files with several gb).
• avoid temporary files on the server

An ideal solution would be to allow django to act as a proxy server that passes S3 chunks directly to the user. I am using boto but have not found a way to stream pieces. Any ideas?

Thanks.

Answer 1

Instead of proxying, why not just redirect?

Use the django view at www.myproject.com/downloads/1 to handle HTTP redirection to the S3 repository URL — it can generate time-limited URLs, for example. see here http://docs.amazonwebservices.com/AmazonS3/2006-03-01/dev/

The client then downloads the file directly from S3, but the content is still safe and access must go through the Django application.