NGinx & Django serving large files (3gb +)

Question

I'm having trouble uploading / downloading large files (3gb +).

Since I use Django, I assume that the problem for serving the file may be from Django or NGinx.

On my NGinx-enabled site, I have

server {
    ...
    client_max_body_size 4G;
    ...
}

And in django, I serve files in block sizes:

def return_file(path):
        filename = os.path.basename(path)
        chunk_size = 8192
        response = StreamingHttpResponse(FileWrapper(open(path), chunk_size), content_type=mimetypes.guess_type(path)[0])
        response['Content-Length'] = os.path.getsize(path)    
        response['Content-Disposition'] = 'attachment; filename={0}'.format(filename)
        return response

This method allowed me to switch from downloads from 600Mb ~ to 2.6Gb, but it seems that the download is truncated to 2.6Gb. I traced the error:

2015/09/04 11:31:30 [error] 831#0: *553 upstream prematurely closed connection while reading upstream, client: 127.0.0.1, server: localhost, request: "GET /chat/download/photorec.zip/ HTTP/1.1", upstream: "http://unix:/web/rsmweb/run/gunicorn.sock:/chat/download/photorec.zip/", host: "localhost", referrer: "http://localhost/chat/2/" 

After reading some posts, I added the following to my NGinx conf:

   proxy_read_timeout 300;
   proxy_connect_timeout 300;
   proxy_redirect off;

But I got the same error with *1instead*553*

I also thought it might be a Django database timeout, so I added:

DATABASE_OPTIONS = {
    'connect_timeout': 14400,
}

But it doesn’t work either. (loading through the development server takes about 30 seconds)

PS: - , Django, , . Django !

!

Answer 1

django , , . Nginx . , , - nginx:

location /static/ {
     try_files $uri =404 ;
     root /var/www/myapp/;
     gzip on;
     gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

}

/var/www/myapp/ django. static/, django manage.py collectstatic.

, . , nginx : http://nginx.org/en/docs/beginners_guide.html#static

Answer 2

, nginx :

<TIMESTAMP> [error] 1221#1221: *913310 upstream prematurely closed connection 
while reading upstream, client: <IP>, server: <IP>, request: "GET <URL> HTTP/1.1",
upstream: "http://unix:<LOCAL_DJANGO_APP_DIR_PATH>/run/gunicorn.sock:
<REL_PATH_LOCAL_FILE_TO_BE_DOWNLOADED>", host: "<URL>", referrer: "<URL>/<PAGE>"

--timeout

<LOCAL_DJANGO_APP_DIR_PATH>/bin/gunicorn_start 
(found at "command:" in /etc/supervisor/conf.d/<APPNAME>.conf)

:

exec /usr/local/bin/gunicorn [...] \
--timeout <OLD_TIMEOUT> \
[...]

300, 1280 ( !). ~ 5 ,

django.views.static.serve(request, <LOCAL_FILE_NAME>, <LOCAL_FILE_DIR>