Reset connection when downloading (large) files

So, I have a rather nasty problem with uploading files to Pylons, and I hope someone can help me. I originally posted the problem here , thinking it was a problem with Pylons, but since then I have come to the conclusion that the problem is with the server, that is, with the paste and I tracked exactly where the problem is happening, but I'm not sure how to solve it . The behavior is as follows. I have a Pylons web application that I serve through a paster. One part of the application allows users to upload files to systems, and sometimes they can be quite large (> 50 MB). During such downloads, the server periodically resets the connection (i.e., in Chrome, the error reads "Error 101 (net :: ERR_CONNECTION_RESET): the connection was reset.".) This does not always happen, only about 50% of the time with large files.

In these cases, the server never passes POST to the application level. An exception is not thrown (at least I can find). I narrowed the problem down to cascade.py (specifically the wsgi_application method called with httpserver.py). From the last:

result = self.server.wsgi_application (self.wsgi_environ, self.wsgi_start_response)

In a function call to the cascade.py function, the file appears to be written to a temporary file. The corresponding cycle begins with:

while copy_len> 0:

He then writes the fragments one by one to an open file. None of these entries raise a separate exception. However, the server will often just β€œleave” during these records, seemingly at random (time does not seem to correlate with this). Therefore, if I print a copy_len size that decreases, I watch it decrease to one point, and then the server unexpectedly sends a β€œreset connection” (or at least the client thinks it does) and the method seems to be completed (copy_len stops printing as if the cycle was completed, although it was not 0).

I am completely at a dead end, as 1) it only happens sporadically, and 2) I cannot track the actual error / exception - it just ... dies.

If that matters, I run pastes 1.7.3, Python 2.7, and Fedora OS. Can anyone understand what might happen, and how can I solve this? Thank you so much for any help.

+6
source share
2 answers

Two ideas come to mind ...

  • Test with another more reliable web server, for example nginx . If necessary, it can be started simultaneously. See if the problem persists. Although paste can handle web applications, it is not designed to handle huge file transfers as far as I know.
  • Do you use Chunked Transfer Encoding at boot time?
+1
source

I have no problem using apache 2.2 + mod_python 3.3 on Ubuntu 10.04. I have successfully uploaded large files (400 MB).

I used uploader to create an html upload form.

0
source

Source: https://habr.com/ru/post/896328/


All Articles