Werkzeug reaction is too slow

Question

I have the following Werkzeug application to return a file to a client:

from werkzeug.wrappers import Request, Response @Request.application def application(request): fileObj = file(r'C:\test.pdf','rb') response = Response( response=fileObj.read() ) response.headers['content-type'] = 'application/pdf' return response 

I want to focus on this part:

 response = Response( response=fileObj.read() ) 

In this case, the response takes about 500 ms ( C:\test.pdf is a 4 MB file. The web server is on my local machine).

But if I rewrote this line:

 response = Response() response.response = fileObj 

Now the answer takes about 1500 ms. (3 times slower)

And if you write it like this:

 response = Response() response.response = fileObj.read() 

Now the answer takes about 80 seconds (right, 80 SECONDS).

Why is there such a big difference between the three methods?
And why is the third sooooo method slow?

Answer 1

After some testing, I think I figured out the mystery.

@Armin already explained why this ...

 response = Response() response.response = fileObj.read() 

... so slow. But this does not explain why this ...

 response = Response( response=fileObj.read() ) 

... so fast. They seem to be the same, but obviously this is not so. Otherwise there would not be that huge difference in speed.

The key here is in this part of the docs: http://werkzeug.pocoo.org/docs/wrappers/

The answer can be any iterable or string. If its string is considered considered, it is iterable with one element, which is the transmitted string.

i.e. when you give a string to a constructor, it is converted to iterable, with the string being only an element. But when you do this: response.response = fileObj.read() , the line is treated as.

To make it behave like a constructor, you must do this:

 response.response = [ fileObj.read() ] 

and now the file is sent as quickly as possible.

Answer 2

The answer to this question is quite simple:

• x.read() <- reads the entire file into memory, inefficient
• setting the response to the file: very inefficient, since the protocol for this object is an iterator. Therefore, you will send the file line by line. If it is binary, you will send it with random block sizes.
• setting response to a string: a bad idea. This is an iterator, as mentioned earlier, so you send each character in a string as a separate package.

The correct solution is to wrap the file in a file wrapper provided by the WSGI server:

 from werkzeug.wsgi import wrap_file return Response(wrap_file(environ, yourfile), direct_passthrough=True) 

The direct_passthrough flag direct_passthrough required so that the response object does not attempt to iterate through the file wrapper, but leaves it untouched for the WSGI server.

Answer 3

I can’t give you an exact answer on why this is happening, however http://werkzeug.pocoo.org/docs/wsgi/#werkzeug.wsgi.wrap_file can help solve your issue of blasting.