HTTP streaming on rails not working when using Rack :: Deflater

Question

I installed a unicorn on rails 3.1, and http streaming works until I turn on Rack :: Deflater. I tried using without using Rack :: Chunked. In curl, I see my answer, and in chrome I get the following error message: ERR_INVALID_CHUNKED_ENCODING

The result is the same in other browsers (firefox, safari) and between development (osx) and production (heroku).

config.ru:

require ::File.expand_path('../config/environment', __FILE__) use Rack::Chunked use Rack::Deflater run Site::Application 

unicorn.rb:

 listen 3001, :tcp_nopush => false worker_processes 1 # amount of unicorn workers to spin up timeout 30 # restarts workers that hang for 30 seconds 

controller:

 render "someview", :stream => true 

Thanks for any help.

Answer 1

The problem is that the Rails ActionController :: Streaming is displayed directly in the Chunked :: Body. This means that the content is first partitioned and then encrypted using the middleware Rack :: Deflater instead of gzipped and then placed.

According to HTTP / 1.1 RFC 6.2.1 , the packet should be the last encoding used for transmission.

Since "chunked" is the only transmission encoding that must be understood by HTTP / 1.1 recipients, it plays a crucial role in distinguishing between messages on a permanent connection. Whenever a transmission encoding is applied to a payload body in a request, the final transmission encoding applied must be “Fragmented”.

I fixed this for us by decapitating ActionController :: Streaming _process_options and _render_template methods in the initializer so that it would not wrap the body in Chunked :: Body and would not allow the use of the Rack :: Chunked middleware instead.

 module GzipStreaming def _process_options(options) stream = options[:stream] # delete the option to stop original implementation options.delete(:stream) super if stream && env["HTTP_VERSION"] != "HTTP/1.0" # Same as org implmenation except don't set the transfer-encoding header # The Rack::Chunked middleware will handle it headers["Cache-Control"] ||= "no-cache" headers.delete('Content-Length') options[:stream] = stream end end def _render_template(options) if options.delete(:stream) # Just render, don't wrap in a Chunked::Body, let # Rack::Chunked middleware handle it view_renderer.render_body(view_context, options) else super end end end module ActionController class Base include GzipStreaming end end 

And leave your config.ru as

 require ::File.expand_path('../config/environment', __FILE__) use Rack::Chunked use Rack::Deflater run Roam7::Application 

Not a very pleasant solution, it will probably break some other means that test / modify the body. If anyone has a better solution, I would like to hear it.

If you are using a new relic, its middleware should also be disabled when streaming .