How to reduce the delay of data sent via REST api

Question

I have an application that receives JSON data from one of our other servers. The problem I am facing is a significant delay in requesting this information. Since a lot of data has been transferred (about 1000 records per request, where each record is quite huge), there is a way that compression would help reduce the speed. If so, what compression scheme would you recommend.

I read in another thread that their data pattern is also of great importance for the type of compression to use. The data structure is consistent and resembles the following

 :desc=>some_description
 :url=>some_url
 :content=>some_content
 :score=>some_score
 :more_attributes=>more_data

Can someone recommend a solution on how I could reduce this delay. The delay is about 6-8 seconds. I use Ruby on Rails to develop this application, and the server providing the data uses Python for the most part.

Answer 1

First, I'll see how much of this 8s delay is related to:

1. Server-side processing (how much was needed for the generated data) This time there are many methods for improvement, including:

   • DB Indices

   • caching

   • faster to_json library

NewRelic Rails http://railslab.newrelic.com/2009/02/09/episode-7-fragment-caching

1. ( )

   • , JSON- ? ; https://github.com/WebReflection/json.hpack/wiki/specs-details http://www.nwhite.net/?p=242

   • (gzip) http://httpd.apache.org/docs/2.0/mod/mod_deflate.html http://wiki.nginx.org/NginxHttpGzipModule

   • , , , , , , thrift: http://www.igvita.com/2007/11/30/ruby-web-services-with-facebooks-thrift/

   • , , / , ( )

Answer 2

gzip . YSlow.