Rake scrambling task apparently suffering from unwanted caching

Question

I'm at a dead end!

I have a rake task that runs every minute.

It logs in, it finds the JSON that interests me, but it can take up to 30 task launches before any changes to the JSON are noticed in the rake task. During this time, I missed a few changes to some JSON objects.

Some caching seems to be happening, I tried disabling Mechanize caching as shown, just not sure what else I can try now.

Any pointers?

Thanks in advance.

agent = Mechanize.new # {|a| a.log = Logger.new(STDERR) } agent.history.clear agent.max_history = 0 agent.user_agent_alias = 'Mac Safari' page = agent.get 'http://website.com' form = page.forms.first form.email = ' [e-mail] ' form.password = 'mypassword' page = agent.submit form page = agent.get 'http://website.com/password_protected_page' jsonDirty = page.search '//script[@type="application/json"]' 

The response from the server:

 {"server"=>"nginx", "date"=>"Thu, 13 Sep 2012 14:16:43 GMT", "content-type"=>"text/html; charset=utf-8", "connection"=>"close", "vary"=>"Cookie", "content-language"=>"plfplen", "set-cookie"=>"csrftoken=pVDg2SJ4KHqONz2OiEkNK7IbKlnJSQQf; expires=Thu, 12-Sep-2013 14:16:43 GMT; Max-Age=31449600; Path=/, affiliate=; expires=Thu, 01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/, one-click-join=; expires=Thu,01-Jan-1970 00:00:00 GMT; Max-Age=0; Path=/", "expires"=>"Thu, 01 Jan 1970 00:00:01 GMT", "cache-control"=>"no-cache", "content-encoding"=>"gzip", "transfer-encoding"=>"chunked"} 

Answer 1

You can try adding a random request parameter to the URL. For instance:

 page = agent.get "http://website.com/password_protected_page?random=#{Time.now.to_i}"