Access all downloaded resources using Capybara or similar

Question

I am looking for an opportunity to easily access and list all the resources downloaded after loading a web page document: scripts, images, style sheets, etc., using a browser without a browser. I'm interested in the file url, status code and type, etc.

Think of a way to programmatically access information on the Network tab (developer tools):

Does anyone know a Ruby library to help me with this, or - even better - if this is a way to achieve this using Capybara (-webkit)?

---

Update

It seems that the Poltergeist has a method called network_traffic that does what Im after. Nevertheless, Haven managed to study it. I will report when I do this.

Answer 1

As mentioned in the update, this seems to be the way to do this with the Poltergeist (Capybara driver). Heres a quick and very "hacker" experiment:

 require 'rubygems' require 'capybara' require 'capybara/poltergeist' driver = Capybara::Poltergeist::Driver.new({}) port = Capybara::Poltergeist::Util.find_available_port server = Capybara::Poltergeist::Server.new(port, 30) client = Capybara::Poltergeist::Client.start(port, :path => driver.options[:phantomjs], :window_size => driver.options[:window_size], :phantomjs_options => driver.phantomjs_options ) browser = Capybara::Poltergeist::Browser.new(server, client, nil) browser.visit('http://www.google.com/') browser.network_traffic.each do |request| # sorry, quick and dirty to see what we get: request.response_parts.uniq(&:url).each do |response| puts "#{response.url}: #{response.status}" end end => http://www.google.com/: 200 http://ssl.gstatic.com/gb/images/b_8d5afc09.png: 200 http://www.google.com/images/srpr/logo1w.png: 200 http://www.google.com/images/srpr/nav_logo80.png: 200 http://www.google.com/xjs/_/js/hp/sb_he,pcc/rt=j/ver=FaiMBboaDLc.en_US./d=1/sv=1/rs=AItRSTMKxoHomLOW7ITf6OnfIEr5jQCEtA: 200 

This, however, is very slow and, of course, far from anything useful. I plan to go deeper into poltergeist to do the same at a lower level.

Answer 2

It seems strange that you need this information during the Capybara test. Good practice is to write your user interface tests to reflect the actual behavior of the user.

Consider a button that uses AJAX to update a block of text on a page. You can click the button, then check that the request has been completed and check the return value. But you'd better test it as a user: click the button, wait for the text block to change, and then confirm that it now displays the expected text.

If you really want to capture network traffic, I would install your test transparent HTTP proxy, connect to it and check the request logs after the fact.

My team uses a similar approach to simulate disconnecting from the Internet during Capybara tests. In the Firefox profile that we use, it is configured to point to a transparent proxy server that starts at the beginning of each function. Thus, we can write scripts like:

 Given I am online When I do something And I am offline Then something doesn't break 

... where the steps am online and am offline just turn proxies on and off.

Answer 3

Based on @polarblau's answer

You can set the debug checkpoint in the test code and run ...

 page.driver.network_traffic.each { |request| request.response_parts.uniq(&:url).each { |response| puts "#{response.url}: #{response.status}" }} 

The difference is that you do not need to launch a new browser and see what your page has loaded.