I have already done some research in this area, but have not found a solution. I have a website where ajax asynchronous calls are made to facebook (using JSONP). I am recording all my HTTP requests on the Ruby side with a VCR, so I thought it would be great to use this function for AJAX calls.
So, I played around a bit and came up with a proxy attempt. I use PhantomJS as a mute browser and poltergeist for integration inside Capybara. Poltergeist is now configured to use a proxy server as follows:
Capybara.register_driver :poltergeist_vcr do |app| options = { :phantomjs_options => [ "--proxy=127.0.0.1:9100", "--proxy-type=http", "--ignore-ssl-errors=yes", "--web-security=no" ], :inspector => true } Capybara::Poltergeist::Driver.new(app, options) end Capybara.javascript_driver = :poltergeist_vcr
For testing purposes, I wrote a proxy server based on WEbrick that integrates VCR:
require 'io/wait' require 'webrick' require 'webrick/httpproxy' require 'rubygems' require 'vcr' module WEBrick class VCRProxyServer < HTTPProxyServer def service(*args) VCR.use_cassette('proxied') { super(*args) } end end end VCR.configure do |c| c.stub_with :webmock c.cassette_library_dir = '.' c.default_cassette_options = { :record => :new_episodes } c.ignore_localhost = true end IP = '127.0.0.1' PORT = 9100 reader, writer = IO.pipe @pid = fork do reader.close $stderr = writer server = WEBrick::VCRProxyServer.new(:BindAddress => IP, :Port => PORT) trap('INT') { server.shutdown } server.start end raise 'VCR Proxy did not start in 10 seconds' unless reader.wait(10)
This works well with every localhost call, and they are well written. HTML, JS, and CSS files are recorded by the VCR. Then I turned on the c.ignore_localhost = true parameter, so it is useless (in my opinion) to record localhost calls.
Then I tried again, but I had to find out that the AJAX calls made on the page were not being recorded. Worse, they no longer work inside tests.
So, to answer this question, my question is: why are all the calls to the JS files on the local host recorded, and the JSONP calls to external ressources are not? This may not be a jsonP problem, because it is a “normal” ajax request. Or is there an error inside phantomjs that AJAX calls are not proxied? If so, how can we fix this?
If it works, I want to integrate the start and stop procedure inside
------- UPDATE -------
I did some research and came to the following point: the proxy has some problems with HTTPS calls and binary data via HTTPS calls.
I started the server and made some curls:
curl --proxy 127.0.0.1:9100 http://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png
This call is recorded as follows. Query and response output from the proxy server
GET http://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png HTTP/1.1 User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5 Host: d3jgo56a5b0my0.cloudfront.net Accept: */* Proxy-Connection: Keep-Alive HTTP/1.1 200 OK Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12) Date: Tue, 20 Nov 2012 10:13:10 GMT Content-Length: 0 Connection: Keep-Alive
But this call is not recorded, there should be a problem with HTTPS:
curl --proxy 127.0.0.1:9100 https://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png
Header output:
CONNECT d3jgo56a5b0my0.cloudfront.net:443 HTTP/1.1 Host: d3jgo56a5b0my0.cloudfront.net:443 User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5 Proxy-Connection: Keep-Alive HTTP/1.1 200 OK Server: WEBrick/1.3.1 (Ruby/1.9.3/2012-10-12) Date: Tue, 20 Nov 2012 10:15:48 GMT Content-Length: 0 Connection: close
So, I thought, maybe the proxy cannot handle HTTPS, but it can (as long as I get output to the console after calling cURL). Then I thought, maybe the VCR cannot mock HTTPS requests. But using this script, the VCR makes fun of HTTPS requests when I do not use it inside the proxy:
require 'vcr' VCR.configure do |c| c.hook_into :webmock c.cassette_library_dir = 'cassettes' end uri = URI("https://d3jgo56a5b0my0.cloudfront.net/images/v7/application/stories_view/icons/bug.png") VCR.use_cassette('https', :record => :new_episodes) do http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE response = http.request_get(uri.path) puts response.body end
So what is the problem? The VCR handles HTTPS and the proxy handles HTTPS. Why don't they play together?