How to exit an asynchronous call when timeout url with ruby ​​/ curb

Question

I use a Ruby limiter to call multiple URLs at once, like

require 'rubygems'
require 'curb'

easy_options = {:follow_location => true}
multi_options = {:pipeline => true}

Curl::Multi.get(['http://www.example.com','http://www.trello.com','http://www.facebook.com','http://www.yahoo.com','http://www.msn.com'], easy_options, multi_options) do|easy|
  # do something interesting with the easy response
  puts easy.last_effective_url
end

I have a problem: I want to break subsequent asynchronous calls when any url timeout occurred, is this possible?

Answer 1

As far as I know, the current API does not provide an instance of Curl :: Multi, since otherwise you could do:

stop_everything = proc { multi.cancel! }
multi = Curl::Multi.get(array_of_urls, on_failure: stop_everything)

The easiest way is to fix Curl :: Multi.http to return the m variable.

See https://github.com/taf2/curb/blob/master/lib/curl/multi.rb#L85

Answer 2

I think this will do exactly what you ask for:

require 'rubygems'
require 'curb'

responses = {}
requests = ['http://www.example.com','http://www.trello.com','http://www.facebook.com','http://www.yahoo.com','http://www.msn.com']
m = Curl::Multi.new
requests.each do |url|
  responses[url] = ""
    c = Curl::Easy.new(url) do|curl|
    curl.follow_location = true
    curl.on_body{|data| responses[url] << data; data.size }
    curl.on_success {|easy| puts easy.last_effective_url }
    curl.on_failure {|easy| puts "ERROR:#{easy.last_effective_url}"; @should_stop = true}
  end
  m.add(c)
end

m.perform { m.cancel! if @should_stop }