Manually enter the site with Typheous

Question

I recently used Mechanize for this kind of thing, but I want to use Typhoeus, which I already use everywhere. I want to imitate mechanizing behavior, the problem is that I would like to enter the site and fulfill the requests registered as a user. Here is a generic version of the script:

require 'rubygems' require 'typhoeus' GET_URL = 'http://localhost:3000' POST_URL = "http://localhost:3000/admins/sign_in" URL = "http://localhost:3000/dashboard" USERNAME_FIELD = 'admin[email]' PASSWORD_FIELD = 'admin[password]' USERNAME = " [e-mail] " PASSWORD = "my_secret_password" def merge_cookies_into_cookie_jar(response) if response.headers_hash['set-cookie'].instance_of? Array response.headers_hash['set-cookie'].each do |cookie| @cookie_jar << cookie.split('; ')[0] end elsif response.headers_hash['set-cookie'].instance_of? String @cookie_jar << response.headers_hash['set-cookie'].split('; ')[0] end end # initialize cookie jar @cookie_jar = [] # for server to establish me a session response = Typhoeus::Request.get( GET_URL, :follow_location => true ) merge_cookies_into_cookie_jar(response) # like submiting a log in form response = Typhoeus::Request.post( POST_URL, :params => { USERNAME_FIELD => USERNAME, PASSWORD_FIELD => PASSWORD }, :headers => { 'Cookie' => @cookie_jar.join('; ') } ) merge_cookies_into_cookie_jar(response) # the page I'd like to get in a first place, # but not working, redirects me back to login form with 401 Unauthorized :-( response = Typhoeus::Request.get( URL, :follow_location => true, :headers => { 'Cookie' => @cookie_jar.join('; ') } ) 

The cookie is sent to the server, but for some reason I have not logged in. I tested it on two different sites (one of which was my administration of Rails applications). Any idea what I'm doing wrong or perhaps a better or more widely applicable solution to this problem?

Answer 1

Here is the code that I can run successfully.

First of all, your cookie is an array, and in my code it should be an array with a replacement (or hash). When I run the code in a real application, the GET_URL response returns a session cookie, but the POST_URL response returns a different session cookie.

 # initialize cookie jar as a Hash @cookie_jar = {} 

Adjust the parsing so that you get each name and cookie value:

 def merge_cookies_into_cookie_jar(response) x = response.headers_hash['set-cookie'] case x ... when String x.split('; ').each{|cookie| key,value=cookie.split('=', 2) @cookie_jar[key]=value } end end 

The cookie box needs to be converted to a string:

 def cookie_jar_to_s @cookie_jar.to_a.map{|key, val| "#{key}=#{val}"}.join("; ") end 

Finally, change the headers to use the new cookie_jar_to_s:

 :headers => { 'Cookie' => cookie_jar_to_s } 

The bonus will be to make the cookie jar your class, maybe something like this:

 class CookieJar < Hash def to_s self.to_a.map{|key, val| "#{key}=#{val}"}.join("; ") end def parse(*cookie_strings) cookie_strings.each{|s| s.split('; ').each{|cookie| key,value=cookie.split('=', 2) self.[key]=value } } end end 

Answer 2

I fixed @joelparkerhenderson CookieJar (since it did not work here). Here is the result:

 class CookieJar < Hash def to_s self.map { |key, value| "#{key}=#{value}"}.join("; ") end def parse(cookie_strings) cookie_strings.each { |s| key, value = s.split('; ').first.split('=', 2) self[key] = value } self end end # Use like this: response = Typhoeus::Request.get("http://www.example.com") cookies = CookieJar.new.parse(response.headers_hash["Set-Cookie"]) Typhoeus::Request.get("http://www.example.com", headers: {Cookie: cookies.to_s}) 

Answer 3

Are your sites using Rails fake protection?

If so, when you get the form page, Rails sends a hidden field, which is the CSRF token.

In HTML, it will look something like this:

 <input type="hidden" name="csrf" value="abcdef"> 

You need to use this hidden value when submitting your form:

 :params => {"csrf" => "abcdef", USERNAME_FIELD => USERNAME, ... 

The hidden field tells Rails that you are the person who requested the form page, so you (and only you) are allowed to post.

Here are my CSRF notes with links to more information:

 http://sixarm.com/about/rails-session-csrf-token-jquery-ajaxprefilter.html 

Related CSRF StackOverflow Information:

 http://stackoverflow.com/questions/941594/understand-rails-authenticity-token