Registering in a website using C #

Question

So, I was browsing the web, trying to learn more about how to programmatically use websites using C #. I do not want to use the web client. I think I want to use something like HttpWebRequest and HttpWebResponse, but I have no idea how these classes work.

I assume that I am looking for someone to explain how they work, and the steps required to successfully log in, such as WordPress, an email account, or any site requiring you to fill out a form with a username and password.

Here is one of my attempts:

// Declare variables string url = textBoxGetSource.Text; string username = textBoxUsername.Text; string password = PasswordBoxPassword.Password; // Values for site login fields - username and password html ID's string loginUsernameID = textBoxUsernameID.Text; string loginPasswordID = textBoxPasswordID.Text; string loginSubmitID = textBoxSubmitID.Text; // Connection parameters string method = "POST"; string contentType = @"application/x-www-form-urlencoded"; string loginString = loginUsernameID + "=" + username + "&" + loginPasswordID + "=" + password + "&" + loginSubmitID; CookieContainer cookieJar = new CookieContainer(); HttpWebRequest request; request = (HttpWebRequest)WebRequest.Create(url); request.CookieContainer = cookieJar; request.Method = method; request.ContentType = contentType; request.KeepAlive = true; using (Stream requestStream = request.GetRequestStream()) using (StreamWriter writer = new StreamWriter(requestStream)) { writer.Write(loginString, username, password); } using (var responseStream = request.GetResponse().GetResponseStream()) using (var reader = new StreamReader(responseStream)) { var result = reader.ReadToEnd(); Console.WriteLine(result); richTextBoxSource.AppendText(result); } MessageBox.Show("Successfully logged in."); 

I do not know if I am on the right track or not. I end up going back to the login screen, wherever I try. I downloaded Fiddler and was able to collect some information about what information is being sent to the server, but I feel completely lost. If anyone could shed light here, I would really appreciate it.

Answer 1

Logging into websites is programmatically difficult and closely related to how the site performs its login process. The reason your code does not work is because you are not dealing with this in your requests and responses.

Take fif.com , for example. When you enter your username and password, the following email request is sent:

 POST https://fif.com/login?task=user.login HTTP/1.1 Host: fif.com Connection: keep-alive Content-Length: 114 Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Origin: https://fif.com User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36 Content-Type: application/x-www-form-urlencoded Referer: https://fif.com/login?return=...== Accept-Encoding: gzip,deflate Accept-Language: en-US,en;q=0.8 Cookie: 34f8f7f621b2b411508c0fd39b2adbb2=gnsbq7hcm3c02aa4sb11h5c87f171mh3; __utma=175527093.69718440.1410315941.1410315941.1410315941.1; __utmb=175527093.12.10.1410315941; __utmc=175527093; __utmz=175527093.1410315941.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=175527093.|1=RegisteredUsers=Yes=1 username=...&password=...&return=aHR0cHM6Ly9maWYuY29tLw%3D%3D&9a9bd5b68a7a9e5c3b06ccd9b946ebf9=1 

Pay attention to cookies (especially the first one, your session token). Please note that the sent uppercase URL is returned. If the server notifies you, it will not allow you to log in.

 HTTP/1.1 400 Bad Request 

Or, even worse, 200 responses to the login page with an error message, somewhere inside.

But let’s just pretend that you were able to collect all these magic values ​​and pass them to the HttpWebRequest object. The site would not know the difference. And he can answer something like this.

 HTTP/1.1 303 See other Server: nginx Date: Wed, 10 Sep 2014 02:29:09 GMT Content-Type: text/html; charset=utf-8 Transfer-Encoding: chunked Connection: keep-alive Location: https://fif.com/ 

I hope you were expecting this. But if you have made it this far, now you can programmatically disable server requests using your trusted session token and return the expected HTML.

 GET https://fif.com/ HTTP/1.1 Host: fif.com Connection: keep-alive Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.103 Safari/537.36 Referer: https://fif.com/login?return=aHR0cHM6Ly9maWYuY29tLw== Accept-Encoding: gzip,deflate Accept-Language: en-US,en;q=0.8 Cookie: 34f8f7f621b2b411508c0fd39b2adbb2=gnsbq7hcm3c02aa4sb11h5c87f171mh3; __utma=175527093.69718440.1410315941.1410315941.1410315941.1; __utmb=175527093.12.10.1410315941; __utmc=175527093; __utmz=175527093.1410315941.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __utmv=175527093.|1=RegisteredUsers=Yes=1 

And that's all for fif.com - this juggling of cookies and tokens and redirects will be completely different for another site. In my experience (with this site in particular) you have three options to get through the entrance wall.

• Write an incredibly complex and fragile script to dance around site procedures.
• Manually enter the site with your browser, take the magic values ​​and connect them to the request objects or
• Create a script to automate selenium to do this for you.

Selenium can handle all the juggling, and in the end you can pull out cookies and turn off your requests normally. Here is an example for fif:

 //Run selenium ChromeDriver cd = new ChromeDriver(@"chromedriver_win32"); cd.Url = @"https://fif.com/login"; cd.Navigate(); IWebElement e = cd.FindElementById("username"); e.SendKeys("..."); e = cd.FindElementById("password"); e.SendKeys("..."); e = cd.FindElementByXPath(@"//*[@id=""main""]/div/div/div[2]/table/tbody/tr/td[1]/div/form/fieldset/table/tbody/tr[6]/td/button"); e.Click(); //Get the cookies foreach(OpenQA.Selenium.Cookie c in cd.Manage().Cookies.AllCookies) { string name = c.Name; string value = c.Value; cc.Add(new System.Net.Cookie(name,value,c.Path,c.Domain)); } //Fire off the request HttpWebRequest hwr = (HttpWebRequest) HttpWebRequest.Create("https://fif.com/components/com_fif/tools/capacity/values/"); hwr.CookieContainer = cc; hwr.Method = "POST"; hwr.ContentType = "application/x-www-form-urlencoded"; StreamWriter swr = new StreamWriter(hwr.GetRequestStream()); swr.Write("feeds=35"); swr.Close(); WebResponse wr = hwr.GetResponse(); string s = new System.IO.StreamReader(wr.GetResponseStream()).ReadToEnd(); 

Answer 2

Checkout . This is another way to do this, and you do not need to install any package, although it can be easier with Selenium.

"You can continue to use WebClient for POST (instead of GET, which is the HTTP verb that you use with DownloadString), but I think it will be easier for you to work with the (slightly) lower level WebRequest and WebResponse classes.

There are two parts to this: the first is to publish the login form, the second is to restore the “Set-cookie” header and send it back to the server as a “Cookie” along with your GET request. The server will use this cookie to identify you from now on (assuming it uses cookie-based authentication, which I'm pretty sure of that. The page returns a Set-cookie header that includes "PHPSESSID").

---

POSTing in the login form

Message forms are easy to simulate, this is just a case of formatting yours, send the data as follows:

 field1=value1&field2=value2 

Using WebRequest and the code that I adapted from Scott Hanselman , here is how you could generate the data in the login form:

 string formUrl = "http://www.mmoinn.com/index.do?PageModule=UsersAction&Action=UsersLogin"; 

NOTE: This is the URL of the POST form, not the URL of the form (you can find this in the action attribute of the HTML form tag

 string formParams = string.Format("email_address={0}&password={1}", "your email", "your password"); string cookieHeader; WebRequest req = WebRequest.Create(formUrl); req.ContentType = "application/x-www-form-urlencoded"; req.Method = "POST"; byte[] bytes = Encoding.ASCII.GetBytes(formParams); req.ContentLength = bytes.Length; using (Stream os = req.GetRequestStream()) { os.Write(bytes, 0, bytes.Length); } WebResponse resp = req.GetResponse(); cookieHeader = resp.Headers["Set-cookie"]; 

Here is an example of what you should see in the Set-cookie header for your login form:

 PHPSESSID=c4812cffcf2c45e0357a5a93c137642e; path=/; domain=.mmoinn.com,wowmine_referer=directenter; path=/; 

domain = .mmoinn.com, languages ​​= a; Path = /; domain = .mmoinn.com, adt_usertype = other, adt_host = -

---

Getting page after login form

Now you can fulfill your GET request to the page that you need is logged in.

 string pageSource; string getUrl = "the url of the page behind the login"; WebRequest getRequest = WebRequest.Create(getUrl); getRequest.Headers.Add("Cookie", cookieHeader); WebResponse getResponse = getRequest.GetResponse(); using (StreamReader sr = new StreamReader(getResponse.GetResponseStream())) { pageSource = sr.ReadToEnd(); } 

EDIT:

If you need to view the results of the first POST, you can restore the HTML it returned with:

 using (StreamReader sr = new StreamReader(resp.GetResponseStream())) { pageSource = sr.ReadToEnd(); } 

Place this directly under cookieHeader = resp.Headers["Set-cookie"]; and then check the line contained in the pageSource file. "