How can I emulate a web browser http request from code?

Question

I am using C # in my WPF project. I want to send an HTTP GET request to a website, but I want to send it in such a way that it looks like a request from a browser.
Now I have a program that sends a GET request and receives a response. I am using the WebRequest to send GET requests.
I know that browsers add some information to their queries, such as browser name, OS name and computer name.
My question is: how can I add this information to my WebRequest ? What properties should all this information be assigned to (browser name, OS name)?

Answer 1

You must use Fiddler to grab the request you want to simulate. You need to look at inspectors> raw materials. This is an example of a request to the chrome fiddler site

 GET http://fiddler2.com/ HTTP/1.1 Host: fiddler2.com Connection: keep-alive Cache-Control: max-age=0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36 Referer: https://www.google.be/ Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8,nl;q=0.6 

You can then set each of these headers in your web request (see http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx ).

 WebRequest request = (HttpWebRequest)WebRequest.Create("http://www.test.com"); request.UserAgent = "Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.94 Safari/537.36"; 

Answer 2

Typically, the information you are interested in (browser, os, etc.) is sent in the "User Agent" header along with the request. You can manage the user agent with your property, here:

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.useragent.aspx

There may be other differences, I recommend using Fiddler to capture your browser traffic and then compare it with traffic with your .NET-based web request.

http://fiddler2.com/

Enjoy.

Answer 3

All such information is sent through the header in the web request. You can also add information to the header such as a key / value pair. However, you only have limited attributes that you can set using the WebRequest header property; many of them are limited. You can also check the list of attributes for the restricted header in the following thread: It is not possible to set some HTTP headers when using System.Net.WebRequest .