Get the contents of the webpage page and the HTTP status code in C #

Question

In a C # Windows Forms application, I can get the contents of a webpage using:

string content = webClient.DownloadString(url); 

And I can get the HTTP header using:

 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); request.Method = "GET"; string response = ((HttpWebResponse)request.GetResponse()).StatusCode.ToString(); 

Is there a way to get both the content and the HTTP status code (if it fails) in one trip to the server instead of two?

Thanks.

Answer 1

You can read data from the stream inside the HttpWebResponse object:

 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); request.Method = "GET"; using (var response = request.GetResponse()) using (var stream = response.GetResponseStream()) using (var reader = new StreamReader(stream)) { HttpStatusCode statusCode = ((HttpWebResponse)response).StatusCode; string contents = reader.ReadToEnd(); } 

Thus, you will have to detect the encoding manually or use the library to detect the encoding. You can read the encoding as a string from the HttpWebResponse object, and when it exists, it is inside the ContentType property. If the page is Html, you will have to analyze it for a possible change in encoding at the top of the document or inside the head.

Reading encoding processing from ContentType header

 var request = (HttpWebRequest)WebRequest.Create(url); request.Method = "GET"; string content; HttpStatusCode statusCode; using (var response = request.GetResponse()) using (var stream = response.GetResponseStream()) { var contentType = response.ContentType; Encoding encoding = null; if (contentType != null) { var match = Regex.Match(contentType, @"(?<=charset\=).*"); if (match.Success) encoding = Encoding.GetEncoding(match.ToString()); } encoding = encoding ?? Encoding.UTF8; statusCode = ((HttpWebResponse)response).StatusCode; using (var reader = new StreamReader(stream, encoding)) content = reader.ReadToEnd(); } 

Answer 2

Webclient

I assume that you are using WebClient because it simplifies the processing of web requests. Unfortunately, WebClient does not provide an HTTP response code. You can assume that the answer was yes ( 2xx ) if you did not receive the exception and read it :

 try { string content = webClient.DownloadString(url); } catch (WebException e) { HttpWebResponse response = (System.Net.HttpWebResponse)we.Response; var statusCode = response.StatusCode; } 

Or, if you are really interested in the success code, you can use reflection, as described here .

---

Httpclient

You can also use HttpClient if you are using .NET 4.5, which displays the response code as described here :

 using (HttpClient client = new HttpClient()) { HttpResponseMessage response = await client.GetAsync(url); string content = await response.Content.ReadAsStringAsync(); var statusCode = response.StatusCode; } 

---

HttpWebRequest

Alternatively, you can simply use HttpWebRequest to get the status and response as described here :

 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url); request.Method = "GET"; var response = (HttpWebResponse)request.GetResponse(); using (Stream stream = response.GetResponseStream()) { StreamReader reader = new StreamReader(stream); string content = reader.ReadToEnd(); var statusCode = response.StatusCode; } 

Answer 3

I think you did not understand that in the second case you have access to the content (although it takes a bit more effort to get it as a string).

Check out the Microsoft documentation: http://msdn.microsoft.com/en-us/library/system.net.httpwebresponse.getresponsestream(v=vs.110).aspx , which shows how to create a response stream from a web response, and then how to get string data from this stream.

Answer 4

And I can get the HTTP header using: request.Method = "GET";

The GET method returns the response parts of HEAD and BODY. HTTP also supports the HEAD method, which returns only the HEAD section.

You can get BODY from HttpWebResponse using the GetResponseStream method .