Why does it display question marks in the message box instead of text

Question

Why question marks are displayed in the message box instead of text

HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://teamxor.net/vb/tx48/"+ page); HttpWebResponse response = (HttpWebResponse)request.GetResponse(); StreamReader sr = new StreamReader(response.GetResponseStream()); string result = sr.ReadToEnd(); Regex r = new Regex("<div>.*?</div>"); MatchCollection mr = r.Matches(result); foreach (Match m in mr) { MessageBox.Show(m.Value, "Test", MessageBoxButtons.OK, MessageBoxIcon.Information, MessageBoxDefaultButton.Button1, MessageBoxOptions.RtlReading); } 

Answer 1

The problem is using a custom code page. Your HTML shows that you are using code page 1256. You must tell .NET that, otherwise it thinks it is UTF-8:

 StreamReader sr = new StreamReader( response.GetResponseStream() , Encoding.GetEncoding(1256) // <-- this one ); 

Use Encoding.GetEncoding to get the correct code page. I suggest using UTF8 instead, as this is easily recognized by .NET.

Answer 2

Web servers can return a response in any encoding, although they usually choose an encoding that matches their preferred browser language.

The encoding used is returned as a charset element of the Content-Type header. In .NET, you can get the encoding used from the HttpWebResponse.CharacterSet property. You can use the returned encoding to create an Encoding object to use to read the response:

 var charset= response.CharacterSet; var encoding = Encoding.GetEncoding(charset); var sr= new StreamReader(response.GetResponseStream(),encoding); ....