Is IIS an illegal character swap? If so, how can this be stopped?

Question

Context: ASP.NET MVC running in IIS with a UTF-8% URL.

Using the standard project template and test action in HomeController , for example:

 public ActionResult Test(string id) { return Content(id, "text/plain"); } 

This is great for most%-coded UTF-8 routes, such as:

 http://mydevserver/Home/Test/%e4%ba%ac%e9%83%bd%e5%bc%81 

with expected result 京都 弁

However, using the route:

 http://mydevserver/Home/Test/%ee%93%bb 

URL received incorrectly.

In addition: %ee%93%bb is a% -coded code point 0xE4FB; basic multilingual aircraft; private sector; but ultimately the actual Unicode code point; You can check this manually or via:

 string value = ((char) 0xE4FB).ToString(); string encoded = HttpUtility.UrlEncode(value); // %ee%93%bb 

Now what comes next depends on the web server; the correct id received on the Visual Studio development server (aka cassini) - a line of length one containing the code point 0xE4FB.

If, however, I do this in IIS or IIS Express, I get another id , in particular "î"»" , code points: 0xEE, 0x201C, 0xBB. You will immediately recognize the first and last as the beginning and end of our percentage encoded string ... so what happened in the middle?

Well:

• code 0x93 " ( source )
• code point 0x201c is " ( source )

I really like that when I processed my URL, IIS did some kind of quotation translation. Now, perhaps this can be used in several scenarios (I don’t know), but it is, of course, bad when it happens in the middle of a block encoded with% -encoded UTF-8.

Please note that HttpContext.Current.Request.Raw also indicates that this translation has occurred, so this does not look like an MVC error; also note Darin’s comment, emphasizing that it works differently in the path to the URL request part.

So (two-part):

• Is there any important subtlety of handling unicode / url in my analysis?
• How can i fix this? (i.e. make sure I get the expected character)

Answer 1

Ultimately, to get around this, I had to use request.ServerVariables["HTTP_URL"] and some manual parsing with a bunch of errors when dealing with errors (additionally compensating for some related Uri crashes). Not large, but only affects a small minority of inconvenient requests.

Answer 2

 id = Encoding.UTF8.GetString(Encoding.Default.GetBytes(id)); 

This will give you the original id. IIS uses the default encoding (ANSI) for path characters. The string encoded by your url is decoded using this, and that is why you get the strange thing.

To get the original identifier, you can convert it back to bytes and get the string using utf8 encoding.

See Unicode and ISAPI Filters

ISAPI Filter is an ANSI API - all values ​​that you can get / set using the API must be ANSI. Yes, I know this is shocking; after all, this is 2006 and everything is currently in Unicode ... but remember that this API came about more than ten years ago when almost nothing was 32-bit, much smaller than Unicode. Also, remember that the HTTP protocol that ISAPI directly manipulates in ANSI, not Unicode.

EDIT: since you mentioned that it works with most other characters, so I assume IIS has some kind of encoding detection mechanism, which in this case does not work. As a workaround, although you can prefix your identifier with this char, and then you can easily detect if a problem has occurred (if this char is missing). Not a very perfect solution, but it will work. You can then write your own middleware and wrapper class in ASP.NET MVC to make your consumption code cleaner.

Answer 3

At one point, the URLs themselves were not in UTF-8. They were on the ANSI code page. This is facilitated by the fact that they are often used to select, well, paths in the server file system. In ancient times, IE was able to say whether you want to send UTF-8 URLs or not.

Perhaps buried in the bowels of the IIS configuration, there is a place where you can specify the URL encoding and maybe not.