How to deal with the encoding of the request parameter?

Question

I assumed that any data sent to the parameter strings would be utf-8, since this is what my entire site uses. Lowe with me, I was wrong.

For this example, it has a character äin utf-8 in the document (from the query string), but proceeds to send B\xe4ule(which is either ISO-8859-1 or windows 1252) when you click the "Submit" button. It also launches an ajax request , which also fails to decode the non-utf8 character.

An in django, my .POST request is really confusing:

>>> print request.POST
<QueryDict: {u'alias': [u'eu.wowarmory.com/character-sheet.xml?r=Der Rat von Dalaran&cn=B\ufffde']}>

How can I just remove all these headaches and work in utf8?

Answer 1

Since Django 1.0, all the values ​​that you get from the form view are unicode objects, not tags, as in Django 0.96 and earlier. To get utf-8 from your values, encode them using utf-8 codec:

request.POST['somefield'].encode('utf-8')

To get the correct query parameters, they must be correctly encoded:

In [3]: urllib.quote('ä')
Out[3]: '%C3%A4'

I think your problem is due to poor coding of the request parameters.

Answer 2

You should also add accept-charset="UTF-8"to the tag <form/>.

Answer 3

AFAIK , HTML-, . , , URL UTF-8, , HTML-, , UTF-8.

Answer 4

According to Get non-UTF-8-form fields like UTF-8 in PHP? , you need to make sure that the page itself is served using UTF8 Encoding.

Answer 5

Getting the utf-8 string from the submitted form should just be a matter of unicode coding:

next = request.POST ['next']. encode ('utf-8')

For an AJAX request, can you confirm that this request is also sent as utf-8 and declared as utf-8 in the headers?