Python and unicode bottle requests

Question

I am creating a small RESTful API with a bottle in python and am currently experiencing a problem with character encodings when working with a request object.

Pressing http://server.com/api?q=äöü and viewing request.query['q'] on the server calls me “äöü”, which is clearly not what I'm looking for.

The same applies to a POST request containing the urlencoded key q form with a äöü value. request.forms.get('q') contains "äöü".

What's going on here? I have no way to decode these elements with a different encoding or do? Is there a general option for storing bottles in Unicode?

Thanks.

Answer 1

request.query['q'] and forms.get('q') return the original byte value sent by the web browser. The äöü value sent by the browser as UTF-8 encoded bytes is '\xc3\xa4\xc3\xb6\xc3\xbc' .

If you print this byte string and the place you print to interpret it as ISO-8859-1 or a similar Windows 1252 code page, you will get äöü . If you are debugging by printing on the Windows command line or the file displayed by Notepad, then why.

If you use alternative direct access to request.query.q or forms.q , the Bottle instead produces Unicode strings decoded from the byte version using UTF-8. It is usually best to work with these Unicode strings wherever possible. (Although you may still have problems printing them on the console. The Windows command line, as you know, does a terrible job at non-ASCII characters, and as such is a bad place to debug Unicode problems.)