Python 3 CGI: how to output raw bytes

Question

I decided to use Python 3 to build my site, but I ran into a Unicode exit problem.

It seems that plain print(html) #html is a str should work, but it doesn’t. I get UnicodeEncodeError: 'ascii' codec can't encode characters[...]: ordinal not in range(128) . This should be because the web server does not support Unicode output.

The next thing I tried was print(html.encode('utf-8')) , but I got something like the output of a byte string output: it is placed inside b'...' , and all the escape characters are in the original view (e.g. \n and \xd0\x9c )

Please show me the correct way to output a Unicode (str) string as an raw UTF-8 encoded byte string in Python 3.1

Answer 1

The problem is that you stdout are not tied to the actual terminal and use ASCII encoding by default. Therefore you need to write sys.stdout.buffer, which is the raw binary output of sys.stdout. This can be done in different ways, the most common seems:

 import codecs, sys writer = codecs.getwriter('utf8')(sys.stdout.buffer) 

And a writer. In a CGI script, you can replace sys.stdout with a record like this:

 sys.stdout = codecs.getwriter('utf8')(sys.stdout.buffer) 

Perhaps you are really working to print normally. Try it!