How to play UnicodeEncodeError?

Question

I get an error in the production system that I cannot reproduce in the development environment:

with io.open(file_name, 'wt') as fd:
    fd.write(data)

An exception:

  File "/home/.../foo.py", line 18, in foo
    fd.write(data)

UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 6400: ordinal not in range(128)

I already tried, but a lot of weird characters in a variable data.

But so far I have not been able to reproduce UnicodeEncodeError.

What should be in dataorder to receive UnicodeEncodeError?

Update

python -c 'import locale; print locale.getpreferredencoding()'
UTF-8

Update2

If I call locale.getpreferredencoding()through the shell and through the web request, the encoding will be "UTF-8".

I updated the exception handling in my code and registered getpreferredencoding()from some days. Now this happened again (still I can not force or reproduce it), and the encoding is "ANSI_X3.4-1968"!

I have no clue where this encoding is set ...

. . : ? .

, ,

Answer 1

; Unicode, , .

io.open() :

encoding - , . . ( locale.getpreferredencoding() ), , Python.

, locale.getpreferredencoding(), ASCII, ASCII , U-0080 .

, ; ASCII, , POSIX , C.

:

with io.open(file_name, 'wt', encoding='utf8') as fd:
    fd.write(data)

UTF-8 ; , , , .

Answer 2

, :

with open(filename, 'wt', encoding='ascii') as fd:
    fd.write('\xa0')

Traceback ( ):
"test.py", 2,     fd.write( '\ xa0')
UnicodeEncodeError: ascii '\ xa0' 0: (128)

Answer 3

Wrap writein try/ exceptand save the data in a binary file - you can find out exactly what data gives you:

with io.open(file_name, 'wt') as fd:
    try:
        fd.write(data)
    except UnicodeEncodeError:
        with open('/path/to/save/error.bin', 'wb') as err:
            err.write(data)
        raise