Perl + Unicode: Wide Strings Error

Question

I am running Active Perl 5.14 on Windows 7. I am trying to write a program that will read the conversion table, then work with the file and replace some templates with other templates - all this is in Unicode (UTF-8). Here is the beginning of the program:

#!/usr/local/bin/perl # Load a conversion table from CONVTABLE to %ConvTable. # Then find matches in a file and convert them. use strict; use warnings; use Encode; use 5.014; use utf8; use autodie; use warnings qw< FATAL utf8 >; use open qw< :std :utf8 >; use charnames qw< :full >; use feature qw< unicode_strings >; my ($i,$j,$InputFile, $OutputFile,$word,$from,$to,$linetoprint); my (@line, @lineout); my %ConvTable; # Conversion hash print 'Conversion table: opening file: E:\My Documents\Perl\Conversion table.txt'."\n"; my $sta= open (CONVTABLE, "<:encoding(utf8)", 'E:\My Documents\Perl\Conversion table.txt'); binmode STDOUT, ':utf8'; # output should be in UTF-8 # Load conversion hash while (<CONVTABLE>) { chomp; print "$_\n"; # etc ... # etc ... 

It turns out that at this moment he says:

 wide character in print at (eval 155)E:/Active Perl/lib/Perl5DB.pl:640]line 2, <CONVTABLE> line 1, etc... 

Why? I think I went through and implemented all the necessary recipes for the correct processing of Unicode strings, decoding and encoding in UTF-8? And how to fix it?

TIA

Helen

Answer 1

The Perl debugger has its own output descriptor other than STDOUT (although it may end up moving to the same location as STDOUT ). You will also want to do something like this at the beginning of your script:

 binmode $DB::OUT, ':utf8' if $DB::OUT; 

Answer 2

I suspect that the problem is in some part of the code that you did not show us. I base this suspicion on the following facts:

• The error message you are reporting is indicated by at (eval 155) . There is no eval in your code.

• The code that you showed us above does not cause a “wide character” warning when you run it, even if the input contains Unicode characters. The only way I can do this is to comment out the use open line and the binmode STDOUT line.

Admittedly, my test environment is not exactly identical to yours: I'm on Linux and my Perl is only v5.10.1, which means I had to lower version requirements and disable unicode_strings (not that you actually use). However, I highly suspect that the problem is not with the code you posted.