Using Ruby quickcsv with character encoding

Question

Using Ruby 1.8.7, I want to accept csv into my system, although this is an admin application, it looks like I can get several different types of csvs. On my mac, if I export from excel using the "windows csv" option, then fastcsv can read it by default. On windows, I seem to get utf-16 encoded csvs (which I still don't understand how to understand)

It seems like a fairly common occurrence allowing users to download csv, which can be in utf8, utf16, ascii, etc. formats, to detect and analyze them. Has anyone figured this out?

I started looking for a UniversalDetector to help me identify and then use Iconv to convert, but it seems complicated and was hoping someone would figure it out :)

Answer 1

According to the FasterCSV Docs , the method initializeaccepts a parameter :encoding:

The encoding used to parse the file. The default value is $ KDOCE. Allowed values: n??? orN ??? for none, e??? orE ??? for EUC, s??? orS ??? for SJIS and u??? orU ??? for UTF-8 (see Regexp.new ()).

Since its list is limited, you may need to use iconvit to perform a preliminary content process and then transfer them to CSV. You can use the Ruby interface for iconv ("Iconv") or a command line version. Iconv is very powerful and flexible and capable of converting UTF-16 by the way.

, . , . , , , , ASCII, .

Ruby 1.9.2 , 1.8.7, , . , , .