The string is not valid UTF-8 (BSON :: InvalidStringEncoding) while saving the string compatible with UTF8 with MongoDB via Mongoid ORM

Question

I am importing data from a MySQL table into MongoDB using Mongoid for my ORM. When I try to save the email address as a string, I get an error. Error:

/Library/Ruby/Gems/1.8/gems/bson-1.2.4/lib/../lib/bson/bson_c.rb:24:in `serialize': String not valid UTF-8 (BSON::InvalidStringEncoding) from /Library/Ruby/Gems/1.8/gems/bson-1.2.4/lib/../lib/bson/bson_c.rb:24:in `serialize' 

From my GUI, this is a screenshot of a table. You can see that it is encoded in UTF8.

Also from my GUI is a screen shot of a field in my MySQL table that I import

This is what happens when I take data from the MySQL CLI.

And finally, when I check the data in my ruby ​​object, I get something similar to this:

I'm a little confused here, because no matter what my table is in UTF-8, and that the funky ones are apparently valid UTF-8 characters as double bytes. Does anyone know why I am getting this error?

Answer 1

Try using this helper:

http://snippets.dzone.com/posts/show/4527

Does he put the utf8 method? on the line. So you can grab a String from mysql and see if it is utf8:

 my_string.utf8? 

If this is not the case, you can try changing the encoding of your string using other methods, for example:

 my_string.asciify_utf8 my_string.latin1_to_utf8 my_string.cp1252_to_utf8 my_string.utf16le_to_utf8 

Perhaps this string is stored in mysql in one of these encodings.