All geek questions in one place

Ruby on Rails imports CSV into MySQL

Question with a newbie ....

Trying to run a project on rails. I have different tables in csv format that I would like to import into a MySQL database in order to be able to manipulate the data.

Looking back at stackoverflow, google etc. I wrote a rake requiring quickcsv to do this job. I keep getting errors, so I hope you can help.

... Okay, so I changed the code to use 'csv' vs 'fastcsv' ... still getting errors . See below

New code for rake file (do 3):

require 'csv' desc "Import gac from csv file" task :import => [:environment] do file = "gac.csv" CSV.foreach(file, :headers => true) do |row| Institution.create({ :institution_name => row[0], :website => row[1], :email => row[2], :category_1 => row[3], :category_2 => row[4], :category_3 => row[5], :category_4 => row[6], :category_5 => row[7], :category_6 => row[8], :category_7 => row[9], :category_8 => row[10], :category_9 => row[11], :category_10 => row[12], :category_11 => row[13], :institution_description => row[14] }) end end

Error Codes:

 Daves-MacBook-Pro:vendor dave$ rake import --trace ** Invoke import (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute import rake aborted! invalid byte sequence in UTF-8 /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `sub!' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `block in shift' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `loop' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `shift' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1791:in `each' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1208:in `block in foreach' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1354:in `open' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1207:in `foreach' /Users/dave/rails_projects/vendor/lib/tasks/import.rake:8:in `block in <top (required)>' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `call' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:205:in `block in execute' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `each' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:200:in `execute' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:158:in `block in invoke_with_call_chain' /Users/dave/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/monitor.rb:211:in `mon_synchronize' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:151:in `invoke_with_call_chain' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/task.rb:144:in `invoke' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:116:in `invoke_task' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block (2 levels) in top_level' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `each' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:94:in `block in top_level' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:88:in `top_level' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:66:in `block in run' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:133:in `standard_exception_handling' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/lib/rake/application.rb:63:in `run' /Users/dave/.rvm/gems/ruby-1.9.3-p0/gems/rake-0.9.2.2/bin/rake:33:in `<top (required)>' /Users/dave/.rvm/gems/ruby-1.9.3-p0/bin/rake:19:in `load' /Users/dave/.rvm/gems/ruby-1.9.3-p0/bin/rake:19:in `<main>' Tasks: TOP => import
+4
source share
3 answers

Make sure you have the Institution model class defined in app/models and / or make sure you migrated db ( rake db:migrate ) to create an institution table - provided that you used a generator (or scaffold) to create models.

+2
source

Lib / tasks / import.rake

 require 'csv' desc "Import gac from csv file" task :import => [:environment] do file = "vender/gac.csv" CSV.foreach(file, :headers => true) do |row| Putthemodelnamehere.create ({ :columnnamewhatever => row[1], :columnname => row[2], :columnname => row[4] }) end end

Then just start importing or merging the rake. Hope this helps (it does not use quickcsv, but it is a solution that I would recommend.)

+3
source

The errors you get are probably due to the fact that there is something incorrectly encoded for UTF-8 in your CSV code.

One way to deal with this is to force coding. You can do the following:

For each association you have above, do this instead:

 :category_1 => row[3].encode("UTF-8", replace: ' '),

Hope this works for you.

+2
source

Source: https://habr.com/ru/post/1391894/

More articles:


All Articles