Parsing a CSV file into a Rails database

G'day guys

I am using quickCSV and rake script to analyze csv with approximately 30 columns in my db rails for the "Trade" element. The script works fine when all the values ​​are set to strings, but when I change it to decimal, int or another value, everything goes to hell.

It's amazing if fastCSV built parsing, etc., or I would have to manage them in my model.

In principle, I am given a huge amount of data about transactions, you need to import them, and then you need to provide feedback, say, the average volume of trade, time, etc. I understand that I can do everything with wonderful records provided to me by activeRecord, but I wondered if there is an easier way to populate a rather large database with this CSV?

Some of the fields do not have values ​​for specific lines, fastCSV seems to work fine when all the lines, but not when I try to get decimal or other information.

+3
source share
2 answers

Have you tried using AR Extensions for bulk import? You get impressive performance improvements when you insert 1000 rows into the database. Visit their website for more details.

See these examples for more details.

Case Study 1

Case Study 2

Case Study 3

+5
source

, MySQL, CSV LOAD DATA INFILE.

Ruby script, , - , , , .

+1

Source: https://habr.com/ru/post/1741714/


All Articles