Rails Version: 4.2.7
Ruby Version: 2.3.3
I am running a rake task to transfer several million objects from CSV and JSON format to my postgres database.
I tried to use activerecord-import to speed up writing objects to the database.
Having simplified the code as much as possible, the first half was processed with one type of object (which came from one data type), and the second with an object type of two.
The first type of object is repeated like this (simplified for the question):
importing_object_one_array = [] my_external_data.each do |element| new_element = ObjectOne.new( title: element[0], body: element[1] ) importing_object_one_array << new_element end ObjectOne.import importing_object_one_array, validate: false
This worked for approximately 250,000 objects and was written without any problems that I checked in the console, and the objects were successfully written.
However, the type of an object two has just a few additional objects, each about the same size and design, as the type of an object is one.
There are approximately 4,040,000 of them.
How long should I wait for ObjectTwo.import to complete? Now we have a watch.
Alternatively, in terms of debugging (since I would prefer not to repeat this rake command if I don't have to do this), which scripts or tactics will be useful to see if ObjectTwo.import is ObjectTwo.import at present (even if it takes forever) or if the task is hanging?
I checked the rails console and we are still in the same number of ObjectTwo in the database as before.
My only thought is that since I did not print the console before running #import (that is, it puts "Now starting import!" ), I have no 100% proof that the objects built in the array are finished.