Ruby resque no load on rails

Question

I have a worker that works great, but is too slow. The main reason for this is that I use activerecord and should download the whole environment, which takes at least 10-20 seconds, just to download (I don’t constantly work with the current employee when I use Heroku and pay for the time when the worker runs ) I use resque worker to capture and analyze data from an external website, and then dump the data into my database.

My question is: should I rewrite the method so as not to use Rails and use DataMapper instead? Or something else that will load faster than activerecord.

Or if I have to extract code (using ActiveRecord) that determines what to do with external data, and bring it out of the work and back to the application?

Hope this makes sense.

Answer 1

I have the same problem. you can configure your environment on rake resque: setup rake task

I have tried this. assuming my rake resque task is on lib/tasks/resque.rake

 require "resque/tasks" task "resque:setup" do root_path = "#{File.dirname(__FILE__)}/../.." db_config = YAML::load(File.open(File.join(root_path,'config','database.yml')))["development"] ActiveRecord::Base.establish_connection(db_config) require "#{root_path}/app/workers/photo_downloader.rb" #workers #Dir.glob("#{root_path}/app/models/*").each { |r| puts r; require r } #require all model require "#{root_path}/app/models/photo.rb" # require model individually end 

I do not have complete success. I use the Paperclip gem, which requires a rail environment.

Answer 2

The Rails loading hopper is very slow; it must be started before a certain time for restart (to eliminate some memory leaks, most likely, any software is not an error) and is not intended to be used as a site that runs for a single request, and then turn it off.

This use is more like a script. If you need to run it using a browser, you can easily use something like Erubis to write the page and use ActiveRecord in the script (I think it can be used outside the rails) or a similar abstraction layer. Myself , for small tasks, I just use Mysql2 .

Answer 3

Use bundler to get active_record and another gem for you without the rails app.

 require 'rubygems' require 'logger' require 'active_record' require 'bundler' require "active_support" require "spreadsheet" require 'net/ping' require 'net/http' Bundler.setup Bundler.require(:default) if defined?(Bundler) $config_logger = Logger.new("./log/dev.log") class Dbconnect def initialize @settings = YAML.load_file('./config/database.yml')["development"] @adapter = @settings["adapter"] if @settings["adapter"] @database = @settings["database"] if @settings["database"] @pool = @settings["pool"] if @settings["pool"] @timeout = @settings["timeout"] if @settings["timeout"] end def connect_to_db ActiveRecord::Base.establish_connection( :adapter => @adapter, :database => @database, :reconnect => @reconnect, :pool => @pool, :timeout => @timeout) $config_logger.info "\n db Connected: to => #{@database} " end end end } Example Gemfile : source "http://rubygems.org" gem 'mail' gem "escape_utils" gem 'json',:require => "json" gem 'json_pure' gem 'resque' gem 'resque-scheduler' gem 'redis-namespace' gem 'resque-status' gem 'rake' gem 'em-udns' gem 'sqlite3' gem 'spreadsheet' gem 'activerecord', '3.2.1', :require => "active_record" gem 'net-scp', :require => 'net/scp' gem 'net-sftp', :require => 'net/sftp' gem 'net-ssh', :require => 'net/ssh' gem 'dir' gem 'amatch' gem 'haml' gem 'net-ping' gem install bundler 

the rest of the thing: installing the package.