A job handler selection error when running delayed_job during production using Thin or Unicorn

Question

I recently ran delayed_job in my Rails 3.1.3 application. In development Everything is fine. I even arranged to release my DJ on the same VPS as my production application using the same production application server (Thin), and everything was fine. However, as soon as I went into production, all hell broke down: none of the jobs were entered correctly in the jobs table, and I began to see the following in the logs for all processed jobs:

2012-02-18T14:41:51-0600: [Worker(delayed_job host:hope pid:12965)] NilClass# completed after 0.0151 2012-02-18T14:41:51-0600: [Worker(delayed_job host:hope pid:12965)] 1 jobs processed at 15.9666 j/s, 0 failed ... 

NilClass and method name? Of course not right. So I looked at the serialized handler in the job in the database and saw:

 "--- !ruby/object:Delayed::PerformableMethod\nattributes:\n id: 13\n event_id: 26\n name: memememe\n api_key: !!null \n" 

There is no class or method name. And when I load YAML into the object and call the # of the object on the resulting PerformableMethod, which I get zero. For kicks, I then launched the console on the broken product application and postponed the same work. This time the handler looked like this:

 "--- !ruby/object:Delayed::PerformableMethod\nobject: !ruby/ ActiveRecord:Domain\n attributes:\n id: 13\n event_id: 26\n name: memememe\n api_key: !!null \nmethod_name: :create_a\nargs: [] \n" 

And, of course, this work is working fine. Puzzled, I then remembered something about the DJ not playing well with Thin. So, I tried Unicorn and it was sad to see the same result. Hours of research later, and I think it has something to do with how the application server loads the YAML libraries Psych and Syck and DJ interaction with them. I cannot, however, state exactly what is wrong.

Please note that I run delayed_job 3.0.1, but tried to upgrade to the master branch and even tried to downgrade to 2.1.4. Here are some notable differences between my scene and production tunings:

• In step I start 1 thin server on a TCP port - there is no web proxy in front
• In production, I run 2+ Thin servers and proxy them with Nginx. They talk through a UNIX socket.
• When I tried the unicorn, it was 1 application server proxied by Nginx over the UNIX connector

Can web proxy / Nginx do something with it? Please, any understanding is greatly appreciated. I spent a lot of time integrating delayed_job and would not want to put off work or, even worse, drop it. Thanks for reading.

Answer 1

I fixed this without using #delay. Instead, I replaced all of my "model.delay.method" code with custom tasks. It works like a charm and, ultimately, more flexible. This fix works fine with Thin. I have not tested with the Unicorn.

Answer 2

I ran into a similar problem with rails 3.0.10 and dj 2.1.4, this is definitely another yaml library loaded when it starts from the console or from the application server; thin, unicorn, nginx. I will share the solution I came up with

Ok, so removing these lines from config / boot.rb fixed this problem for me.

 require 'yaml' YAML::ENGINE.yamler = 'syck' 

This was put there to fix the YAML parsing error by forcing YAML to use "syck". Removing this required me to fix the underlying problems with .yml files. Read more about it here.

Now my slow-motion recording handlers match between the processed server (the unicorn in my case) and the console. Both my server and deferred work workers start bundled

Unicorn

 cd #{rails_root} && bundle exec unicorn_rails -c #{rails_root}/config/unicorn.rb -E #{rails_env} -D" 

DJ

 export LANG=en_US.utf8; export GEM_HOME=/data/reception/current/vendor/bundle/ruby/1.9.1; cd #{rail 

s_root}; / usr / bin / ruby1.9.1 / data / reception / current / script / delayed_job start an internship