Deferred work eliminates the queue

Question

I have a deferred job queue that contains particularly slow running tasks that I want to compromise with my own set of dedicated employees, so the risk will be the least difficult for the rest of the work pipeline.

RAILS_ENV=production script/delayed_job --queue=super_slow_stuff start

However, I also want the general pool of workers for all other queues, I hope, without the need to specify them separately (since their names, etc. often change / add). Something similar to:

RAILS_ENV=production script/delayed_job --except-queue=super_slow_stuff start

I could use the *charecter wildcard, but I suppose that would cause the second worker to also load super slow jobs too?

Any suggestions on this?

Answer 1

You can define a global constant for your application with all the queues.

QUEUES={
  mailers: 'mailers',
  etc..
}

then use this constant in your delay method methods

object.delay(queue: QUEUES[:mailers]).do_something

and try to build delayed_job_args dynamically

system("RAILS_ENV=production script/delayed_job --pool=super_slow_stuff --pool:#{(QUEUES.values-[super_slow_stuff]).join(',')}:number_of_workers start")

Answer 2

Unfortunately, this function is not implemented in deferred work.

See:

https://github.com/collectiveidea/delayed_job/pull/466

https://github.com/collectiveidea/delayed_job/pull/901

https://github.com/collectiveidea/delayed_job/pull/466.

GitHub, , , https://github.com/collectiveidea/delayed_job/pull/466, .

Update: . (exclude_queues): https://github.com/one-more-alex/delayed_job/tree/exclude_queues
https://github.com/one-more-alex/delayed_job_active_record/tree/exclude_queues

Readme.md

.

# Option --exclude-specified-queues will do inverse of queues processing by skipping onces from --queue, --queues.
# If both --pool=* --exclude-specified-queues given, no exclusions will by applied on "*".

If EXCLUDE_SPECIFIED_QUEUES set to YES, then queues defined by QUEUE, QUEUES will be skipped instead. See opton --exclude-specified-queues description for specal case of queue "*"

, :

RAILS_ENV=production script/delayed_job --queue=super_slow_stuff --exclude-specified-queues start

, , DelayedJobs " " , . https://github.com/collectiveidea/delayed_job/pull/1019

Active Record: https://github.com/collectiveidea/delayed_job_active_record/pull/151

ActiveRecord.