Rails

Question

I use resque stone to handle my background process.

I set up three queues with one worker each.

Can anyone explain how memory is consumed with an increase and decrease in the number of workers in the queue.

I heard that every worker loads a separate Rails environment. It's true?

Answer 1

As far as I know, yes, each worker launches a separate Rails environment, so if you have three employees, you will download three Rails environments. Having more workers will not make your turn grow longer, it will be the number of jobs standing in line that will grow.

In any case, if you have no main reason to have 3 separate employees, I suggest you have only one worker for all the queues and divide them as your application scales with time.

Answer 2

It is not true. As I explained in my comment here, the exact goal of Resque is not to download a rail environment for each worker (see https://github.com/blog/542-introducing-resque ).

Due to the limitation caused by the Ruby Green solution, you must run at least one worker on the processor core in order to be able to use the entire processor. This is why Resque's default behavior starts each worker in a separate process. This means starting N parallel processes at the same time, each of which with a full set of gems is loaded independently. This is the main reason for using high memory for Resque and any other Ruby tool. You can see here how things can get better using native streaming using the JVM.

So, if you want your use of Resque memory to reduce your working dependencies as low as possible. And it is always very important to use a monitoring tool, such as God or Blumill, to monitor the process.