Mutex for ActiveRecord

Question

The My User model has an unpleasant method that cannot be called simultaneously for two instances of the same record. I need to execute two HTTP requests per line and at the same time make sure that any other thread does not execute the same method for the same record at the same time.

class User ... def nasty_long_running_method // something nasty will happen if this method is called simultaneously // for two instances of the same record and the later one finishes http_request_1 // before the first one finishes http_request_2. http_request_1 // Takes 1-3 seconds. http_request_2 // Takes 1-3 seconds. update_model end end 

For example, this will break everything:

 user = User.first Thread.new { user.nasty_long_running_method } Thread.new { user.nasty_long_running_method } 

But that would be normal, and this should be allowed:

 user1 = User.find(1) user2 = User.find(2) Thread.new { user1.nasty_long_running_method } Thread.new { user2.nasty_long_running_method } 

What would be the best way to ensure that a method is not invoked simultaneously for two instances of the same record?

Answer 1

I found a pearl remote lock while finding a solution to my problem. This is a mutex solution that uses Redis in the backend.

It:

• available for all processes
• does not lock the database
• is in memory → fast and no IO

Now the method is as follows

 def nasty $lock = RemoteLock.new(RemoteLock::Adapters::Redis.new(REDIS)) $lock.synchronize("capi_lock_#{user_id}") do http_request_1 http_request_2 update_user end end 

Answer 2

I would start by adding a mutex or semaphore. Read about the mutec: http://www.ruby-doc.org/core-2.1.2/Mutex.html

 class User ... def nasty @semaphore ||= Mutex.new @semaphore.synchronize { # only one thread at a time can enter this block... } end end 

If your class is an ActiveRecord object, you can use Rails lock and database transactions. See: http://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html

 def nasty User.transaction do lock! ... save! end end 

---

Update:. You updated your question in more detail. And it looks like my solutions are no longer suitable. The first solutions do not work if you have multiple instances running. The second blocks only the database row; this does not prevent the simultaneous entry of several threads into the code block.

Therefore, if you think about creating a semaphore based on a database.

 class Semaphore < ActiveRecord::Base belongs_to :item, :polymorphic => true def self.get_lock(item, identifier) # may raise invalid key exception from unique key contraints in db create(:item => item) rescue false end def release destroy end end 

The database must have a unique index spanning rows for polymorphic association for an element. This should protect multiple threads from getting a lock for the same element at the same time. Your method will look like this:

 def nasty until semaphore semaphore = Semaphore.get_lock(user) end ... semaphore.release end 

There are several problems you can solve about this: how long do you want to wait for the semaphore? What happens if external HTTP requests take ages? Do you need to store additional pieces of information (host name, pid) to identify which thread is blocking the element? You will need some sort of cleanup task that will remove the locks that still exist after a certain period of time or after the server restarts.

Also, I think it is a terrible idea to have something like this on a web server. At the very least, you should transfer all this to background jobs. Which can solve your problem if your application is small and you only need one background task to get it done.

Answer 3

I suggest rethinking your architecture, because it will not be scalable - imagine that you have a lot of ruby ​​processes, process failures, timeouts, etc. Also, lock and spawning threads in the process are very dangerous for application servers.

If you want to sleep well with the products, try creating asynchronous background processing for long-running tasks with a sequential queue that will ensure the order in which tasks are completed. Just try RabbitMQ or check out this QA Best Practice for the Rails App to complete a long task in the background? , eventually try DB but Optimistic Locking.

Answer 4

You declare that this is an ActiveRecord model, in which case the usual approach would be to use a database lock for this record. As far as I know, there is no need for additional locking mechanisms.

Take a look at the short (one page) Rails Guides pessimistic blocking section - http://guides.rubyonrails.org/active_record_querying.html#pessimistic-locking

Basically, you can get a lock for one record or the whole table (if you update a lot of things)

In your case, something like this should do the trick ...

 class User < ActiveRecord::Base ... def nasty_long_running_method with_lock do // something nasty will happen if this method is called simultaneously // for two instances of the same record and the later one finishes http_request_1 // before the first one finishes http_request_2. http_request_1 // Takes 1-3 seconds. http_request_2 // Takes 1-3 seconds. update_model end end end