I recently worked on a Rails application where race conditions were causing issues in the background tasks operating on my ActiveRecord objects. In this article I will explain how I used pessimistic locking to get around these issues.Defining an example
Firstly, consider a sample Rails application with the following features/rules:Administrator can see a list of clients; Administrator can visit the client’s profile page and send an SMS to the client; Administrator cannot send more than one SMS to each client per day;
Let’s keep the implementation as simple as possible. Consider a Clientmodel with an attribute last_sms_sent_onandthe following class(which can be used by a controller or a Sidekiqtask):class ClientSMSSender def self.perform(client, message) client.transaction do if client.last_sms_sent_on.blank? || !client.last_sms_sent_on.today? client.last_sms_sent_on = Date.today client.save SMSGateway.send(client.phone_number, message) end end endend Analysing the race condition issue
Imagine that there are some external issues and SMSGateway.sendhangs for an average of 30 seconds. In this meantime another administrator makes a new request to send an SMS to the client. Due to race conditions we’ll end up sending more than one message to the client on the same day.
This is what will happen:Administrator Amakes a request ( Request A) to send an SMS to the client Clienthas not received any messages today Request Ais hanging due to external issues Administrator Bmake a new request ( Request B) to send the same SMS to the client Clienthas not received any messages today (the Request Astill hanging) Request Bis hanging due to external issues as well Request Afinishes and client.last_sms_sent_onis updated Request Bstill holding the previous state of the client object Request Bfinishes, send the SMS again and re-update client.last_sms_sent_on Work around
In order to work around that issue, you can make use of the method with_lockfrom ActiveRecord::Locking::Pessimistic.
Have a look at the code below:class ClientSMSSender def self.perform(client, message) client.with_lock do if client.last_sms_sent_on.blank? || !client.last_sms_sent_on.today? client.last_sms_sent_on = Date.today client.save SMSGateway.send(client.phone_number, message) end end endend
Under the hood, with_lock:opens up a database transaction reloads the record (in order to obtain the last state of the record) requests exclusive access to the record from the database
When using with_lock:Administrator Amakes a request ( Request A) to send an SMS to the client Request Alocks the client record in database Client has not received any messages today Request Ais hanging due to external issues Administrator Bmake a new request ( Request B) to send the same SMS to the client As the client is currently locked by Request A, the Request Bhangs until the database releases the record Request Afinishes and client.last_sms_sent_onis updated Database releases the client record Request B(was hanging and waiting for database) now starts the execution Request Blocks the client record Request Breloads the client record in order to obtain the latest state of the object Client has already received a message today Request Bfinishes without sending a new SMS.
If you’re still confusedabout that, here’s an easy way to see how with_lockworks from your Rails console:Grab any existing Rails project and open up twoRails consoles In the first console execute the following: u = User.firstu.with_lock do u.email = "[email protected]" u.save sleep 40 # emulates a very slow external processend In the second console execute: u = User.firstu.email = "[email protected]"u.save
You’ll notice in the second console that the execution of u.savewill hang until the first console finishes the whole process. You should becareful not to lock your entire app unnecessarily, otherwise you’re likely to introduce anew bottleneck.Conclusion
The method with_lockis handy, but use it sparingly. Sticking with_lockeverywhere might bring you good business logicconsistency, but it can come at the expense ofperformance.