Ensuring data consistency in distributed systems has always been a headache. When multiple services try to access the same resource simultaneously, it can lead to conflicts and inconsistent data. This is where distributed locks come in; however, choosing the right locking mechanism often goes beyond a technical preference, becoming a pragmatic decision that varies based on the application’s workload, fault tolerance, and even budget.
In my twenty years of experience, I’ve wrestled with distributed locks in many different scenarios, from a simple UPDATE query to complex stock movements in a production ERP. Here, I’ll share these different alternatives, my experiences with them, what I chose in which situations, and why.
Introduction: Why Do We Need Distributed Locks?
In distributed systems, we need distributed locks to prevent multiple processes or services from simultaneously accessing a shared resource (a file, a database record, inventory information). These locks provide a singular control mechanism over the resource, preventing data corruption or unexpected situations. For example, when withdrawing money from a user’s balance, we need to prevent two different transactions from simultaneously debiting the balance.
I first encountered this while updating order statuses on an e-commerce site. When both payment confirmation and shipping preparation for the same order were triggered simultaneously, the order status was updated multiple times, leading to inconsistency. To solve this, I had to resort to a simple database locking mechanism.
Database Locks: A Reliable But Costly Option
Databases are a natural candidate for distributed locks. Thanks to the atomic nature of transactions and built-in locking mechanisms, ensuring data consistency is relatively easy. Especially in powerful databases like PostgreSQL, both row-level locks (SELECT FOR UPDATE) and advisory locks (pg_advisory_lock) can be used.
In a production ERP, using SELECT FOR UPDATE was indispensable for me when processing stock movements. When updating a product’s stock quantity, I needed to prevent another process from simultaneously reading that stock and making an incorrect decision. While this could lead to performance bottlenecks, especially in high-volume transactions, it was a cost worth paying when data consistency was critical.
BEGIN;
SELECT stock_quantity FROM products WHERE product_id = 123 FOR UPDATE;
-- stock_quantity'yi oku ve yeni değeri hesapla
UPDATE products SET stock_quantity = new_quantity WHERE product_id = 123;
COMMIT;pg_advisory_lock, on the other hand, provides a lighter lock that can be managed at the application level. In one of my side projects, I used pg_advisory_lock to ensure that a specific background task was run by only one instance. Since these locks are not tied to database rows, they reduce the risk of deadlocks and offer more flexible usage. However, ensuring that locks are released correctly is entirely the developer’s responsibility.
Redis Locks: Speed and Considerations
Redis, thanks to its in-memory structure, offers a very fast lock, and implementing distributed locks with the SETNX (SET if Not eXists) command is quite common. To acquire a lock, you write a specific key to Redis with a certain duration (TTL - Time To Live). If the key already exists, the lock cannot be acquired.
In a task management application I developed, I used Redis locks to prevent users from triggering the same task multiple times. When a user clicked the start button for a task, I would acquire a Redis lock with the task’s ID and release it when the operation was complete. This prevented the same task from running multiple times in the background.
import redis
import uuid
r = redis.Redis(host='localhost', port=6379, db=0)
def acquire_lock(lock_name, acquire_timeout=10, lock_timeout=10):
identifier = str(uuid.uuid4())
end = time.time() + acquire_timeout
while time.time() < end:
if r.set(lock_name, identifier, ex=lock_timeout, nx=True):
return identifier
time.sleep(0.001)
return False
def release_lock(lock_name, identifier):
pipe = r.pipeline(True)
while True:
try:
pipe.watch(lock_name)
if pipe.get(lock_name).decode('utf-8') == identifier:
pipe.multi()
pipe.delete(lock_name)
pipe.execute()
return True
pipe.unwatch()
break
except redis.exceptions.WatchError:
pass
return FalseHowever, Redis locks have some weaknesses. While the Redlock algorithm was proposed to address these weaknesses, issues can arise in network partition scenarios or when a Redis instance crashes and restarts. Last year, I observed some locks being released prematurely when my Redis instance on my own VPS was OOM-killed. Therefore, it’s necessary to set the eviction policy to noeviction and pay attention to Redis’s memory limits.
Simple File Locks and Other Local Solutions
For applications running on a single server without distributed systems, file locks or simple operating system tools like mkdir might suffice. The flock command provides singular access to a file, preventing multiple processes from writing to the same file simultaneously.
Once upon a time, I had a script that ran as a cron job and generated specific reports. To prevent this script from running twice simultaneously, I used a simple file lock. When the script started, it would try to create a file named /tmp/rapor_uret.lock; if the file already existed, it would exit. Such a simple solution was perfectly adequate for that scenario.
#!/bin/bash
LOCK_FILE="/tmp/my_script.lock"
# Kilidi almaya çalış
if ( set -o noclobber; echo "$$" > "$LOCK_FILE") 2> /dev/null; then
trap 'rm -f "$LOCK_FILE"; exit $?' INT TERM EXIT
echo "Script çalışıyor, kilit alındı."
# Gerçek script mantığı buraya gelir
sleep 30
echo "Script bitti."
rm -f "$LOCK_FILE"
else
echo "Script zaten çalışıyor. Çıkılıyor."
exit 1
fiSplitting Work and Singularity with Queue Systems
Sometimes, instead of needing distributed locks, it makes more sense to approach the problem from a different angle. Queue systems (like Kafka or RabbitMQ) can solve the distributed lock problem indirectly by splitting work into singular queues or using idempotency patterns.
On a bank’s internal platform, we used an event-sourcing architecture when processing financial transaction approvals. Each approval request landed in a queue and was processed by a single consumer. This eliminated the risk of the same transaction being approved multiple times. Additionally, each transaction was marked with a unique correlation_id to make it idempotent. Even if the transaction was reprocessed, it would yield the same result.
{
"event_id": "b3e0c7a1-8d2a-4f5c-9c7d-0a1b2c3d4e5f",
"correlation_id": "tx-12345-abcde",
"event_type": "PaymentApprovalRequested",
"payload": {
"account_id": "ACC001",
"amount": 100.00,
"currency": "TRY"
},
"timestamp": "2026-05-25T10:00:00Z"
}This approach reduces the coordination overhead that distributed locks introduce and improves the system’s scalability. However, queue systems themselves can have a complex structure and need to be managed correctly. For instance, scenarios such as reprocessing messages or routing them to dead-letter queues when a queue consumer crashes must be designed well.
Pragmatic Choices and Trade-offs
Each distributed lock alternative has its own advantages and disadvantages. In my experience, the choice was usually shaped by the application’s criticality level, performance requirements, and fault tolerance.
| Lock Mechanism | Advantages | Disadvantages | Use Case |
|---|---|---|---|
| Database Locks | High consistency, ACID guarantee | Performance bottleneck, deadlock risk | Financial transactions, critical inventory updates |
| Redis Locks | Fast, scalable | Network partition, Redis crash risks | Short-lived tasks, rate limiting, cache locks |
| File Locks | Simple, easy to set up | Not distributed, deadlock risk | Single-server cron jobs, local resource access |
| Queue Systems | Scalable, idempotency support | More complex architecture, latency risk | Asynchronous processing, event-driven architectures |
While doing production planning in a manufacturing company’s ERP, we had production planning algorithms. Since these algorithms took a long time, the same planning operation needed to not be triggered more than once. Here I chose Redis locks, because Redis was fast and, since the algorithms took an average of 30 seconds to complete, it was easy to manage with a TTL. However, to guarantee Redis’s high availability, we used a Sentinel architecture.
Last month, in one of my own side projects, I relied on Redis locks to manage the limit on users downloading a specific report at the same time. But during a restart of one Redis server for maintenance, I saw that some users downloaded the same report twice. This pushed me to think more carefully about failover and network partition scenarios when using Redis locks.
Conclusion: My Approach to Distributed Locks
Distributed locks are an unavoidable part of distributed systems, and when chosen correctly, they provide great convenience. However, there is no such thing as a single “best” lock mechanism. You need to make a pragmatic choice by considering the application’s needs, performance expectations, and fault tolerance. My personal approach is to avoid database locks as much as possible, because they are usually the slowest.
Instead, I lean toward lighter and more scalable solutions like Redis locks or queue systems. But when using these solutions, I always account for possible failure scenarios (network partition, server crash, lock expiration) and incorporate retry mechanisms, timeout durations, and idempotency principles into the implementation. Let’s not forget that nothing is one hundred percent guaranteed in distributed systems, so it is essential to always keep a fallback mechanism or an error-handling strategy on hand.