Distributed Locks & Coordination

Redis’s single-threaded core (Lesson 1) has a powerful side effect: because only one command runs at a time, Redis is a natural coordinator for distributed systems. Operations like INCR and SETNX are atomic for free — no locks, no races — which makes Redis the go-to for distributed locks, rate limiters, one-time tokens and job claiming. But distributed locks in particular have sharp edges that catch people out. This lesson is how to coordinate with Redis correctly.

The atomic building blocks

Because each command is atomic on the single thread, Redis gives you race-free primitives:

• INCR — an atomic counter. Two clients both INCR and each gets a distinct, correct value — no lost updates. The basis of counters and rate limits.
• SETNX (or SET ... NX) — “set only if the key doesn’t exist.” Exactly one client succeeds; the rest fail. The basis of locks and once-only operations (claim a job, send a welcome email once).
• INCR + EXPIRE — a fixed-window counter that resets: the basis of rate limiting.
• Lua (EVAL) — multi-step logic executed atomically as one command (Lesson 5), for anything that needs several steps to be indivisible.

There’s never a race between the check and the set, because there’s no gap — it’s one command on one thread. That’s what makes Redis coordination so clean compared to rolling your own with a database.

A correct distributed lock

Distributed locks are where people most often get it subtly wrong. The goal: across many processes, ensure only one holds a lock on some resource at a time (e.g. “only one worker processes order 7”). A correct Redis lock needs three things together:

SET lock:order:7 <unique-token> NX EX 30

1. NX (set-if-absent) — only one client can acquire the lock; everyone else’s SET NX fails because the key exists. This is the mutual exclusion.
2. EX 30 (a TTL) — the lock auto-expires after 30 seconds. This is critical: if the holder crashes while holding the lock, without a TTL the lock would be held forever — a permanent deadlock that blocks the resource for everyone. The TTL guarantees the lock is eventually released even if the holder dies. (Set it longer than the work takes, with margin.)
3. A unique token, released via Lua — when you’re done, you must release only your own lock. The naive DEL lock:order:7 is a bug: if your work ran long and your lock already expired and was re-acquired by another client, a plain DEL would delete their lock. So you store a unique token as the value, and release with a Lua script that deletes only if the token matches:

if redis.call('GET', KEYS[1]) == ARGV[1] then
  return redis.call('DEL', KEYS[1])
else
  return 0
end

This check-token-then-delete must be atomic (hence Lua), so no one re-acquires between the check and the delete. Get all three right — NX for exclusion, EX to avoid deadlock, token+Lua to release safely — and you have a correct single-instance lock.

Redlock, and the honest caveat

The lock above is correct against a single Redis instance. But what if that instance fails over (Lesson 8)? Because replication is async, a just-acquired lock might not have replicated before a failover, so two clients could briefly hold “the” lock. For stronger guarantees across independent nodes, there’s Redlock — an algorithm that acquires the lock on a majority of several independent Redis instances, so a single node’s failure doesn’t break mutual exclusion.

Redlock is more robust but genuinely debated — distributed-systems experts disagree about whether it provides correct locking under all failure and timing conditions (clock skew, process pauses). The pragmatic stance:

• For the common case — coordination where an occasional, rare double-acquire during a failover is tolerable (and your work is idempotent anyway) — a single-instance lock with NX+EX+token is simple and good enough.
• For cases where mutual exclusion must hold under all conditions, don’t rely on any Redis lock as the sole guarantee: make the protected operation idempotent or fence it (e.g. a monotonically increasing fencing token checked by the resource), so a spurious double-acquire can’t cause harm. A lock is an optimisation to reduce contention, not a substitute for making the critical operation safe.

The honest summary: Redis locks are great for reducing contention and coordinating the common case; don’t bet correctness under all failures on a distributed lock alone — make the operation itself safe.

Rate limiting and other coordination

The same atomic primitives give you other coordination patterns cheaply:

• Fixed-window rate limit — INCR rate:user:42:<minute>; if it exceeds the limit, reject; EXPIRE the key so the window resets. Simple and atomic.
• Sliding-window / token-bucket rate limit — a sorted set of request timestamps (trim old ones, count the window), or a Lua token-bucket script for precise control — one atomic operation per request.
• Once-only operations — SET done:<id> 1 NX; the first succeeds and does the work, the rest skip. De-duplicate deliveries, send a one-time email, claim a job.
• Leader election / job claiming — SET NX EX on a well-known key: whoever sets it is the leader/owner for the TTL; renew it to stay leader.

The unifying rule, carried from Lesson 5: when a coordination task needs several steps to be one atomic operation, write a Lua script. Redis’s single thread guarantees the script runs indivisibly, which is exactly what correct coordination requires. Next we look at the advanced patterns the data structures unlock.

Pitfalls to Avoid

1. A lock with no TTL. If the holder crashes, the lock is held forever — a permanent deadlock. Always set an expiry (EX) longer than the work, so a dead holder’s lock auto-releases.
2. Releasing with a plain DEL. If your lock already expired and someone else re-acquired it, DEL deletes their lock. Store a unique token and release with a Lua script that deletes only if the token matches.
3. Non-atomic check-then-act. Checking a value then acting on it in two commands is a race. Use SET NX, INCR, WATCH/MULTI, or a Lua script so the operation is atomic.
4. Betting correctness on a distributed lock. Failover + async replication can allow a rare double-acquire. For must-not-happen-twice operations, make them idempotent or use a fencing token — don’t rely on the lock alone.
5. TTL too short for the work. If the lock expires before the work finishes, another client acquires it and you get concurrent processing. Size the TTL above the worst-case work time, and renew it for long tasks.
6. Over-reaching with Redlock. Redlock adds complexity and is debated. For most apps a single-instance lock + idempotent work is simpler and sufficient. Reserve multi-node Redlock for cases that truly need it (and still make the operation safe).

Quick Recap

• Redis’s single thread makes INCR, SETNX and Lua atomic with no locking — ideal for coordination (locks, rate limits, once-only, job claiming).
• A correct lock needs three things: SET lock <token> NX EX <ttl> — NX (one winner), EX (TTL so a crashed holder never deadlocks), and a unique token released via Lua (delete only if it’s yours).
• Redlock locks across multiple nodes (more robust, debated). For most apps, a single-instance lock + idempotent/fenced operation is simpler and safer — don’t bet correctness on a lock alone.
• Rate limiting: INCR+EXPIRE (fixed window) or a Lua token bucket (precise). The rule: multi-step atomicity → Lua.

Next lesson: the patterns the data structures unlock — leaderboards, rate limiting, HyperLogLog, bitmaps, geo.