Pattern: Dealing with Contention

Some of the most memorable interview questions are really contention questions in disguise. “Design Ticketmaster” is “how do you sell the last seat to exactly one person.” “Design a flash sale” is “how do you not oversell when a million people hit buy at once.” “Design a wallet” is “how do you debit a balance without ever losing or duplicating money.” The shared difficulty is concurrency on a single resource: many requests want to modify the same row at the same instant, and naïve code gets it wrong.

This is a pattern worth internalizing because the failure mode is subtle and the fixes are a small, reusable toolkit. Reach for the right tool and these problems become routine; reach for the wrong one and you either oversell or grind the system to a halt.

The race everyone writes by accident

The classic bug is the read-modify-write race. Imagine two requests buying the last unit of stock:

Request A: read stock = 1
Request B: read stock = 1          # both saw 1
Request A: 1 > 0, write stock = 0  # sells it
Request B: 1 > 0, write stock = 0  # sells it AGAIN  -> oversold

Both requests read the same value before either wrote, so both think a unit is available. The window is tiny — microseconds — but at scale, with thousands of concurrent buyers, “tiny and rare” becomes “constant.” Every solution below is a different way to make that read-modify-write safe.

In the interview: when the question has a scarce, shared resource — seats, inventory, a balance, a counter — say “there’s contention here; let me make sure two requests can’t both grab the last one.” Naming the race before you solve it is exactly the instinct interviewers are checking for.

Pessimistic vs optimistic

Pessimistic locking

Assume conflict will happen, so take a lock before you touch the row and make everyone else wait:

BEGIN;
SELECT stock FROM products WHERE id = 42 FOR UPDATE;  -- locks the row
-- now no one else can read-for-update this row until we commit
UPDATE products SET stock = stock - 1 WHERE id = 42;
COMMIT;

This is correct and easy to reason about. The cost is that it serializes access to the row: under heavy contention, requests queue up behind the lock, throughput collapses, and you risk deadlocks if transactions lock multiple rows in different orders. Pessimistic locking is the right call when conflicts are frequent and correctness is non-negotiable (money, inventory) — but keep the locked section as short as possible.

Optimistic concurrency control (OCC)

Assume conflict is rare, so don’t lock — just detect a conflict if it happens. Read a version number with the data, then write only if the version hasn’t changed:

-- read: stock = 1, version = 7
UPDATE products
   SET stock = stock - 1, version = version + 1
 WHERE id = 42 AND version = 7;   -- succeeds only if no one else changed it
-- if 0 rows affected, someone else won the race -> re-read and retry

When collisions are uncommon this is wonderful: no locks, high concurrency, no deadlocks. When collisions are common (the last few tickets) it degrades into a retry storm, with many requests repeatedly failing and retrying. The decision between pessimistic and optimistic is really a question about one number: how often do requests actually collide?

The wider toolkit

Locks are not the only tool. Often you can sidestep the race entirely.

Atomic operations

The cleanest fix is to let the datastore do the read-modify-write in one indivisible step, so there is no window to race in:

-- the WHERE clause makes the check-and-decrement atomic
UPDATE products SET stock = stock - 1 WHERE id = 42 AND stock > 0;
-- rows affected = 1 -> you got one; = 0 -> sold out

Redis:  DECR stock:42        # atomic; returns the new value

A single conditional UPDATE or an atomic INCR/DECR removes the race without any explicit lock. Prefer this whenever the operation is simple enough to express atomically — it is the simplest correct answer and the one to reach for first.

Distributed locks

When the critical section spans more than one datastore — “deduct from the wallet and call the payment provider” — a database row lock isn’t enough. A distributed lock (Redis SET key val NX PX 30000, or a ZooKeeper/etcd ephemeral node) lets one node hold exclusive ownership across the cluster. Two cautions that interviewers probe for:

• Always set a TTL, so a crashed holder’s lock is eventually released and the system doesn’t deadlock forever.
• Use a fencing token — a monotonically increasing number handed out with the lock. A holder that paused (GC, network blip) and resumed after its lock expired can be rejected by the resource because its token is stale. Without fencing, a zombie holder can corrupt state.

Idempotency keys

Retries are a fact of distributed life — a client times out and resends, a queue redelivers — and a retried “charge $10” must not charge twice. Have the client send a unique idempotency key with the write; the server records processed keys and, on a duplicate, returns the original result instead of re-applying the operation. This converts “at-least-once” delivery into “exactly-once effect” and is essential anywhere money or inventory moves.

Serialize through a single writer

Sometimes the simplest way to remove contention is to remove the concurrency: route all updates for a given key to one place that processes them one at a time. Partition a queue by resource id so all operations on account 42 land on the same consumer, and they execute in order with no locking at all. This trades a little latency for dramatically simpler correctness, and it is how many high-throughput systems handle per-entity updates.

Sharded counters

A single hot counter (likes on a viral post, global stock of a flash-sale item) becomes a contention point because every increment fights for the same row. Split it into N sub-counters, increment a random one, and sum them on read. You trade a slightly more expensive read for N× the write throughput — the write-side mirror of the hot-key fix from the scaling-pitfalls lesson.

Worked mini-example: selling the last seat

Putting it together for “book the last concert seat without overselling”:

1. Atomic claim. UPDATE seats SET status='held', holder=:user WHERE id=:seat AND status='available'. Exactly one concurrent request affects a row; everyone else gets 0 rows and is told the seat is gone. No explicit lock needed.
2. Hold with a TTL. The held seat expires after a few minutes (via a TTL / scheduled sweep) if the user doesn’t pay, releasing it back to the pool.
3. Idempotent purchase. The checkout call carries an idempotency key so a double-submit or retry confirms the same booking rather than charging twice.

For a flash sale with extreme contention on one SKU, swap the row update for an atomic Redis DECR on the stock counter as the gate, and only the winners proceed to the slower database write — moving the contention to an in-memory operation that can take hundreds of thousands of ops per second.

Where this pattern shows up

• Ticketing / seat booking — atomic claim + hold TTL + idempotent purchase.
• Flash sales / limited inventory — atomic decrement as the gate, sharded if the SKU is hot.
• Wallets / payments — pessimistic lock or atomic balance update + idempotency keys.
• Counters (likes, views, stock) — atomic increments, sharded when hot.
• Any “exactly one winner” decision — leader election, claiming a job from a queue.

Key Takeaways

1. The enemy is the read-modify-write race. Two requests read the same value and both write, so you oversell or lose data. Name it before you solve it.

2. Pessimistic vs optimistic depends on collision frequency. Frequent conflicts → lock (SELECT ... FOR UPDATE). Rare conflicts → version/CAS with retry. Don’t lock what rarely collides, and don’t optimistically retry what always collides.

3. Prefer an atomic operation when you can express one. A single conditional UPDATE ... WHERE stock > 0 or a Redis DECR removes the race with no explicit lock — reach for this first.

4. Distributed locks need a TTL and a fencing token. Otherwise a crashed or paused holder deadlocks the system or corrupts state as a zombie.

5. Idempotency keys make retries safe. They turn at-least-once delivery into exactly-once effects — mandatory anywhere money or inventory moves.

6. Remove contention structurally when you can. Serialize updates per key through a single writer, and shard hot counters to spread the write load.