Race Conditions: A Simple Example and Four Ways to Fix It

Race conditions rarely show up in code review, and they pass every unit test. They appear when real concurrent traffic hits the system. After working on backend systems across different domains, I’ve seen this same pattern repeat in many forms, so I’ll use one simplified, generic example to walk through it.

The Example: Limited Stock Reservation

Imagine a system where several services can reserve units of a limited resource: a web checkout, a mobile API, and a bulk-order service. The reservation logic looks harmless:

async function reserve(itemId, qty) {
  const item = await db.query(
    'SELECT stock FROM items WHERE id = $1', [itemId]
  );

  if (item.stock < qty) {
    throw new Error('Not enough stock');
  }

  await db.query(
    'UPDATE items SET stock = $1 WHERE id = $2',
    [item.stock - qty, itemId]
  );
}

Read, check, write. In a single-threaded test it works every time.

Now suppose an item has 10 units left, and two requests each try to reserve 7 at nearly the same moment. Both read 10, both pass the check, and both write 3. We’ve sold 14 units from a stock of 10, and the data now says 3 are left.

This is the classic lost update, a check-then-act race:

Time   Request A              Request B
t1     READ stock = 10
t2                            READ stock = 10
t3     CHECK 10 >= 7 ✓
t4                            CHECK 10 >= 7 ✓
t5     WRITE stock = 3
t6                            WRITE stock = 3   ← A's update is lost

Neither request did anything wrong on its own. The bug lives in the gap between the read and the write. Every solution below is a different way of closing that gap.

Why It’s So Hard to Find (and How to Reproduce It Locally)

The nasty part of a race condition is that it’s intermittent and timing-dependent. The window between the read and the write is a few milliseconds wide. Most of the time two requests don’t land inside it, so everything works. The logs look clean, each request succeeded on its own, and nobody can say which request was “wrong.” You only notice later, when the numbers don’t add up.

Unit tests won’t catch it either, because they run one request at a time. You have to recreate the conditions that cause it: multiple instances of the service handling requests at the same moment.

Step 1: Run multiple instances locally

Start the same service on different ports, all pointing at the same database:

PORT=3001 node server.js &
PORT=3002 node server.js &
PORT=3003 node server.js &

If you want it closer to production, put nginx or any load balancer in front. Even without one, you can send requests directly to different ports. The point is that the requests are handled by separate processes, so in-process protection (like an app-level mutex) can’t help you.

Step 2: Fire concurrent requests at the same record

A small script is enough. This sends 30 reservations of 1 unit each, spread across the three instances, against an item with only 10 units in stock:

const ports = [3001, 3002, 3003];

async function run() {
  const requests = Array.from({ length: 30 }, (_, i) =>
    fetch(`http://localhost:${ports[i % ports.length]}/reserve`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ itemId: 1, qty: 1 }),
    }).then(r => r.status)
  );

  const results = await Promise.all(requests); // all fired at once
  const ok = results.filter(s => s === 200).length;
  console.log(`Successful reservations: ${ok} (expected max: 10)`);
}

run();

Step 3: Check the invariant, not the response codes

Every request may return 200 OK, so success codes tell you nothing. Check the data instead:

• Successful reservations should never exceed the initial stock.
• initial stock - final stock should equal the number of successful reservations.
• Stock should never go negative.

With the unprotected code, you’ll see something like 22 successes against a stock of 10, or a final stock that doesn’t match the number of reservations. That mismatch is your race condition.

Tips to make it show up reliably

• Widen the window. Temporarily add a small delay (await sleep(50)) between the read and the write. This makes a rare race happen almost every time, and it’s great for proving a fix works.
• Repeat the test. Run it 20 to 50 times. A race that doesn’t appear once may appear on the fifth run.
• Use the same record. Concurrency across different rows won’t conflict. The requests have to compete for one row.
• Reset the data between runs so each run starts from a known state.

Once you can reproduce the bug on demand, each solution below becomes testable: apply the fix, run the same script, and confirm the invariant holds on every run. If it passes only once, you haven’t proven anything.

Solution 1: Optimistic Locking

Assume conflicts are rare. Let everyone read freely, and detect a conflict at write time using a version column.

ALTER TABLE items ADD COLUMN version INT NOT NULL DEFAULT 0;

async function reserve(itemId, qty) {
  for (let attempt = 0; attempt < 3; attempt++) {
    const item = await db.query(
      'SELECT stock, version FROM items WHERE id = $1', [itemId]
    );

    if (item.stock < qty) throw new Error('Not enough stock');

    const result = await db.query(
      `UPDATE items
       SET stock = $1, version = version + 1
       WHERE id = $2 AND version = $3`,
      [item.stock - qty, itemId, item.version]
    );

    if (result.rowCount === 1) return; // success
    // someone else changed the row; retry with fresh data
  }
  throw new Error('Too much contention, try again');
}

If another request modified the row in between, the version no longer matches, the UPDATE affects zero rows, and we retry.

Pros

• No locks held, so no blocking and no deadlocks.
• Works well when conflicts are rare.

Cons

• Under high contention (a hot item during a sale), retries pile up and work is wasted.
• Every writer must follow the convention. One service that skips the version check silently breaks the guarantee.
• You need retry limits, back-off, and a failure path for the caller.

Solution 2: Queueing Through a Single Processor

Remove concurrency entirely. All reservation requests go into a queue (Kafka, SQS, RabbitMQ, Redis Streams), and a single consumer processes them one at a time, or one consumer per partition keyed by item_id.

Web checkout ─┐
Mobile API   ─┼──► [ queue ] ──► single processor ──► DB
Bulk orders  ─┘

Since only one worker touches an item at a time, the read-check-write sequence can’t interleave.

Pros

• Simple mental model: sequential processing means no races.
• Natural buffering during traffic spikes.
• Good for work that doesn’t need an immediate answer.

Cons

• Delay. The caller can’t get an answer in the request cycle, so you need polling, webhooks, or WebSockets. If a user is waiting on a confirmation screen, that’s a real cost.
• The processor is a throughput bottleneck and a possible single point of failure.
• Operational overhead: ordering guarantees, dead-letter queues, and duplicate delivery (which means you now need idempotency too).
• It only works if every writer goes through the queue. One direct DB write bypasses it.

Solution 3: Application-Level Mutexes

Wrap the critical section in a lock inside the application.

const mutex = new Mutex(); // in-process lock

async function reserve(itemId, qty) {
  await mutex.runExclusive(async () => {
    const item = await getItem(itemId);
    if (item.stock < qty) throw new Error('Not enough stock');
    await saveStock(itemId, item.stock - qty);
  });
}

This is fine on a single instance. But in production you run multiple instances behind a load balancer, and an in-process mutex protects nothing across processes. Request A on instance 1 and request B on instance 2 hold different mutexes and race exactly as before. (This is also why the multi-instance local setup above matters: a single local process would make this “fix” look like it works.)

The next step is a distributed lock (Redis SET NX PX, Redlock, ZooKeeper, etcd), which brings its own problems:

• The lock can expire while the holder is paused (GC, network stall), so two holders both believe they own it.
• If the lock service goes down, you have to choose between failing open and failing closed.
• It’s another piece of infrastructure, just to guard data that already lives in a database.
• Every service must use the same lock service and the same key naming.

Pros: Flexible, and it can guard resources that aren’t in a database.

Cons: In-process locks don’t scale horizontally. Distributed locks are tricky to get right and add a new failure mode.

Solution 4: Database Row-Level Locking

Let the database do what it is built to do. Lock the row for the duration of the transaction:

async function reserve(itemId, qty) {
  await db.transaction(async (tx) => {
    const item = await tx.query(
      'SELECT stock FROM items WHERE id = $1 FOR UPDATE',
      [itemId]
    );

    if (item.stock < qty) throw new Error('Not enough stock');

    await tx.query(
      'UPDATE items SET stock = stock - $1 WHERE id = $2',
      [qty, itemId]
    );
  });
}

SELECT ... FOR UPDATE makes the second request wait until the first transaction commits or rolls back. When it proceeds, it sees the updated stock of 3, fails the check, and correctly returns “Not enough stock.”

For a simple decrement, you can collapse this into one atomic statement and check the affected row count:

UPDATE items SET stock = stock - $1 WHERE id = $2 AND stock >= $1;

Why this is often the best fit

The deciding factor is that the database is the one place every service already shares. Different services, possibly in different languages and deployed independently, all talk to the same database. Getting all of them to agree on a queue, a lock service, or a versioning convention is a coordination problem. With row locking, the lock lives where the data lives, so consistency is enforced at a single point that every service already uses.

It also keeps response times predictable. A queue introduces delay and forces an async flow. With row locks, a request waits only as long as the competing transaction takes, usually milliseconds, and the caller still gets a synchronous, definitive answer.

Things to watch

• Very hot rows can build up waiting requests. Keep transactions short, and never call external APIs while holding a lock.
• Deadlocks are possible when transactions lock multiple rows in different orders. Always lock in a consistent order, for example by ascending ID.
• Set a lock timeout so requests fail fast instead of hanging.
• This ties correctness to a single database. Across multiple databases or shards, you need other techniques.

Comparison

	Optimistic locking	Single-processor queue	App-level mutex	DB row locking
Works across multiple instances	Yes	Yes	Only with a distributed lock	Yes
Response time	Fast, retries under contention	Delayed (async)	Fast	Fast, brief waits
Extra infrastructure	None	Queue + consumer	Lock service	None
Enforced for all services	Only if everyone follows the convention	Only if everyone uses the queue	Only if everyone uses the lock	Enforced by the database
Best for	Low contention	Async workloads, spike buffering	Single instance, non-DB resources	Shared data needing strong consistency

Takeaways

1. Race conditions live in the gap between “check” and “act.” Find that gap first, then choose a tool.
2. Put the guarantee where the data lives. When many services share one database, row-level locking gives you consistency without coordination between teams.
3. Use optimistic locking when contention is genuinely low, and queues when the work is naturally asynchronous. Don’t add a queue just to avoid thinking about locks, because you pay for it in latency.
4. Be wary of application-level mutexes in anything that scales horizontally. They fix the bug on your laptop and fail in production.
5. Reproduce it before you fix it. Run multiple instances locally, send concurrent requests at the same record, and verify the data invariant. Then use the same test to prove your fix.

The fix is rarely clever. It usually comes down to deciding who is in charge of ordering and making that decision explicit.