Concurrency Primitives: Locks, Semaphores & Condition Variables

The race. counter += 1 is really read, add, write. Two threads can read the same value, both add one, and both write back: one increment is lost. The bug appears only under load and is hard to reproduce, which is why the fix must be structural, not 'add a sleep'.

Lock (mutex). Only one thread holds it; others wait. Keep the critical section small and always release it, with with lock: so an exception cannot leave it held:

import threading

lock = threading.Lock()
count = 0

def add():
    global count
    with lock:
        count += 1

Semaphore. A counter of permits: acquire takes one (blocking at zero), release returns one. Semaphore(5) allows five concurrent users of a resource, which is exactly what a limit like 'at most 5 database connections' means.

Condition variable. Lets a thread wait for a *state change* without spinning. Always wait in a loop, because a wake-up does not guarantee the condition is still true:

cond = threading.Condition()
items = []

def consume():
    with cond:
        while not items:      # re-check after every wake-up
            cond.wait()
        return items.pop()

def produce(x):
    with cond:
        items.append(x)
        cond.notify()

A thread-safe connection pool combines them: a list of idle connections guarded by a lock, a condition (or queue) so callers wait when none are free, a maximum size, and a timeout so a caller fails instead of hanging forever. Returning a connection must notify one waiter, and a broken connection must be replaced, not returned.

Deadlock. Thread A holds lock 1 and wants lock 2; thread B holds 2 and wants 1: both wait forever. Prevent it by always acquiring locks in one global order, by using timeouts, or by holding only one lock at a time.

Prefer higher-level tools. queue.Queue is already thread-safe and covers most producer/consumer needs. In Python the GIL protects interpreter internals but not your compound operations, so you still need locks. Share nothing where you can, and pass messages instead.