# Python Threading & Concurrency Cheatsheet

- URL: https://karandeepsingh.ca/cheatsheets/python/python-threading-concurrency/
- Author: Karandeep Singh
- Published: 2025-06-03
- Tags: python
- Summary: Python concurrency notes covering threading primitives, queue producer/consumer, concurrent.futures, multiprocessing, asyncio, and when to use which.

Half my automation scripts start out sequential and end up needing to hit 200 hosts at once. That's when this sheet comes out. Python has four concurrency tools (threads, processes, executors, asyncio), and picking the wrong one for the workload is the classic mistake. If you're new to Python itself, start with the [Python Basics & CLI Cheatsheet](https://karandeepsingh.ca/cheatsheets/python/python-basics-cli/) first.

## The GIL in One Minute

> **Docs:** [Global interpreter lock in the Python glossary](https://docs.python.org/3/glossary.html#term-global-interpreter-lock)

- CPython's Global Interpreter Lock allows only **one thread to execute Python bytecode at a time**.
- Threads still help for **I/O-bound** work (HTTP calls, disk, DB, subprocess) because the GIL is released while waiting on I/O.
- Threads do **not** help CPU-bound work (parsing, hashing, compression). Use processes for that.
- C extensions (numpy, hashlib on large buffers) often release the GIL internally, so results vary. Measure.

The claim is worth measuring rather than believing. Four CPU-bound tasks,
four workers, the same function both times:

```text
$ python3 gil_demo.py
ThreadPoolExecutor   0.61s
ProcessPoolExecutor  0.16s
# Four threads took longer than four processes by a factor of four — they
# were taking turns on one core the whole time. Swap the work for an HTTP
# call and the numbers invert, because the GIL is released during I/O.
```

## Which One to Use When

> **Docs:** [Concurrent Execution, Python 3 docs](https://docs.python.org/3/library/concurrency.html)

| Workload | Best tool | Why |
|----------|-----------|-----|
| I/O-bound, few dozen tasks | `ThreadPoolExecutor` | Simple, GIL released during I/O |
| I/O-bound, thousands of tasks | `asyncio` | One thread, cheap coroutines |
| CPU-bound | `ProcessPoolExecutor` / `multiprocessing` | Real parallelism, sidesteps GIL |
| Mixed pipeline, decoupled stages | threads + `queue.Queue` | Producer/consumer backpressure |
| Fire-and-forget background work | `Thread(daemon=True)` | Dies with the main process |

## threading Module

> **Docs:** [Python threading docs](https://docs.python.org/3/library/threading.html)

### Thread

```python
import threading

def worker(host: str) -> None:
    print(f"[{threading.current_thread().name}] pinging {host}")

t = threading.Thread(target=worker, args=("web1",), name="ping-1", daemon=True)
t.start()
t.join(timeout=5)            # wait for it (with a timeout)
t.is_alive()                 # True if still running
```

### Lock & RLock

```python
lock = threading.Lock()
counter = 0

def increment() -> None:
    global counter
    with lock:               # always use `with` - releases on exception
        counter += 1

# RLock: same thread may re-acquire (needed when locked methods call each other)
rlock = threading.RLock()

class Registry:
    def __init__(self):
        self._lock = threading.RLock()
        self._data = {}

    def set(self, k, v):
        with self._lock:
            self._data[k] = v

    def set_many(self, items):
        with self._lock:          # re-enters fine because it's an RLock
            for k, v in items:
                self.set(k, v)
```

### Event, Semaphore, Condition

```python
# Event - one-shot broadcast flag (shutdown signals, "ready" gates)
stop = threading.Event()

def poller():
    while not stop.is_set():
        check_health()
        stop.wait(timeout=5)  # sleep, but wake immediately on set()

stop.set()                    # tell all waiters to stop

# Semaphore - cap concurrency (e.g. max 5 simultaneous API calls)
sem = threading.BoundedSemaphore(5)

def call_api(url):
    with sem:
        return fetch(url)

# Condition - wait for a state change under a lock
cond = threading.Condition()
items = []

def consumer():
    with cond:
        cond.wait_for(lambda: len(items) > 0, timeout=10)
        item = items.pop()

def producer(x):
    with cond:
        items.append(x)
        cond.notify()         # notify_all() to wake every waiter
```

## queue.Queue Producer/Consumer

> **Docs:** [Python queue docs](https://docs.python.org/3/library/queue.html)

```python
import queue, threading

q: queue.Queue = queue.Queue(maxsize=100)   # maxsize gives you backpressure
SENTINEL = object()

def producer():
    for job in jobs:
        q.put(job)                # blocks when full
    q.put(SENTINEL)

def consumer():
    while True:
        job = q.get()             # blocks when empty; q.get(timeout=5) raises queue.Empty
        if job is SENTINEL:
            q.put(SENTINEL)       # let sibling consumers see it too
            break
        try:
            process(job)
        finally:
            q.task_done()

threads = [threading.Thread(target=consumer, daemon=True) for _ in range(4)]
[t.start() for t in threads]
producer()
q.join()                          # blocks until every task_done() is called
```

`queue.LifoQueue` (stack) and `queue.PriorityQueue` (`(priority, item)` tuples) share the same API.

## concurrent.futures

> **Docs:** [concurrent.futures on docs.python.org](https://docs.python.org/3/library/concurrent.futures.html)

The interface I reach for 90% of the time. Same code works for threads and processes.

```python
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor, as_completed

urls = [f"https://host{i}/health" for i in range(50)]

# map - results in submission order
with ThreadPoolExecutor(max_workers=10) as pool:
    for status in pool.map(fetch, urls, timeout=30):
        print(status)

# submit + as_completed - results as they finish, per-task error handling
with ThreadPoolExecutor(max_workers=10) as pool:
    futures = {pool.submit(fetch, url): url for url in urls}
    for fut in as_completed(futures, timeout=60):
        url = futures[fut]
        try:
            print(url, fut.result())
        except Exception as e:
            print(f"{url} failed: {e}")

# CPU-bound? One-line switch:
with ProcessPoolExecutor(max_workers=4) as pool:
    checksums = list(pool.map(sha256_file, big_files))
```

**warning:**
`pool.map` swallows nothing: the first task that raised will re-raise when you iterate its result, killing the loop. For fan-out where some tasks may fail (health checks across a fleet), use `submit` + `as_completed` and try/except each `fut.result()`.

## multiprocessing Basics

> **Docs:** [Python multiprocessing docs](https://docs.python.org/3/library/multiprocessing.html)

```python
import multiprocessing as mp

def crunch(chunk):
    return sum(x * x for x in chunk)

if __name__ == "__main__":                # REQUIRED guard on every entry point
    with mp.Pool(processes=mp.cpu_count()) as pool:
        results = pool.map(crunch, chunks)

    # Explicit Process + IPC
    q = mp.Queue()
    p = mp.Process(target=worker, args=(q,))
    p.start(); p.join()

    # Shared state (rarely needed - prefer passing data)
    counter = mp.Value("i", 0)
    with counter.get_lock():
        counter.value += 1
```

Notes: arguments and results must be picklable; each process has its own memory (no shared globals); default start method is `spawn` on macOS/Windows and since 3.14 on Linux too, so keep everything under the `__main__` guard.

## asyncio Quick Reference

> **Docs:** [Python asyncio docs](https://docs.python.org/3/library/asyncio.html)

```python
import asyncio
import aiohttp   # pip install aiohttp - requests is blocking, don't use it here

async def fetch(session: aiohttp.ClientSession, url: str) -> int:
    async with session.get(url) as resp:
        return resp.status

async def main() -> None:
    async with aiohttp.ClientSession() as session:
        # gather - run concurrently, keep order
        statuses = await asyncio.gather(
            *(fetch(session, u) for u in urls),
            return_exceptions=True,          # exceptions become results, not crashes
        )

        # Tasks - start now, await later
        task = asyncio.create_task(fetch(session, "https://x/health"))
        status = await task

        # Timeouts
        try:
            async with asyncio.timeout(5):           # 3.11+
                await fetch(session, slow_url)
        except TimeoutError:
            print("gave up")
        # pre-3.11 equivalent: await asyncio.wait_for(coro, timeout=5)

        # Structured concurrency (3.11+) - all-or-nothing task group
        async with asyncio.TaskGroup() as tg:
            for u in urls:
                tg.create_task(fetch(session, u))

asyncio.run(main())
```

```python
# Other essentials
await asyncio.sleep(1)                       # never time.sleep() in async code
sem = asyncio.Semaphore(10)                  # cap concurrent coroutines
async with sem: ...
await asyncio.to_thread(blocking_func, arg)  # run blocking code without freezing the loop
```

## Common Pitfalls

> **Docs:** [Developing with asyncio, Python 3 docs](https://docs.python.org/3/library/asyncio-dev.html)

| Pitfall | Fix |
|---------|-----|
| `time.sleep()` inside async code | `await asyncio.sleep()` |
| Shared `counter += 1` across threads | Wrap in `Lock`; `+=` is not atomic |
| Forgetting `if __name__ == "__main__"` with multiprocessing | Spawn re-imports your module: infinite process bomb |
| Unbounded thread creation per request | Use an executor with `max_workers` |
| Daemon threads doing writes at shutdown | Non-daemon + `Event` for clean shutdown |
| Calling `fut.result()` with no timeout | Pass `timeout=` so hung tasks can't hang you |

Go handles all this very differently, with goroutines that are cheap and CPU-parallel by default. See the [Go (Golang) Cheatsheet](https://karandeepsingh.ca/cheatsheets/languages/golang/) for the comparison.

**Question:** Think of the slowest loop you have parallelised. Was it waiting on a network, or on the CPU — and did you pick threads or processes to match?

## Related Cheatsheets

- [Python Basics & CLI](https://karandeepsingh.ca/cheatsheets/python/python-basics-cli/): comprehensions, pathlib, argparse, and the interpreter flags these concurrency examples assume
- [Go (Golang)](https://karandeepsingh.ca/cheatsheets/languages/golang/): goroutines, channels, and select, which is what concurrency looks like without a GIL
- [Python Boto3](https://karandeepsingh.ca/cheatsheets/python/python-boto3/): where I use these pools most, for parallel S3 transfers and multi-region API sweeps

There's more in the [Python group](https://karandeepsingh.ca/cheatsheets/python/), and the full [cheatsheet library](https://karandeepsingh.ca/cheatsheets/) has everything else.