Threading and the GIL

Thread-based concurrency, synchronisation primitives, and what the Global Interpreter Lock really means.

Overview

Python's `threading` module lets you run multiple threads inside one process. Threads share memory, so they communicate easily but they also require careful synchronisation to avoid race conditions. The **Global Interpreter Lock (GIL)** is a mutex inside CPython that ensures only one thread executes Python bytecode at a time. This means pure-Python CPU-bound threads do NOT run in parallel. However, the GIL is released during I/O operations (network, disk, sleep), so threads are highly effective for I/O-bound work network scrapers, file processors, and similar tasks benefit greatly. For CPU-bound parallelism you need `multiprocessing` (separate processes, no GIL) or native extensions that release the GIL (NumPy, etc.). Key synchronisation tools: `Lock` (mutual exclusion), `RLock` (reentrant lock), `Semaphore` (limit concurrent access), `Event` (signal between threads), `Queue` (thread-safe data exchange).

Key Points

•Create threads with `threading.Thread(target=fn, args=(...))`; start with `.start()`, wait with `.join()`
•GIL prevents true parallel CPU execution in CPython use threads for I/O-bound work only
•`threading.Lock()` prevents race conditions; use `with lock:` as context manager
•`threading.Event` lets one thread signal another `.set()`, `.wait()`, `.clear()`
•`queue.Queue` is thread-safe for producer-consumer patterns no Lock needed
•Daemon threads (`t.daemon = True`) exit when the main thread ends good for background workers
•`threading.local()` gives each thread its own copy of a variable (thread-local storage)

Examples

Basic threading I/O-bound tasks run concurrently

import threading
import time

def fetch(url):
    print(f"Start {url}")
    time.sleep(1)          # simulates network I/O (GIL released during sleep)
    print(f"Done  {url}")

urls = ["site-A", "site-B", "site-C"]

threads = [threading.Thread(target=fetch, args=(u,)) for u in urls]
for t in threads:
    t.start()
for t in threads:
    t.join()

print("All done")
# All 3 complete in ~1 s (concurrent), not 3 s (sequential)

Lock prevents a race condition on shared state

import threading

counter = 0
lock = threading.Lock()

def increment(n):
    global counter
    for _ in range(n):
        with lock:          # only one thread modifies counter at a time
            counter += 1

threads = [threading.Thread(target=increment, args=(10000,)) for _ in range(5)]
for t in threads: t.start()
for t in threads: t.join()

print(counter)   # always 50000 without lock it would be less

Queue for producer–consumer pattern

import threading
import queue
import time

q = queue.Queue(maxsize=5)

def producer():
    for i in range(8):
        q.put(i)           # blocks when full
        print(f"produced {i}")
        time.sleep(0.05)
    q.put(None)            # sentinel to stop consumer

def consumer():
    while True:
        item = q.get()     # blocks when empty
        if item is None:
            break
        print(f"  consumed {item}")
        q.task_done()

t1 = threading.Thread(target=producer)
t2 = threading.Thread(target=consumer)
t1.start(); t2.start()
t1.join(); t2.join()

Common Mistakes

Using threads for CPU-heavy work expecting speedup the GIL means no parallelism; use multiprocessing instead
Forgetting to call `.join()` the main thread may exit before child threads finish
Modifying shared mutable objects without a Lock produces non-deterministic bugs that are hard to reproduce

Note

A fast rule of thumb: if the bottleneck is waiting (network, file, sleep) → threads. If the bottleneck is computation → multiprocessing or numpy/C extensions.

Threading and the GIL

Overview

Key Points

Examples

Common Mistakes

Note

Exercises

Knowledge Check

What does the Global Interpreter Lock (GIL) prevent in CPython?