Concurrency & GIL

The GIL (Global Interpreter Lock) is a mutex letting only one thread execute Python bytecode at a time, protecting interpreter internals like reference counts. A thread releases it on I/O and periodically.

async def defines a coroutine; await suspends it until an awaitable completes, yielding control to a single-threaded event loop that runs other ready coroutines meanwhile. No extra threads.

A coroutine generalizes a subroutine: it has multiple entry points and can suspend and resume, preserving its state. In Python they use async def/await (or yield), enabling cooperative concurrency.

Yes — the threading module creates native OS (POSIX/Windows) threads scheduled by the operating system, not interpreter-level green threads. The GIL still serializes their Python bytecode execution.

Threads share one address space — the same memory, variables, and modules — so they communicate cheaply but need synchronization. Processes are independent with separate spaces and must use IPC.

Use asyncio.gather to schedule the coroutines concurrently on one thread: await asyncio.gather(fetch(1), fetch(2), fetch(3)). await asyncio.sleep yields control so all three overlap and finish in ~1s, not 3s. A blocking time.sleep would stall the whole loop instead.

The two threads are no faster than sequential — often slower, from GIL contention — because count is pure-Python CPU work and the GIL lets only one thread run bytecode at a time. To parallelize, use multiprocessing.Process or a ProcessPoolExecutor.

I/O-bound tasks benefit: a thread releases the GIL while waiting on network or disk, letting others run, so concurrency hides latency. CPU-bound tasks don't — the GIL serializes them.

Each process has its own interpreter and its own GIL, so processes run Python bytecode in true parallel across cores, sidestepping the single-GIL serialization that throttles threads on CPU-bound work.

Race condition: counter += 1 is a read-modify-write — not atomic at the bytecode level — so threads interleave and lose updates (the GIL guarantees per-bytecode atomicity, not multi-step operations). Fix: guard the increment with a threading.Lock (with lock: counter += 1).

Lock-ordering inversion: t1 takes a then b, t2 takes b then a. If each acquires its first lock at once, neither can get the second → deadlock. Fix: acquire locks in a single consistent global order everywhere, or use a timeout (lock.acquire(timeout=...)).

Lightweight user-space threads scheduled by the runtime or library (e.g. greenlet, gevent), invisible to the OS — to it the process looks single-threaded. The library switches them cooperatively.

Use ProcessPoolExecutor to sidestep the GIL: with ProcessPoolExecutor() as pool: results = list(pool.map(square, range(10))). Each process has its own interpreter and GIL, so the work runs truly in parallel. Functions and arguments must be picklable to cross the boundary.

It blocks the entire event loop: one thread runs all coroutines cooperatively and only await yields control, so a synchronous blocking call stalls every other coroutine until it returns.

Use asyncio with an async HTTP client (e.g. aiohttp), reading the Server response header per site. Bound concurrency with an asyncio.Semaphore — without it you exhaust sockets/file descriptors. Set a per-request timeout and wrap each fetch in try/except so one failure does not sink the batch (count it as unknown). Stream the CSV rather than loading it all. Sockets/FDs run out first: the work is I/O-bound, so CPU and memory stay low while open connections pile up.

Yes. The GIL already serializes the bytecode, so there is no speedup; on top of that the runtime keeps forcing GIL acquire/release handoffs between contending threads, and that pure switching overhead can drag total throughput below the single-threaded baseline.

Use asyncio: a single-threaded event loop handles vast numbers of concurrent I/O waits cheaply, which is exactly this workload. Threads add OS-thread overhead and context-switch cost that does not scale to a million; processes are for CPU-bound work and waste memory here; single-threaded is far too slow. Bound concurrency with an asyncio.Semaphore so you don't exhaust sockets or file descriptors, and gather results per URL.