MiddleCodeOccasionalNot answered yet
Parallelize CPU-bound work with a process pool
Compute the squares of 0..9 across multiple processes (real CPU parallelism).
Requirements:
- Use a process pool so the work bypasses the GIL.
- Collect the results into a list in input order.
from concurrent.futures import ProcessPoolExecutor
def square(n):
return n * n
# your code here: run square over range(10) across processesWrite the implementation.
Use ProcessPoolExecutor to sidestep the GIL: with ProcessPoolExecutor() as pool: results = list(pool.map(square, range(10))). Each process has its own interpreter and GIL, so the work runs truly in parallel. Functions and arguments must be picklable to cross the boundary.
- ✗Using threads for CPU-bound work (the GIL blocks parallelism)
- ✗Passing a non-picklable function or argument across processes
- ✗Submitting the whole iterable as one argument instead of mapping per item
- →Why must the function and arguments be picklable for
ProcessPoolExecutor? - →When is
ThreadPoolExecutorthe right choice instead ofProcessPoolExecutor?
Contents
Task
Compute the squares of 0..9 in parallel across multiple processes, bypassing the GIL.
Solution
from concurrent.futures import ProcessPoolExecutor
def square(n):
return n * n
if __name__ == "__main__":
with ProcessPoolExecutor() as pool:
results = list(pool.map(square, range(10)))
print(results) # [0, 1, 4, 9, ..., 81], computed across processesKey points
ProcessPoolExecutorgives each process its own interpreter and GIL — true CPU parallelism.pool.mappreserves input order in the results.- The function and arguments must be picklable; for I/O work choose
ThreadPoolExecutor.
Contents
