The Global Interpreter Lock and concurrency

The Global Interpreter Lock, or GIL, is a mutex in CPython that protects access to Python objects, ensuring that only one thread executes Python bytecode at a time within a process. It simplifies memory management and makes single-threaded code fast, but it limits true parallelism in CPU-bound multithreaded programs.

The GIL exists because CPython uses reference counting for memory management. Without a global lock, multiple threads modifying reference counts could corrupt them, leading to memory errors or crashes. The GIL trades multithread parallelism for simplicity and single-thread speed.

CPU-bound multithreaded Python programs are limited because only one thread can execute bytecode at a time. I/O-bound programs are not limited because the GIL is released during I/O operations, allowing other threads to run. Numerical libraries like NumPy also release the GIL for heavy computation in C code.

Several approaches exist: use multiprocessing to spawn separate processes, each with its own GIL; use C extensions that release the GIL for compute work; offload work to libraries like NumPy, Pandas, or PyTorch that release the GIL internally; or use the new free-threaded Python build from version 3.13 onward.

Free-threaded Python, introduced experimentally in Python 3.13 and officially supported in Python 3.14, is a build of CPython without the GIL. It enables true multithreaded parallelism for pure Python code. It is the result of PEP 703. Free-threaded builds are opt-in because they require ecosystem updates and have slightly different single-threaded performance.

Free-threaded Python enables true CPU parallelism in threads but introduces some single-threaded performance overhead, currently around five to ten percent. It also requires C extensions to be made thread-safe. As of 2026, the major scientific libraries are progressively adding free-threaded support, but ecosystem readiness varies by package.

Threading uses operating system threads inside one process. Threads share memory but in standard CPython are limited by the GIL. Multiprocessing uses separate processes, each with its own Python interpreter and memory. Processes can run truly in parallel on multiple cores, at the cost of more memory and slower data exchange.

Use threading for I/O-bound work like network requests, file operations, or database queries, where threads spend most of their time waiting. Use multiprocessing for CPU-bound work like numerical computation, data processing, or any code that spends most of its time executing Python bytecode.

Concurrent.futures provides a high-level interface for asynchronously executing callables. It offers ThreadPoolExecutor for I/O-bound tasks and ProcessPoolExecutor for CPU-bound tasks. Both share the same API: submit returns a Future, map applies a function across an iterable, and the executor is used as a context manager.

Concurrent.interpreters is a new standard library module added in Python 3.14 that provides a Python API for working with subinterpreters. Each subinterpreter has its own state and, in supported builds, its own GIL. Subinterpreters offer isolation similar to processes with the lower overhead of shared memory inside a single OS process. ---