Why the dough won't go faster: the GIL and parallelism

One in the morning, service over. Four batches of noodle dough for tomorrow sit on the counter. Minho, Sua, Mr. Tak and you. Four people, one batch each, so it should go four times as fast.

But the chef's timer stops at almost the same number as when one person kneads alone. Minho tilts his head. "Everyone's hands were busy, though?"

Chef Yoon Seoyeon points at the knife block on the counter. There's exactly one knife in it. "This kitchen has an old rule. Only one person cuts or kneads at a time. Tonight we talk about that rule."

8-1. Four cooks, same time (5 min)

The orders in Lesson 1 were waiting work: waiting for soup to boil. Tonight's dough is different. It's a loop that adds numbers two million times: work that never lets your hands rest (CPU work).

import threading
import time

def knead(n=2_000_000):
    s = 0
    for i in range(n):   # nonstop calculation: kneading the dough
        s += i
    return s

start = time.perf_counter()
for _ in range(4):
    knead()
print(f"sequential {time.perf_counter() - start:.2f}s")

start = time.perf_counter()
cooks = [threading.Thread(target=knead) for _ in range(4)]
for th in cooks:
    th.start()
for th in cooks:
    th.join()
print(f"4 threads  {time.perf_counter() - start:.2f}s")
# sequential 0.24s
# 4 threads  0.23s   (measured on a 12-core PC, Python 3.14)

• Four threads really did run, yet it took almost exactly as long as doing them one by one.
• This PC has 12 cores. It's not for lack of cores.
• The numbers vary a little between PCs and between runs. What matters is that the two are almost the same.

8-2. The GIL: only one knife (8 min)

CPython (the ordinary Python we use) has a GIL (Global Interpreter Lock). It has one rule:

In kitchen terms, it's the "only one knife" rule. Four cooks or not, only the one holding the knife can cut, and the rest wait for the knife to come back. The knife passes to the next person roughly every 0.005 seconds (sys.getswitchinterval(), Lesson 3).

The four take turns kneading, so all four batches make a little progress. There's concurrency but no parallelism (Lesson 1). The total amount of work is unchanged, so the time is unchanged too.

Why have such a rule? Python counts, for every object, "how many places are using me" (its reference count). If several threads changed that number at the same time, it would go wrong just like the chalkboard in Lesson 3. Long ago, CPython took the easy road of guarding the whole interpreter with one lock, and that's the GIL.

Put the knife down while you wait

The GIL has an important exception. While a thread is waiting (time.sleep, reading a file, a network response), it lets go of the GIL. Someone waiting for soup to boil doesn't need the knife.

def simmer():
    time.sleep(0.3)   # waiting for the soup to boil: the GIL is released meanwhile

start = time.perf_counter()
cooks = [threading.Thread(target=simmer) for _ in range(4)]
for th in cooks:
    th.start()
for th in cooks:
    th.join()
print(f"4 threads {time.perf_counter() - start:.2f}s")
# 4 threads 0.30s   (1.2 seconds if done one by one)

Functions written in C can also release the GIL while doing heavy computation. hashlib does this when hashing large data. It's a job run by a dough machine (C code) rather than by hand, so it sits outside the knife rule.

import hashlib

dough = b"x" * 50_000_000   # 50MB

def press():
    hashlib.sha256(dough).hexdigest()   # releases the GIL while the C code computes

# timed the same way as above (roughly, on this PC)
# sequential 0.11s
# 4 threads  0.04s

And the GIL doesn't protect your code. It only protects the interpreter's internals; multi-step work like count += 1 still gets interrupted midway. That's why the chalkboard in Lesson 3 went wrong even with the GIL.

The lab below is a kitchen where real threads call time.sleep on the deployment server. The server runs Python with the GIL, yet because this is waiting work, the bars overlap.

8-3. A branch kitchen: multiprocessing (7 min)

If there's only one knife, set up another kitchen with its own knife. That's a process. Each process runs its own Python interpreter with its own GIL.

import multiprocessing as mp
import os

def branch(name):
    print(name, "pid", os.getpid(), "parent", os.getppid())

if __name__ == "__main__":
    print("main pid", os.getpid())
    p = mp.Process(target=branch, args=("branch-1",))
    p.start()
    p.join()
    print("exitcode", p.exitcode)
# e.g.:
# main pid 39516
# branch-1 pid 40268 parent 39516
# exitcode 0

Process is used the same way as threading.Thread (target, args, start, join). The difference is os.getpid(), the process ID. The branch is a separate program with a different number from the main kitchen.

Even the fridge is separate

A branch doesn't share memory. Whatever the branch puts in a list, the main kitchen's list stays the same.

orders = []

def add(n):
    orders.append(n)
    print("in branch:", orders)

if __name__ == "__main__":
    p = mp.Process(target=add, args=("#1",))
    p.start()
    p.join()
    print("in main:", orders)
# in branch: ['#1']
# in main: []

The chalkboard problem from Lesson 3 can't even arise, but in exchange, getting results back takes a separate step. When you're farming out several jobs, a Pool is handy.

def knead(n):
    s = 0
    for i in range(n):
        s += i
    return s

if __name__ == "__main__":
    with mp.Pool(processes=4) as pool:
        print(pool.map(knead, [10, 100, 1000, 10_000]))
# [45, 4950, 499500, 49995000]

Why if __name__ == "__main__": is a must

Windows and macOS open branches with the spawn method: a fresh Python starts up and imports your file again from the top. It has to reread the function definitions to know what knead is.

Without that check, the branch, while reading the file, runs into the mp.Process(...).start() line again and tries to open a branch of its own. Python stops this and raises an error.

# starting right away at the top level, without the check (Windows)
p = mp.Process(target=knead, args=(1,))
p.start()
# in the branch:
# RuntimeError:
#         An attempt has been made to start a new process before the
#         current process has finished its bootstrapping phase.
#         ...

That's why the multiprocessing examples are saved as a file and run with python file.py. In an interactive shell or a notebook, the branch often fails because it can't find the function.

8-4. Agency cooks, branch edition: ProcessPoolExecutor (6 min)

Remember ThreadPoolExecutor from Lesson 7? ProcessPoolExecutor hands work to processes with the same interface. submit, Future, result(), map and as_completed all work just the same.

import os
from concurrent.futures import ProcessPoolExecutor

def knead(n):
    s = 0
    for i in range(n):
        s += i
    return os.getpid(), s

if __name__ == "__main__":
    print("main", os.getpid())
    with ProcessPoolExecutor(max_workers=2) as ex:
        future = ex.submit(knead, 1000)
        print(future.result())
        for pid, s in ex.map(knead, [10, 100, 1000]):
            print(pid, s)
# e.g.:
# main 43452
# (42784, 499500)
# 42784 45
# 42784 4950
# 42784 499500

The process ID differs from main. Which branch takes which batch of dough varies from run to run.

Whatever you send must be picklable

Branches and the main kitchen don't share memory, so functions, arguments and results are packed into bytes (pickled) to be sent back and forth. Anything that can't be packed can't be sent. The classic example is a lambda.

def double(x):
    return x * 2

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=2) as ex:
        print(ex.submit(double, 21).result())
        try:
            ex.submit(lambda x: x * 2, 21).result()
        except Exception as e:
            print(type(e).__name__, e)
# 42
# PicklingError Can't pickle <function <lambda> at 0x...>: it's not found as __main__.<lambda>

• Functions are packed by name. The branch looks up "the double in the file" and uses it. A lambda has no name, so the branch can't find it.
• So define functions for a branch with def at the top level of the file. Nested functions and lambdas won't work.
• Open files, locks, sockets and the like can't be sent as arguments either. Stick to simple values: numbers, strings, lists, dicts.

8-5. The cost of opening a branch (4 min)

A branch isn't free. Python has to start up fresh, reimport the file, and pack values to send back and forth. If the dough is small, that setup takes longer than the work itself.

# 4 small batches from 8-1 (two million each) on 4 processes
with ProcessPoolExecutor(max_workers=4) as ex:
    results = list(ex.map(knead, [2_000_000] * 4))
# 4 processes 0.31s   (measured on a 12-core PC; one by one was 0.24 s)

It actually got slower. Make the dough ten times bigger (twenty million) and the story changes. On this PC it was roughly 2.4–2.7 seconds one by one and 0.9–1.1 seconds with 4 processes. The setup cost stayed about the same while the work grew.

• Use processes for computations where each piece is big enough. When you're sending thousands of small jobs, bundle several into one send (map's chunksize=).
• Create a pool once and reuse it. Opening a new with ProcessPoolExecutor() for every job means setting up new branches every time.

8-6. Three ways to open a branch: spawn, fork, forkserver (3 min)

How a branch gets set up is called the start method.

• From 3.14, the default on Linux is forkserver. On Windows and macOS it's spawn. The old Linux default, fork, was changed because cloning a program with running threads can leave locks in a tangle.
• Windows only has spawn.

import multiprocessing as mp

if __name__ == "__main__":
    print(mp.get_start_method())        # spawn   (Windows)
    print(mp.get_all_start_methods())   # ['spawn']

    ctx = mp.get_context("spawn")       # spawn just for this pool
    with ctx.Pool(processes=2) as pool:
        print(pool.map(knead, [10, 100]))   # [45, 4950]

• mp.get_context("spawn") applies only to the Pool and Process objects made from that context. Inside a library, this is the safe choice.
• To change it for the whole program, call mp.set_start_method("spawn") just once, right at the start of if __name__ == "__main__":. Call it twice and you get RuntimeError: context has already been set.
• The default differs between operating systems, so if your code must behave the same everywhere, set spawn explicitly and test with it.

8-7. The remodel: GIL-free Python 3.14t (5 min)

From 3.14, the free-threaded build is officially supported. It's Python built without the GIL, and its executable is called python3.14t (note the t at the end). It's as if the kitchen were remodeled to have one knife per cook.

• Install the free-threaded binaries alongside the regular ones in the python.org installer, or with uv, run uv run --python 3.14t file.py.
• Check whether the GIL is currently on with sys._is_gil_enabled().

# knead_threads.py: add just two lines to the top of the 8-1 code
import sys
print(sys._is_gil_enabled())
...

# python knead_threads.py
# True
# sequential 0.24s
# 4 threads  0.23s

# python3.14t knead_threads.py
# False
# ...
# 4 threads  0.09s   (measured on a 12-core PC, about 2.7×)

Not a single character of the code changed. The same thread code got about 2.7 times faster just by switching interpreters. It's not four times faster because of the cost of starting and managing threads, plus the cost of the internal machinery that keeps things safe without the GIL. On the other hand, code that uses only one thread can run a little slower than on the regular build.

Two things to keep in mind:

• C extensions can turn the GIL back on. If you import an extension module that hasn't declared support for free threading, Python issues a warning and turns the GIL back on. You can force it off with the environment variable PYTHON_GIL=0 (or -X gil=0), and with PYTHON_GIL=1 the GIL is on even in 3.14t.
• Race conditions happen more easily. The chalkboard experiment from Lesson 3 gave 2410 out of 4000 on 3.14t (measured). When things truly run at once, locks matter even more.

In the lab below, the top four bars were measured on a 12-core PC while preparing this lesson. "Knead on the server" below them runs the dough (600,000 iterations × 4) live on this site's deployment server, one by one and with 4 threads. The deployment server runs a GIL build and its core count can't be known in advance, so the numbers may differ from the measured bars. Look at whether the two bars come out about the same length.

8-8. Partitioned kitchens: subinterpreters (2 min)

Branches are expensive, and not every library has caught up with the remodel yet. There's one more option in between: partitioned kitchens inside one building, or subinterpreters. You keep several interpreters inside one process, and each partition has its own GIL (knife). In 3.14 they joined the standard library as the concurrent.interpreters module.

from concurrent import interpreters

booth = interpreters.create()
booth.exec("menu = ['#1', '#2']; print('booth menu:', menu)")
booth.exec("menu.append('#3'); print('booth menu:', menu)")
print("main has menu?", "menu" in globals())

def knead(n):
    s = 0
    for i in range(n):
        s += i
    return s

print(booth.call(knead, 1000))
booth.close()
# booth menu: ['#1', '#2']
# booth menu: ['#1', '#2', '#3']
# main has menu? False
# 499500

• create() makes a partition, exec(code string) runs code inside it, and call(function, args) calls a function and gets the result back.
• Variables inside a partition (menu) exist only inside it. Like a branch, it doesn't share memory.

In practice, InterpreterPoolExecutor (3.14), shaped just like the tools from Lesson 7, is the convenient option.

from concurrent.futures import InterpreterPoolExecutor

if __name__ == "__main__":
    with InterpreterPoolExecutor(max_workers=4) as ex:
        results = list(ex.map(knead, [2_000_000] * 4))
# roughly 0.18 s on this PC (3.14 with the GIL; one by one was 0.24 s)

Each partition has its own knife, so the dough gets kneaded side by side even on regular Python. Keep in mind that many C extensions don't support subinterpreters yet.

Closing time (10 min)

Tonight's goal was to finish the four batches of dough at least twice as fast as one person alone. It's pure Python computation, so threads won't help, and each batch is big, so the cost of opening branches is worth paying.

import time
from concurrent.futures import ProcessPoolExecutor

BIG = 20_000_000   # one big batch of dough

def knead(n):
    s = 0
    for i in range(n):
        s += i
    return s

if __name__ == "__main__":
    start = time.perf_counter()
    alone = [knead(BIG) for _ in range(4)]
    print(f"sequential {time.perf_counter() - start:.2f}s")

    start = time.perf_counter()
    with ProcessPoolExecutor(max_workers=4) as ex:
        branches = list(ex.map(knead, [BIG] * 4))
    print(f"4 processes {time.perf_counter() - start:.2f}s", branches == alone)
# e.g. (on this PC):
# sequential 2.43s
# 4 processes 0.93s True

About 2.5 times faster, and the results match what one person got alone. With 3.14t, running knead on four threads would give a similar effect.

The chef unties her apron. "We solved the dough by adding kitchens. But what about the floor? Ten tables: do we put ten servers out there, or have one person cover all ten?"

Exercises

1. Change knead from 8-1 to time.sleep(0.06) and time it again one by one and with 4 threads. Explain, using the GIL, why threads speed things up this time.
2. Change the dough size in 8-5 to 200,000, 2 million and 20 million and write down the times for one by one and for ProcessPoolExecutor in a table. At what size do processes start winning on your PC?
3. submit a function defined inside another function (a nested function) to a ProcessPoolExecutor, see what error you get, and fix it.