Hiring more cooks: threading basics

Night two, half past nine. Three line cooks tie on their aprons. Minho already has a knife in hand, Sua is sorting the tickets in order, and Mr. Tak is slowly wiping down the burners.

At ten the orders start coming in, and each of them puts a pot on. Thirty minutes later the chef asks, "Who made the udon for table 3?" Nobody can answer.

The chef taps the timer hanging from her neck. "Hiring cooks, we've done. Tonight you learn to give them name tags, hand them work, and wait until they're done."

2-1. Bringing in one cook: Thread, start, join (10 min)

In Python, one cook is one thread. Threads share the same kitchen (memory) while each gets on with its own work. You create one with threading.Thread.

import threading
import time

def cook(name, dish):
    print(f"{name}: start {dish}")
    time.sleep(0.3)          # wait for the soup to boil
    print(f"{name}: done {dish}")

minho = threading.Thread(target=cook, args=("minho", "ramen"))
minho.start()                # Minho gets to work
print("chef: next order")    # the chef moves on without waiting
minho.join()                 # wait until Minho is done
print("chef: served")
# minho: start ramen
# chef: next order
# minho: done ramen
# chef: served

• target= takes the work for the thread (a function), and args= takes the arguments for that function as a tuple. You pass the function itself rather than calling it, so it's cook, not cook().
• Creating a Thread(...) does nothing by itself. You've hired the cook, that's all. Only when you call start() does cook begin running in a new thread.
• start() returns right away. So while Minho cooks the ramen, the chef (the main thread, the one that started the program) moves on to the next line.
• join() waits until that thread is finished: "stand at the pass until Minho hands over the plate."

Give one order to each of the three and you get this:

def cook(name, dish):
    time.sleep(0.3)
    print(f"{name}: done {dish}")

crew = [("minho", "ramen"), ("sua", "udon"), ("tak", "soba")]
threads = [threading.Thread(target=cook, args=c) for c in crew]

start = time.perf_counter()
for th in threads:
    th.start()        # send all three off first
for th in threads:
    th.join()         # then wait for all of them
print(f"{time.perf_counter() - start:.2f}s")
# e.g.:
# sua: done udon
# minho: done ramen
# tak: done soba
# 0.30s

The order in which the three finish changes from run to run. All three finish almost at once after 0.3 seconds, so it depends on which one the operating system wakes up first. In concurrent code, "print order" is not a promise.

Don't put start() and join() in the same loop. If you alternate th.start(); th.join(), each person has to finish before the next one starts, and the whole thing takes 0.9 seconds.

You can also put a limit on the wait. join(timeout=seconds) waits only that long and returns, and is_alive() tells you whether the thread is still working.

slow = threading.Thread(target=time.sleep, args=(1.0,))
slow.start()
slow.join(timeout=0.2)
print(slow.is_alive())   # True: still boiling
slow.join()
print(slow.is_alive())   # False

• With 1 cook it takes about 1.8 seconds, exactly like "Alone, one by one." A single thread has nothing to speed up.
• 2 cooks take about 0.9 seconds, 3 take about 0.6, and 6 take about 0.3.
• 4 and 5 cooks both take about 0.6 seconds. Split six orders among four people and someone has to do two. The time is set by whoever has the most work.
• One row is one thread. The cooks on this timeline do exactly what the code above does: create a Thread, then call start() and join().

2-2. Calling run() is just a function call (5 min)

Thread also has a method called run(). The name says "run", but calling it yourself is a trap.

def cook(name, dish):
    time.sleep(0.3)
    print(f"{threading.current_thread().name}: done {dish}")

threads = [threading.Thread(target=cook, args=c) for c in crew]

start = time.perf_counter()
for th in threads:
    th.run()          # run(), not start()
print(f"{time.perf_counter() - start:.2f}s")
# MainThread: done ramen
# MainThread: done udon
# MainThread: done soba
# 0.90s

• run() is an ordinary method that calls target. It doesn't create a new thread; it runs everything one by one in the current thread. That's why MainThread did all three, and why it took 0.3 × 3 = 0.9 seconds.
• start() is what launches a new thread. start() creates the thread and then calls run() inside it.

A thread can only start once. Calling start() twice on the same thread raises an error. To do the same work again, create a new Thread.

th = threading.Thread(target=time.sleep, args=(0.1,))
th.start()
th.start()
# RuntimeError: threads can only be started once

2-3. Name tags: name, current_thread, enumerate (7 min)

Tonight's problem was "nobody knows who did it." Threads can wear name tags.

def cook(dish):
    me = threading.current_thread()       # the thread running this code
    print(me.name, dish, threading.get_native_id())
    time.sleep(0.2)

a = threading.Thread(target=cook, args=("ramen",), name="minho")
b = threading.Thread(target=cook, args=("udon",))      # what if we give no name?
a.start(); b.start()
print([t.name for t in threading.enumerate()])
print(threading.active_count())
a.join(); b.join()
print(threading.current_thread() is threading.main_thread())
print([t.name for t in threading.enumerate()])
# e.g.:
# minho ramen 25680
# Thread-1 (cook) udon 38072
# ['MainThread', 'minho', 'Thread-1 (cook)']
# 3
# True
# ['MainThread']

• b, which got no name, is Thread-1 rather than Thread-2 because only unnamed threads are numbered.
• The number from get_native_id() changes every run. The number Python uses internally is th.ident (or threading.get_ident()), and both are set only once the thread has started.
• A thread that has finished drops out of enumerate(). A cook who has gone home isn't on the roster.

2-4. Threads don't hand the plate back (6 min)

A function hands back its result with return, but a thread doesn't take that value for you. The return value of the target function is simply thrown away, and join() returns nothing either.

def make(dish):
    time.sleep(0.1)
    return f"{dish} ok"

th = threading.Thread(target=make, args=("ramen",))
th.start()
print(th.join())     # join returns nothing
# None

So you set out a container for the results ahead of time and have the threads put them there. The simplest way is to give each cook a slot of their own.

dishes = ["ramen", "udon", "soba"]
results = [None] * len(dishes)          # make the slots in advance

def cook(i, dish):
    results[i] = make(dish)             # write only to your own slot

threads = [threading.Thread(target=cook, args=(i, d)) for i, d in enumerate(dishes)]
for th in threads:
    th.start()
for th in threads:
    th.join()
print(results)
# ['ramen ok', 'udon ok', 'soba ok']

If you'd rather look things up by name, use a dict.

done = {}

def cook2(name, dish):
    done[name] = make(dish)

ts = [threading.Thread(target=cook2, args=(n, d)) for n, d in [("minho", "ramen"), ("sua", "udon")]]
for t in ts:
    t.start()
for t in ts:
    t.join()
print(sorted(done.items()))
# [('minho', 'ramen ok'), ('sua', 'udon ok')]

• When each cook writes to a different slot or a different key, they never collide. Once they start reading and rewriting the same slot, it's a different story, and that's tomorrow night's incident.
• Always read the results after join() has finished. Read them earlier and some slots may still be None.
• If building a container every time feels tedious, the Future returned by ThreadPoolExecutor in Lesson 7 does this job for you. You get the plate directly with future.result().

2-5. daemon=True: threads nobody waits for at lights-out (6 min)

Even after the main thread ends, a Python program waits until every non-daemon thread has finished before it exits. Threads are non-daemon by default.

def clean():
    for i in range(1, 6):
        print("cleaning", i)
        time.sleep(0.1)

th = threading.Thread(target=clean)      # daemon=False (the default)
th.start()
time.sleep(0.25)
print("lights off")
# cleaning 1
# cleaning 2
# cleaning 3
# lights off
# cleaning 4
# cleaning 5

Even when the chef shouts "lights off," the doors don't close until the cook who's cleaning has finished. Now the same code with just daemon=True added:

th = threading.Thread(target=clean, daemon=True)
th.start()
time.sleep(0.25)
print("lights off", th.daemon)
# cleaning 1
# cleaning 2
# cleaning 3
# lights off True

• When the program ends, nobody waits for daemon threads. The lights go out in the middle of cleaning, and steps 4 and 5 never happen.
• If you do want to wait, call join() even on a daemon thread. join() waits regardless of the daemon flag.
• Use daemon threads only for "work that's fine to cut off midway," like a watcher thread that prints status every so often. Hand a daemon something that must finish, like writing a file or recording orders, and you'll end up with half-written files.
• Set the daemon flag before start(). If you don't set it, a thread inherits the value of the thread that created it, and since the main thread isn't a daemon, that's usually False.

2-6. More tools: subclassing Thread, Timer, local (6 min)

Subclassing Thread and overriding run. Useful when you want each cook to carry its own state (the dish, the result). As 2-2 showed, start() calls run() in the new thread, so you only need to rewrite run().

class Cook(threading.Thread):
    def __init__(self, name, dish):
        super().__init__(name=name)     # always call the parent's __init__ first
        self.dish = dish
        self.result = None

    def run(self):                      # start() calls this in the new thread
        time.sleep(0.1)
        self.result = f"{self.dish} ok"

tak = Cook("tak", "soba")
tak.start()
tak.join()
print(tak.name, tak.result)
# tak soba ok

threading.Timer: once, after n seconds. Like a timer for boiling noodles, it's a thread that calls a function once after a set time. If it hasn't gone off yet, cancel() stops it.

start = time.perf_counter()

def ring(msg):
    print(f"{time.perf_counter() - start:.1f}s", msg)

noodles = threading.Timer(0.5, ring, args=("noodles up",))
noodles.start()                     # rings once after 0.5 seconds
eggs = threading.Timer(1.0, ring, args=("eggs up",))
eggs.start()
eggs.cancel()                       # cancel before it rings and nothing happens
noodles.join()
eggs.join()
print("end")
# 0.5s noodles up
# end

threading.local(): an apron pocket for each cook. One variable name, but a different value for each thread. Which pocket you get depends on who's reaching in.

pocket = threading.local()

def cook(name, n):
    pocket.name = name
    pocket.count = 0
    for _ in range(n):
        pocket.count += 1
        time.sleep(0.01)
    print(pocket.name, pocket.count)

ts = [threading.Thread(target=cook, args=a) for a in [("minho", 3), ("sua", 5)]]
for t in ts:
    t.start()
for t in ts:
    t.join()
print(hasattr(pocket, "name"))      # the chef's (main thread's) pocket is empty
# minho 3
# sua 5
# False

pocket is a single global variable, yet Minho's count and Sua's count never mix. It's a common way to give each thread its own database connection.

Closing time (10 min)

Tonight's goal was to split six orders among three people, finish in under 0.6 seconds, and record who finished what at which second. Give everyone a name tag (name=) and have each of them write a line in a shared log whenever they finish.

start = time.perf_counter()
log = []                                # (finish time, cook, order)

def cook(dishes):
    me = threading.current_thread().name
    for dish in dishes:
        time.sleep(0.3)
        log.append((round(time.perf_counter() - start, 1), me, dish))

plan = {"minho": ["#1", "#4"], "sua": ["#2", "#5"], "tak": ["#3", "#6"]}
threads = [threading.Thread(target=cook, args=(d,), name=n) for n, d in plan.items()]
for th in threads:
    th.start()
for th in threads:
    th.join()

for row in sorted(log):
    print(*row)
print(f"total {time.perf_counter() - start:.2f}s, {len(log)} orders")
# 0.3 minho #1
# 0.3 sua #2
# 0.3 tak #3
# 0.6 minho #4
# 0.6 sua #5
# 0.6 tak #6
# total 0.60s, 6 orders

• The argument is a single list, hence the comma in args=(d,).
• log.append(...) is a list operation that completes in one go, so no lines disappear even when several threads call it. Read the log after join(), sorted.

The chef skims the log and nods. "Good, now we know who did what. So tomorrow let's try one more thing. If the four of you each add your own count to a single number on the chalkboard, will that number be right?"

Exercises

1. In the three-cook example from 2-1, change the code so start() and join() sit in the same loop. Predict how many seconds it will take, then time it.
2. In the closing-time code, make only Mr. Tak take 0.5 seconds per order (hint: vary the sleep time by cook name). Who sets the total time? Compare it with a timeline where just one row gets longer.
3. Use threading.Timer to build a kitchen timer that rings "three times, every 0.3 seconds." A Timer rings only once, so each ring has to start the next Timer.