One floor manager for every table: first steps in asyncio

From tonight the floor works differently. There's one person on it: Lou. Ten tables. The chef hasn't hired anyone else.

Lou never stops circling the dining room. She takes an order, passes it to the kitchen, and instead of waiting for the food, heads straight to the next table. When the kitchen bell rings, she picks up that plate and takes it out. One lap, then another. Because she runs the same loop over and over, reacting to bells (events), the cooks gave her a nickname on her very first day: the Event Loop.

Tying her apron strings, Lou says, "What I hate most is standing at a table, just waiting. While I wait, the other tables are waiting for me."

The chef nods. "Up to yesterday, we added people. Tonight it's about how one person uses the time spent waiting. In Python it's called asyncio."

9-1. async def makes a coroutine (7 min)

Put async in front of a function and it becomes a coroutine function. Let's start with the strange part: calling it doesn't run it.

import asyncio

async def take_order(table):
    print("order", table)
    return f"T{table}"

c = take_order(1)
print(type(c).__name__)
print(c)
# coroutine
# <coroutine object take_order at 0x...>
# <sys>:0: RuntimeWarning: coroutine 'take_order' was never awaited

• take_order(1) didn't print "order 1". Not a single line of the function body ran.
• What came back is a coroutine object. It's like a slip of paper that says "take table 1's order."
• If you throw the slip away without ever running it, Python warns was never awaited at the end. When you see that warning, you've almost always forgotten an await.

There are two ways to run the slip: at the top level, pass it to asyncio.run(); inside another coroutine, await it.

print(asyncio.run(take_order(1)))
# order 1
# T1

c = take_order(2)
asyncio.run(c)
asyncio.run(c)
# order 2
# RuntimeError: cannot reuse already awaited coroutine

A coroutine object can only be used once. To take the same order again, call take_order(2) again to write a new slip.

9-2. await and asyncio.run (7 min)

await means "I'll wait right here until this is done; meanwhile the loop is free to do other things." You can only use await inside async def. Use it inside a plain def and you get SyntaxError: 'await' outside async function.

import asyncio
import time

async def table(n, wait):
    print(f"T{n} order")
    await asyncio.sleep(wait)   # Lou is free while waiting on the kitchen
    print(f"T{n} served")
    return f"T{n}"

async def one_by_one():
    a = await table(1, 0.3)
    b = await table(2, 0.3)
    return [a, b]

start = time.perf_counter()
print(asyncio.run(one_by_one()), f"{time.perf_counter() - start:.2f}s")
# T1 order
# T1 served
# T2 order
# T2 served
# ['T1', 'T2'] 0.62s

• asyncio.sleep(0.3) is the asyncio version of time.sleep(0.3). While it waits, it hands the turn over to the loop.
• Yet the result is 0.3 + 0.3 = about 0.6 seconds, because await table(1, ...) has to finish before the next line runs. It's like Lou standing at table 1 until the food comes out.
• await doesn't mean "at the same time"; it means "wait." To wait on several things at once, you need the tools in the next section.

asyncio.run(main()) is the entry point of the program.

1. It creates a new event loop (Lou clocks in).
2. It runs the main() coroutine until it finishes.
3. It cancels any leftover tasks and closes the loop (Lou clocks out).

You usually call it once in a program, at the top level only. Call it again where a loop is already running and you get RuntimeError: asyncio.run() cannot be called from a running event loop.

9-3. What the event loop does (6 min)

The event loop is a loop running in a single thread. What it does is exactly what Lou does on the floor.

1. Pick one coroutine that can make progress right now.
2. Run that coroutine up to its next await.
3. Note what it's waiting for (a timer, a network response) and move on to the next coroutine.
4. If nothing can make progress, wait for whichever bell rings first and wake that coroutine up again.

Hand her three tables at once (with gather, coming up next) and Lou's route looks like this. Table 1's dish takes 0.3 seconds, table 2's 0.2, and table 3's 0.1.

Remember two things.

• There is one person. There's only one thread, and at any moment only one coroutine is running. It gets faster because the waiting overlaps, not because there are more hands.
• The turn changes only at await. The operating system can interrupt a thread at any moment (Lesson 3), but a coroutine steps away only when it awaits. So the code between two awaits is never interrupted midway.

9-4. gather: waiting at the same time (8 min)

asyncio.gather(coroutine, coroutine, ...) puts several coroutines on the loop at once and waits until all of them are done.

async def together():
    return await asyncio.gather(table(1, 0.3), table(2, 0.2), table(3, 0.1))

start = time.perf_counter()
print(asyncio.run(together()), f"{time.perf_counter() - start:.2f}s")
# T1 order
# T2 order
# T3 order
# T3 served
# T2 served
# T1 served
# ['T1', 'T2', 'T3'] 0.31s

• The time taken is not the sum (0.6 seconds) but the longest one (0.3 seconds).
• They finished in the order T3 → T2 → T1, but the result list is in the order you passed them in, just like executor.map in Lesson 7.
• When you've built a list of coroutines, unpack it with a star: gather(*coros).

On the lab below, switch between "await table by table" and "All at once with gather." One row is one table, and every bar is in the "Waiting" color: Lou hardly spends any time actually working; nearly all of it is waiting on the kitchen.

• Table by table: the time grows with the number of tables. About 2.6 seconds for 10.
• gather: about 0.25 seconds whether there's 1 table or 12. The effect we got in Lesson 1 by hiring eight cooks, we now get with a single server.
• This lab, too, runs a real asyncio loop on the server and draws the moment each table started and finished.

When one table goes wrong: return_exceptions

If one of the coroutines passed to gather raises an exception, gather raises it as is. To get every result back, pass return_exceptions=True.

async def table(n):
    await asyncio.sleep(0.1 * n)
    if n == 2:
        raise ValueError(f"T{n} wrong dish")
    return f"T{n}"

async def main():
    results = await asyncio.gather(table(1), table(2), table(3), return_exceptions=True)
    print(results)
    try:
        await asyncio.gather(table(1), table(2), table(3))
    except ValueError as e:
        print("ValueError:", e)

asyncio.run(main())
# ['T1', ValueError('T2 wrong dish'), 'T3']
# ValueError: T2 wrong dish

• With return_exceptions=True, an exception takes up its own slot in the result list. Check the results one by one with isinstance(r, Exception).
• With the default (False), even when an exception comes up, the other tables are not cancelled and keep going. "If one fails, stop the rest" is what TaskGroup does for you in Lesson 10.

9-5. create_task: hand it off early (6 min)

asyncio.create_task(coroutine) wraps the coroutine in a task and schedules it on the loop right away. The task starts when Lou has a free hand (when the current coroutine awaits). It doesn't wait for the task to finish before moving to the next line, so you can put an order in first and get on with other work.

async def table(n, wait):
    print(f"T{n} order")
    await asyncio.sleep(wait)
    print(f"T{n} served")
    return f"T{n}"

async def main():
    start = time.perf_counter()
    t1 = asyncio.create_task(table(1, 0.3))
    t2 = asyncio.create_task(table(2, 0.2))
    print("Lou: 2 tasks created")
    await asyncio.sleep(0.1)          # Lou does something else for a moment
    print("Lou: back", t1.done(), t2.done())
    print(await t1, await t2, f"{time.perf_counter() - start:.2f}s")

asyncio.run(main())
# Lou: 2 tasks created
# T1 order
# T2 order
# Lou: back False False
# T2 served
# T1 served
# T1 T2 0.31s

• "Lou: 2 tasks created" prints before T1 order. A task is scheduled the moment it's created, but it first runs when main steps away at an await.
• await t1 waits for a task that's already running to finish. The two tasks waited together, so it takes about 0.3 seconds, not 0.5.
• gather wraps the coroutines it receives in tasks internally. Use gather when you just want the results collected; use create_task when you want to hold on to a task, check its state, or wait for it later.

Hold on to a reference to your tasks

The event loop holds tasks only by weak reference. A task you don't keep in a variable or collection can be garbage-collected before it finishes. Even for jobs where you don't need the result, like refilling water, keep hold of a reference.

background = set()

async def refill_water(table):
    await asyncio.sleep(0.1)
    print(f"T{table} water")

async def main():
    for n in (1, 2, 3):
        task = asyncio.create_task(refill_water(n))
        background.add(task)                         # keep hold of a reference
        task.add_done_callback(background.discard)   # removes itself when done
    print("pending", len(background))
    await asyncio.sleep(0.2)
    print("pending", len(background))

asyncio.run(main())
# pending 3
# T1 water
# T2 water
# T3 water
# pending 0

A line that throws the result away, like asyncio.create_task(refill_water(n)), usually works fine, but "usually" is the problem. A task that occasionally vanishes is the hardest kind of bug to track down.

9-6. asyncio.sleep vs time.sleep (3 min)

Both mean "wait a moment," but to Lou they're completely different things.

async def polite(n):
    await asyncio.sleep(0.3)   # hands the turn to the loop while waiting

async def stubborn(n):
    time.sleep(0.3)            # stops the whole loop

async def main(fn):
    start = time.perf_counter()
    await asyncio.gather(*(fn(n) for n in range(3)))
    print(fn.__name__, f"{time.perf_counter() - start:.2f}s")

asyncio.run(main(polite))
asyncio.run(main(stubborn))
# polite 0.32s
# stubborn 0.90s

• The moment you await it, asyncio.sleep tells the loop "wake me in 0.3 seconds" and steps away.
• time.sleep stops the thread itself. The loop runs in just one thread, so it's like Lou standing at a table, just waiting. Meanwhile nobody looks after the other tables.
• So the three tables took 0.3 seconds each in turn, about 0.9 seconds in all. Using gather doesn't help.

Not just time.sleep: long-running calculations and old libraries that don't hand over the turn while they wait stop the loop in exactly the same way. That problem and its fix are the subject of Lesson 11.

9-7. Coroutines, tasks, futures, and what they cost (3 min)

Three similar names come up in asyncio.

print(issubclass(asyncio.Task, asyncio.Future))   # True

async def main():
    coro = take_order(1)
    task = asyncio.create_task(coro)
    print(type(coro).__name__, type(task).__name__)   # coroutine Task

The idea is the same as concurrent.futures.Future from Lesson 7. The difference is that an asyncio future can be awaited, and is used only within the same loop. You'll rarely create a future yourself; coroutines and tasks are usually all you need.

Tasks are light

Threads are created by the operating system, so each one comes with stack memory and OS bookkeeping. A task is just one Python object.

async def table(n):
    await asyncio.sleep(0.5)
    return n

async def main():
    return await asyncio.gather(*(table(n) for n in range(10_000)))

# 10,000 tasks: about 0.57 s   (10,000 tables each waiting 0.5 s)
# 1,000 threads: about 0.62 s  (1,000 threads each waiting 0.5 s)
# measured on this PC

Starting 10,000 tasks costs less than starting 1,000 threads. That's why asyncio is a good fit when there are thousands or tens of thousands of waiting jobs (a chat server, sending requests to lots of URLs).

Closing time (10 min)

Tonight's goal was to get food out to all ten tables in under 0.5 seconds without adding staff. One Lou, one event loop, one gather, and you're done.

import asyncio
import time

async def table(n):
    await asyncio.sleep(0.3)   # waiting for the food to come out of the kitchen
    return f"T{n}"

async def hall():
    return await asyncio.gather(*(table(n) for n in range(1, 11)))

start = time.perf_counter()
served = asyncio.run(hall())
print(served)
print(f"{time.perf_counter() - start:.2f}s")
# ['T1', 'T2', 'T3', 'T4', 'T5', 'T6', 'T7', 'T8', 'T9', 'T10']
# 0.30s

Not a single thread was created. Ten 0.3-second tables finished in about 0.3 seconds.

Lou sets down her empty tray. "Ten tables, done in one lap." The chef looks at her timer and asks, "Good. But tomorrow there's a group booking. If one of the six tables cancels, how are you going to stop the food for the other five?"

Exercises

1. Run, one after the other, a line that calls take_order from 9-1 without await and a line that runs it with asyncio.run(). Compare the output and the warnings.
2. If you delete await asyncio.sleep(0.1) from main in 9-5, how does the output order change? Write down your prediction first, then run it to check.
3. Increase the tables in the closing-time code to 100 and then 1,000 and time it. Compare with doing the same job on 1,000 threads.