Unclaimed bags: memory and GC

The two bags on conveyor 13, reported since your very first day, are still going around today. They carry no tags at all. By the rules, a bag without tags goes straight to the disposal area that same day.

Supervisor Do Miran spreads three old records on her desk: the X-ray from Lesson 3, the note from Lesson 6 and the partner record from Lesson 7. "Today we find out what those two are. But first you need to know the disposal rule exactly."

11-1. Counting tags: reference counting (9 min)

Every object in CPython carries a number that counts how many references point to it. This is its reference count. Hang a tag and it goes up by 1; remove one and it goes down by 1; the moment it reaches 0, the object is thrown away on the spot.

import sys

a = []
print(sys.getrefcount(a))
# 2
b = a
print(sys.getrefcount(a))
# 3
c = [a, a]
print(sys.getrefcount(a))
# 5
del b, c
print(sys.getrefcount(a))
# 2

• The first value is 2, not 1, because calling sys.getrefcount(a) also counts the reference passed in as an argument.
• c = [a, a] puts two tags into the list, so the count goes up by 2. Tags inside a bag are tags too.
• del b, c removes the name b, and as the list c is thrown away, the two tags inside it disappear with it.

Don't rely on the numbers themselves. Since 3.14, the interpreter applies an optimization that skips some reference count updates. Run the same code inside a function and the first value is 1 on CPython 3.14 but 2 on 3.13. What to look at is the difference: "1 per tag".

You can confirm that an object is thrown away the moment its count hits 0 with __del__, the dunder method called just before an object is discarded (why you shouldn't trust it comes in 11-4).

class Bag:
    def __init__(self, label):
        self.label = label
    def __del__(self):
        print("discard", self.label)

a = Bag("A")
b = a
del a
print("after del a")
del b
print("after del b")
# after del a
# discard A
# after del b

def check_in():
    tmp = Bag("T")
    print("inside")

check_in()
print("back")
# inside
# discard T
# back

• After del a, the tag b is still there, so nothing is thrown away. "discard A" is printed right at del b, which removes the last tag.
• When a function ends, its call box disappears and the local name tmp falls off with it. That is why the bag is discarded before "back".
• This is what the supervisor meant in Lesson 6. Even things from a finished function stay around if a closure's cell holds them, because their reference count never reaches 0.

The GC observatory below creates real objects in a worker process and runs the real gc. Each bag shows its "refs" count (name tags + tags inside other bags). A bag that cannot be reached by starting from any name tag and following the bags inside is drawn with a red dotted line. A dotted tag shows what it currently points to as w() → ….

11-2. Two bags holding each other: reference cycles and GC (12 min)

Piece the three records together and the truth comes out. The [[...]] on the Lesson 3 X-ray was a bag holding itself. In the Lesson 7 record, the two bags pointed to each other through their partner attributes. Turned straight into code, it looks like this.

del a, b removed every name tag. But inside A there is still a tag pointing to B, and inside B a tag pointing to A. Both have a reference count of 1 that will never reach 0. This is a reference cycle.

Objects that cannot be reached from any name are garbage: the program has no way to use them again. That is also why they vanished from the tag board. But reference counting alone cannot clear this garbage.

So CPython has a second cleaner: the cyclic garbage collector (the gc module).

• It examines objects that can hold other objects, such as lists, dicts and instances.
• Within a group of them, it counts references while leaving out the ones they exchange among themselves. A group with no references coming from outside is garbage that only holds itself up.
• It throws such groups away whole. gc.collect() makes it do this right now and returns the number of objects cleared.

import gc
gc.disable()   # switch off automatic GC for a moment so the example stays stable

class Bag:
    def __init__(self, label):
        self.label = label
        self.partner = None
    def __del__(self):
        print("discard", self.label)

a = Bag("A")
b = Bag("B")
a.partner = b
b.partner = a
del a, b
print("tags gone")
print(gc.collect() > 0)
gc.enable()
# tags gone
# discard A
# discard B
# True

• Nothing is thrown away after del a, b. "discard" is printed only inside gc.collect(). The order of those two lines is not guaranteed.
• Normally you don't need to call gc.collect(). When you create lots of objects, the interpreter runs it on its own. gc.disable() was used only for the example.
• When, and in how many generations, automatic GC runs differs between versions, and has even changed within 3.14. Don't write code that depends on a particular timing or number.

The list holding itself from Lesson 3 is the same case.

import gc

a = []
a.append(a)
print(a, a[0] is a)
# [[...]] True
del a
print(gc.collect())
# e.g.: 23

The number returned also counts any other garbage piled up by then, so it varies with the situation and the version. On this PC, the same file gave 1 on 3.13.9 and 23 on 3.14.2. The observatory below cleans up once when it starts, so it gives the same value every time.

Cycles get cleared eventually, so is everything fine? Two things differ.

• Reference counting discards an object on the very line its count hits 0, but the cyclic GC clears things later, all at once. Until then they keep occupying memory.
• If an object in a cycle holds a file or a connection, that is not closed until the GC runs either.

There are two ways to get rid of a cycle: remove one of the tags yourself when you are done (as in break), or make one side a weak reference from the start.

11-3. Dotted tags: weak references (8 min)

A weak reference is a dotted tag that does not count toward the reference count. It points to the bag but does not hold on to it. When the bag is thrown away, the dotted tag empties itself.

import weakref

class Bag:
    def __init__(self, label):
        self.label = label

a = Bag("A")
w = weakref.ref(a)
print(w() is a)
# True
del a
print(w())
# None

• You get the bag only by calling it, as in w(). If the bag is already gone, you get None, so always check before using it.
• Not every object can carry a dotted tag. weakref.ref([1, 2]) gives TypeError: cannot create weak reference to 'list' object. The same goes for int, str, tuple and dict, but instances of your own classes work.

The most common use of weak references is a cache. If you build a cache from an ordinary dict, the cache holds every object you put in forever, and memory never shrinks. WeakValueDictionary holds its values with dotted tags, so an entry disappears by itself once nothing else uses that object.

import weakref

class Bag:
    def __init__(self, label):
        self.label = label

desk = weakref.WeakValueDictionary()
c = Bag("C")
desk["C"] = c
print("C" in desk, len(desk))
# True 1
del c
print("C" in desk, len(desk))
# False 0

There is also WeakKeyDictionary, which holds its keys with dotted tags, and WeakSet, which holds its elements that way. Use them when you want to attach extra information to objects without extending their lifetime.

To register something to do when a bag is thrown away, use weakref.finalize.

import weakref

class Bag:
    def __init__(self, label):
        self.label = label

d = Bag("D")
f = weakref.finalize(d, print, "discard D")
print(f.alive)
# True
del d
# discard D
print(f.alive)
# False

finalize is called at most once, and if the bag is still alive when the program ends, it is called at exit. It is more dependable than __del__ in the next section. But be careful: if the function or arguments you pass hold on to the bag (passing d.close, for example), the bag will never be thrown away.

11-4. Don't trust __del__: clean up with with (6 min)

__del__ looks like a good place for cleanup code, but it cannot be trusted, for three reasons.

1. You don't know when it will be called. Inside a cycle, it is postponed until the GC runs. On other Python implementations that don't use reference counting, such as PyPy, it can be even later.
2. Exceptions raised inside it never come out. They are printed once to standard error and ignored.
3. For objects still alive when the program ends, there is no guarantee __del__ is called at all.

class Jam:
    def __del__(self):
        raise RuntimeError("belt jam")

x = Jam()
del x
print("still running")
# (stderr) e.g.: Exception ignored while calling deallocator <function Jam.__del__ at 0x...>:
# (stderr) ... RuntimeError: belt jam
# still running

Resources that must be closed, such as files, sockets and locks, are closed with with. The moment the block is left, __exit__ is called, even if an exception was raised (Lesson 8).

class Belt:
    def __enter__(self):
        print("belt on")
        return self
    def __exit__(self, *exc):
        print("belt off")
        return False

with Belt() as belt:
    print("moving")
# belt on
# moving
# belt off

try:
    with Belt():
        raise ValueError("stuck")
except ValueError as e:
    print("caught", e)
# belt on
# belt off
# caught stuck

Files returned by open() are used with with open(path) as f: too. The cleanup happens at a point you can see in the code.

11-5. Immortal objects, the weight of a bag, and 3.14t (5 min)

Objects used everywhere, such as None, True and small integers, are immortal objects (3.12, PEP 683). Their reference count is fixed, so however many tags you hang or remove, they are never thrown away.

import sys

print(sys.getrefcount(None))
# 3221225472
x = None
print(sys.getrefcount(None))
# 3221225472
print(sys._is_immortal(None), sys._is_immortal([]))
# True False

• The number is a marker value meaning "immortal", and it differs between versions. On CPython 3.14 it is 3221225472; on 3.13 it was 4294967295.
• sys._is_immortal was added in 3.14. Its name starts with an underscore, so it is a CPython implementation detail.

sys.getsizeof weighs only the bag itself. It does not count the contents.

import sys

print(sys.getsizeof([]), sys.getsizeof([1, 2, 3]))
# 56 88
big = [0] * 1000
print(sys.getsizeof(big), sys.getsizeof([big]))
# 8056 64

On 64-bit CPython, a list is a 56-byte empty body plus 8 bytes for each slot that holds a tag. Depending on how it was built, a list may reserve extra slots, so [1, 2, 3] is 88 rather than 80. [big] is a bag with one slot, so it is 64 bytes. Even though the big inside it weighs 8056 bytes, that is not counted.

A word on the free-threaded build (3.14t). In a build without the GIL, several threads change reference counts at the same time. So the thread that created an object counts its own share quickly, while other threads count a separate shared share atomically (biased reference counting). The visible rules are the same. Running this lesson's examples on 3.14.3t (where sys._is_gil_enabled() is False) gave the same reference counts, the same immediate disposal, and gc.collect() still cleared the cycles. The only difference was the number gc.collect() returned.

Case closed (10 min)

The two bags were a reference cycle, each holding the other through its partner. They had no tags, but their reference counts stayed at 1, so they were never thrown away and remained on the conveyor until the cyclic GC ran. The supervisor runs gc.collect(), and conveyor 13 is empty.

To keep it from happening again, have each bag remember its partner with a dotted tag. Wrapped in a property from Lesson 10, the code that uses it stays the same.

import weakref

class Bag:
    def __init__(self, label):
        self.label = label
        self._partner = None

    @property
    def partner(self):
        return self._partner() if self._partner is not None else None

    @partner.setter
    def partner(self, other):
        self._partner = weakref.ref(other)   # remember the partner with a dotted tag

a = Bag("A")
b = Bag("B")
a.partner = b
b.partner = a
print(a.partner.label, b.partner.label)
# B A
watch = weakref.ref(a)
del a
print(watch() is None, b.partner)
# True None

With the single line del a, A is thrown away immediately and B's partner becomes None. There is no cycle waiting for the GC in the first place.

Before the end of the shift, Yoon brings one more line from the records. Ghost bag B was stored in the lost-and-found lockers last month, and at the time nobody could get it out. The supervisor puts the locker key on her desk. "We'll look at that tomorrow. Last one."

Exercises

1. Predict sys.getrefcount(a) after a = [], a.append(a). What do you get at module level, and why isn't it 2?
2. Try putting a string as a value into weakref.WeakValueDictionary(). What error do you get, and what happens if you wrap it in an instance of your own class?
3. Write a class that keeps a file open and closes it in __del__, then change it to close the file with __enter__ and __exit__. Check that it is closed even when an exception is raised inside the block.