Sealed bags: mutable and immutable

At the morning briefing, Supervisor Do Miran holds up a seal sticker. "Nobody opens a bag with one of these on it. But today's report says a sealed bag got fuller."

The code from the duty-free inventory officer is short. Trying to put an item into a tuple raised a TypeError. Yet when they printed the tuple again, the item was inside. The supervisor says, "The seal is on the outer bag. Nobody sealed the bag inside it."

2-1. Sealed bags and open bags (9 min)

Objects fall into two groups. If you can change their contents after creating them, they are mutable; if you cannot, they are immutable.

Immutable objects simply have no way to change their contents.

word = "gate"
print(word.upper(), word)
# GATE gate
word[0] = "G"
# TypeError: 'str' object does not support item assignment

fs = frozenset({1, 2})
fs.add(3)
# AttributeError: 'frozenset' object has no attribute 'add'

word.upper() does not modify word. It builds a new string in upper case and returns it. Every string method works this way.

So what is n = n + 1? It does not modify the bag holding the integer 10. It gets a new integer object, 11, and moves the tag n onto it. "Changing" an immutable object is always a new object + rebinding.

The tag board below draws integers as bags of their own too. Watch where the arrow of n goes after line 3.

2-2. The two faces of += (10 min)

On the surface, a += b does one thing. But Python handles it like this.

1. If the type of a has an __iadd__ (in-place add) method, Python calls it. The bag is changed in place and the same bag is returned.
2. If not, it works like a = a + b: __add__ builds a new object and the tag moves.

print(hasattr(list, "__iadd__"), hasattr(int, "__iadd__"))
# True False
print(hasattr(str, "__iadd__"), hasattr(tuple, "__iadd__"))
# False False

Lists are mutable, so they have __iadd__. Immutable types do not; a sealed bag cannot be changed in place. That is why the same += splits like this.

a = [1, 2]
b = a
a += [3]
print(a, b, a is b)
# [1, 2, 3] [1, 2, 3] True

a = (1, 2)
b = a
a += (3,)
print(a, b, a is b)
# (1, 2, 3) (1, 2) False

a = "T1"
b = a
a += "-A"
print(a, b, a is b)
# T1-A T1 False

This is why a = a + [3] from Lesson 1 and a += [3] on a list give different results. The first makes a new list; the second keeps the same list. |= on a set and |= on a dict also change the object in place, just like lists.

Switch between the three types and run each one.

2-3. The seal is only on the outside: a list inside a tuple (8 min)

A tuple being immutable means precisely this: "the references a tuple holds cannot be changed". Which object each slot points to is fixed. But if a slot points to a list, the list itself can change as much as it likes.

t = (1, [2])
t[1].append(3)
print(t)
# (1, [2, 3])
t[1] = [9]
# TypeError: 'tuple' object does not support item assignment

The reporter's code used += instead of append. That is where something strange happens.

t = (1, [2])
try:
    t[1] += [3]
except TypeError as e:
    print("TypeError:", e)
print(t)
# TypeError: 'tuple' object does not support item assignment
# (1, [2, 3])

An error was raised, yet the list changed. That is because t[1] += [3], written out, is two steps.

t = (1, [2])
tmp = t[1]
tmp = tmp.__iadd__([3])   # step 1: change the list in place (succeeds)
print(t)
# (1, [2, 3])
t[1] = tmp                # step 2: put it back into the tuple slot (fails)
# TypeError: 'tuple' object does not support item assignment

By step 1 the list has already changed. The slot assignment in step 2 fails and raises an error, but there is no way to undo step 1. That is why the scale cried "a sealed bag cannot be opened" while the item went in anyway.

There is a simple way to check whether a bag is sealed all the way down: see whether hash() works. A tuple that can be hashed contains only objects that can be hashed too, and those are usually immutable (Lesson 12).

print(hash(("passport", ("snack",))) == hash(("passport", ("snack",))))
# True
hash(("passport", ["snack"]))
# TypeError: unhashable type: 'list'

2-4. The cost of joining strings with += (6 min)

Strings are immutable, so in principle s += "x" builds a new string every time. If you build a string of length n one character at a time, the number of characters copied grows to 1 + 2 + … + n.

def plus(n):
    s = ""
    for i in range(n):
        s += "x"
    return s

def plus_shared(n):
    s = ""
    for i in range(n):
        last = s          # the same string carries one more tag
        s += "x"
    return s

def join(n):
    return "".join(["x" for i in range(n)])

print(plus(5) == plus_shared(5) == join(5))
# True

In CPython, when a string has only one tag, += quietly applies an optimization and grows it in place. That makes plus faster than you might expect. But a single extra tag is enough to switch the optimization off. Here are example timings from this PC (CPython 3.14.2).

When the count doubled, plus_shared took almost four times as long. That is the amount of copying growing with the square. The optimization is not something the language promises, so when you collect strings in a loop, put the pieces in a list and join them once with "".join().

2-5. Small integers and interning: why not to compare values with is (7 min)

Immutable objects never change, so when values are equal it is safe for many names to share one object. CPython takes advantage of this and creates frequently used objects ahead of time.

The small-integer cache. In CPython, the integers from -5 to 256 are created at startup, and the same object is always returned for them.

print(int("256") is int("256"), int("257") is int("257"))
# True False
print(int("-5") is int("-5"), int("-6") is int("-6"))
# True False

Creating the integers at run time with int("…") is deliberate. If you write literals directly in the code, like 257 is 257, the compiler may merge identical constants into one and give True. Python even warns about such comparisons.

x = 256
print(x is 256)
# SyntaxWarning: "is" with 'int' literal. Did you mean "=="?
# True

String interning. CPython keeps string constants that look like names as one shared object (interning). Strings built at run time can be different objects even when their values are equal. With sys.intern() you can make equal strings share one object.

import sys
a = "gate"
b = "".join(["ga", "te"])
print(a == b, a is b)
# True False
print(sys.intern(b) is a)
# True

Since the result depends on caching and interning, comparing values with is works for some values and fails for others. Always compare values with ==, and use is only for one-of-a-kind objects like None.

Case closed (10 min)

Here is the reporter's code, and how to make a bag that truly cannot be opened.

sealed = ("passport", ["snack"])
try:
    sealed[1] += ["perfume"]
except TypeError as e:
    print("scale:", e)
print(sealed)
# scale: 'tuple' object does not support item assignment
# ('passport', ['snack', 'perfume'])

sealed = ("passport", ("snack",))   # make the inside a tuple too
try:
    sealed[1] += ("perfume",)
except TypeError as e:
    print("scale:", e)
print(sealed)
# scale: 'tuple' object does not support item assignment
# ('passport', ('snack',))

• The first bag was sealed only on the outside. The inner list's __iadd__ succeeded first, and only the slot assignment failed.
• In the second bag the inside is a tuple as well. A tuple has no __iadd__, so Python only builds a new tuple and then fails at the slot assignment. The original bag is untouched.

The supervisor puts a new seal sticker on the inner bag too. "A seal doesn't stop at the outside. You have to check the bags inside."

Exercises

1. After running x = 1000, y = x, x += 1, what is y? Draw the tag picture, then check.
2. If you run s = {1}, t = s, s |= {2}, what happens to t? Try it again with frozenset({1}).
3. With t = ([],), how do t[0] += [1] and t[0].extend([1]) each end? Print t afterwards as well.