Determinants and inverses

In an image editor, if you rotate or stretch a photo and don't like it, you press Ctrl+Z to undo. Is there an undo for linear transformations too? If you rotated by 90°, rotate by −90°; if you stretched sideways by 2, shrink by half. The matrix that undoes a transformation like this is the inverse.

Some transformations, though, cannot be undone. Squash the whole plane onto a single line, like Lesson 5's 'Squash', and different points pile up on the same spot, so there is no way to know where they originally were. It is like smearing a photo into a single line: you cannot bring the original photo back.

The single number that tells you whether a transformation can be undone is the determinant. It is the same det you briefly met in Lesson 5 as the "area factor". In this lesson you will understand det through pictures, derive the 2 × 2 determinant and inverse formulas yourself, and then compute them with NumPy.

7-1. Determinant = area factor (10 min)

Think back to the blue cell in the Lesson 5 lab. The square cell of area 1, made by î and ĵ, became a parallelogram after the transformation. A linear transformation turns every grid cell into exactly the same shape, so once you know how many times one cell's area has grown, you know how many times the area of any shape in the plane grows. A circle or a drawing of a cat: chop it into small grid cells and each cell grows by the same factor.

This area factor is the determinant, written det A or |A| in books.

The lab below is the same screen as in Lesson 5, with an inverse box and an 'Undo with A⁻¹' button added. Under det A on the right, a message tells you how many times areas grow.

Start with the easy case. If $\hat{\imath}$ goes to $(a, 0)$ and $\hat{\jmath}$ goes to $(0, d)$, the cell is a rectangle $a$ wide and $d$ tall, so its area is $ad$. If the cell only tilts, as with 'Shear', the base and height stay the same, so the area stays the same too.

For the general case, take the rectangle that encloses the picture and subtract the leftover pieces. Let's use the $\hat{\imath} \to (2, 1)$, $\hat{\jmath} \to (1, 3)$ you made in the lab.

The parallelogram's four corners are $(0, 0)$, $(2, 1)$, $(3, 4)$ and $(1, 3)$. The rectangle around it is $a + b = 3$ wide and $c + d = 4$ tall, with area 12. Take the parallelogram out of the rectangle and six pieces are left over, in pairs of the same shape.

So the parallelogram's area is $12 - 2 - 3 - 2 = 5$. Doing the same calculation with letters gives this.

This is the $2 \times 2$ determinant formula. Remember it as multiply along the diagonals and subtract.

Reading the book's notation

The vertical bars in $\lvert A \rvert$ look just like the absolute value sign for numbers, but around a matrix they mean the determinant, which can be negative. Since that is easy to mix up, this course writes $\det A$.

7-2. What the sign and zero mean (6 min)

An area cannot be negative, but det can. In the Lesson 5 lab, 'Flip' had det −1, and the blue cell turned yellow.

A negative det means the orientation has flipped. At the start, ĵ sits 90° counterclockwise from î. If after the transformation ĵ is still on the counterclockwise side of î, det is positive; if it has crossed over to the clockwise side, det is negative. It means front and back have swapped, as if you turned the paper over and looked at it from behind. The size of det (its absolute value) is still the area factor. det = −2 means "areas double, and orientation flips".

A det of 0 means the plane has been squashed. The area is multiplied by 0, so the cell has been pressed into a line segment or a point. This means column 1 and column 2 lie on one line, which is the same as linear dependence from Lesson 4. For 'Squash', column 2 (2, 1) is exactly twice column 1 (1, 0.5). ad − bc = 1 × 1 − 2 × 0.5 = 0.

In the lab, turn off 'Snap to grid' and slowly drag the tip of ĵ toward î and past it. The cell gets thinner and thinner, det becomes 0 at the moment î and ĵ lie on one line, and once ĵ crosses over the cell turns yellow and det goes negative. det never jumps from one sign to the other; it always passes through 0.

It is interesting that 'Stretch' changes the shape so much yet leaves the area unchanged. It shrinks vertically by as much as it stretches horizontally. det looks only at area, not at shape.

7-3. The determinant of a product is the product of the determinants (4 min)

In Lesson 6, $AB$ meant "first $B$, then $A$". $B$ multiplies areas by $\det B$, then $A$ multiplies the result by $\det A$, so overall areas change by the product of the two factors.

For example, a transformation that makes areas 5 times larger followed by one that makes them 6 times larger gives 30 times. The swapped $BA$ has the same area factor too. Lesson 6 said $AB$ and $BA$ are usually different matrices, but their determinants are always the same, because multiplying numbers doesn't care about order.

One fact follows straight from this property. To undo a transformation that made areas 5 times larger, you have to make areas $\tfrac{1}{5}$ as large. So the determinant of the inverse is the reciprocal of the original determinant, and if $\det A = 0$ there is no factor that undoes "times 0", so the transformation cannot be undone.

7-4. Inverse = undo (9 min)

The transformation that puts back what $A$ moved is called the inverse of $A$, written $A^{-1}$. Doing $A$ and then $A^{-1}$ is the same as doing nothing, so it gives the identity matrix $I$ from Lesson 6. The other order works too.

Press 'Undo with A⁻¹' in the lab and the grid slowly goes from the current $A$ back to the original checkerboard. A moment later it returns to how $A$ looks, so press it a few times. For the ready-made matrices you can guess the inverse straight from the picture.

The 2 × 2 inverse formula

For a $2 \times 2$ matrix you can get the inverse straight from a formula. Swap the two numbers on the diagonal, flip the sign of the other two, then divide everything by det.

Let's apply it to the $A$ from 7-1. Since $\det A = 5$, we get this.

Multiply to check that it is right. Use Lesson 6's "dot product of a row and a column".

That is $I$ times 5, so multiplying by $\tfrac{1}{5}$ gives exactly $I$. The formula divides by $ad - bc$, so if $\det A = 0$ you would divide by 0 and the formula breaks down. The "cannot be undone" you saw in the picture in 7-3 shows up in the formula too.

Why a squashed plane cannot be undone

Move the two points $(2, 0)$ and $(0, 1)$ with the 'Squash' matrix and both land on $(2, 1)$. Combining columns as in Lesson 5, $(2, 0)$ is "two of column 1" $= (2, 1)$ and $(0, 1)$ is "one of column 2" $= (2, 1)$. When an undo transformation receives $(2, 1)$, it cannot decide whether to send it to $(2, 0)$ or to $(0, 1)$. The moment the plane is squashed, information is lost, and no matrix can undo it.

Undoing transformations done in sequence

To undo "first $B$, then $A$", you have to undo the last one first. If you put on socks and then shoes, you take off the shoes first.

$A^{-1}$ is on the right, so it happens first, and then $B^{-1}$. Lesson 6's "read from the right" holds here as well.

7-5. Determinants and inverses in NumPy (12 min)

The lab's NumPy panel shows np.round(np.linalg.det(A), 10), A_inv = np.linalg.inv(A), A_inv and A @ A_inv in turn, and when det is 0 it shows an error along with a comment saying there is no inverse. Let's go through them one at a time.

np.linalg.det: an integer matrix, yet a decimal comes out

>>> import numpy as np
>>> A = np.array([[2, 1], [1, 3]])
>>> np.linalg.det(A)
np.float64(5.000000000000001)

By hand, 2 × 3 − 1 × 1 = 5, but out came 5.000000000000001. NumPy's det does not use the ad − bc formula. It uses a method that works for matrices of any size (LU decomposition, which reshapes the matrix into a triangular form). That process involves division, which lets in the floating-point error from Lesson 3. Even for an integer matrix the result is always a real number (float64), and you should expect the last digits to be slightly off.

That is why the lab's NumPy panel shows det as np.round(np.linalg.det(A), 10): to wipe away that residue. Rounding to 10 decimal places with the np.round you saw in Lesson 5 makes the error in the last digits disappear. Leave out np.round and run just np.linalg.det(A), and you get a value with slightly-off last digits like the one above.

>>> np.round(np.linalg.det(A), 10)
np.float64(5.0)

To round it as a Python number, or to compare without rounding, do this.

>>> round(float(np.linalg.det(A)), 10)
5.0
>>> np.isclose(np.linalg.det(A), 5)
np.True_

float(...) turned the NumPy number into a Python number, and then round rounded it. To compare, use Lesson 3's np.isclose instead of ==. np.True_ is NumPy's true value, and you can use it in an if statement exactly like Python's True. Lesson 6's np.allclose returns Python's True/False, which is why it looked slightly different. A 3 × 3 matrix goes into np.linalg.det the same way. In that case it is a volume factor instead of an area factor. A matrix that stretches width, depth and height by 2, 3 and 4 makes volumes 24 times larger.

>>> D = np.array([[2, 0, 0],
...               [0, 3, 0],
...               [0, 0, 4]])
>>> np.linalg.det(D)
np.float64(23.999999999999993)

The last digits are off here too, even for an easy matrix that is 0 everywhere off the diagonal. Always take a det value as "almost that value".

Checking det(AB) = det(A) det(B)

>>> B = np.array([[2, 0], [0, 3]])
>>> np.linalg.det(A @ B)
np.float64(29.99999999999999)
>>> np.linalg.det(A) * np.linalg.det(B)
np.float64(30.000000000000007)
>>> np.isclose(np.linalg.det(A @ B), np.linalg.det(A) * np.linalg.det(B))
np.True_

One is a little under 30 and the other a little over, so the two values differ in the last digits, but np.isclose says they are the same. When you work with floating point, the habit that matters is asking "are they close enough?" rather than "are they exactly equal?".

np.linalg.inv: the inverse

>>> A_inv = np.linalg.inv(A)
>>> A_inv
array([[ 0.6, -0.2],
       [-0.2,  0.4]])

This matches what you worked out by hand in 7-4. Now try the last line of the lab's NumPy panel, A @ A_inv. In theory it should be exactly I.

>>> A @ A_inv
array([[ 1.00000000e+00,  0.00000000e+00],
       [-5.55111512e-17,  1.00000000e+00]])

In the bottom left, a residue of -5.55111512e-17, about 0.0000000000000000555, is left over. That is because 0.6 and 0.2 are not exactly 0.6 and 0.2 inside the computer. With numbers of very different sizes mixed together, NumPy shows every slot in e notation, as with Lesson 6's rot(90). 1.00000000e+00 is just 1. Check whether the result is I with np.allclose.

>>> np.allclose(A @ A_inv, np.eye(2))
True
>>> np.allclose(A_inv @ A, np.eye(2))
True
>>> np.allclose(np.linalg.inv(A @ B), np.linalg.inv(B) @ np.linalg.inv(A))
True

The last line is 7-4's "take off the shoes first", (AB)⁻¹ = B⁻¹A⁻¹.

Press 'Rotate 90°' in the lab and you will see -0. in A_inv in the NumPy panel. You get the same if you try it yourself.

>>> np.linalg.inv(np.array([[0, -1], [1, 0]]))
array([[ 0.,  1.],
       [-1., -0.]])

-0. is a "negative zero" that exists only in floating point. It appears when 0 is multiplied by a negative number during a calculation, but as a value it is exactly the same as 0: -0.0 == 0.0 is True. You don't need to worry about it. The matrix itself is [[0, 1], [−1, 0]] from the table in 7-4, a 90° turn the other way.

When there is no inverse: LinAlgError

Use inv on a matrix with det 0 and you get an error. It is the same error the NumPy panel shows when you press 'Squash' in the lab.

>>> F = np.array([[1, 2], [0.5, 1]])
>>> np.linalg.det(F)
np.float64(0.0)
>>> np.linalg.inv(F)
Traceback (most recent call last):
  ...
numpy.linalg.LinAlgError: Singular matrix

A singular matrix is a matrix that has no inverse. LinAlgError is the error the np.linalg functions raise to say "this matrix can't be computed with". Let's also check 7-4's "two points pile up on one point".

>>> F @ np.array([2, 0])
array([2., 1.])
>>> F @ np.array([0, 1])
array([2., 1.])

If your program handles matrices that might have no inverse, catch this error with try/except.

>>> try:
...     np.linalg.inv(F)
... except np.linalg.LinAlgError:
...     print("This transformation cannot be undone")
This transformation cannot be undone

7-6. Common mistakes (3 min)

Testing det with == 0. Because of floating-point error, a matrix whose det is 0 in theory can come out as a tiny nonzero number. In the matrix below, the second row is exactly 10 times the first, so it is a squashing matrix, but 0.1 and 0.3 are not exact inside the computer, so det does not come out as 0.

>>> G = np.array([[0.1, 0.3], [1, 3]])
>>> np.linalg.det(G) == 0
np.False_
>>> np.isclose(np.linalg.det(G), 0)
np.True_
>>> bool(np.abs(np.linalg.inv(G)).max() > 1e15)
True

The last line is the more troubling one. Because det is not exactly 0, inv raises no error and returns an "inverse" full of absurdly large numbers, over 10¹⁵. np.abs(...) takes the absolute value of every slot, and .max() takes the largest of them. A matrix whose det is very close to 0 should be treated as one that, in practice, cannot be undone. The proper tool for measuring "how close is dangerous" (the condition number) is covered in the next course, numerical analysis.

Mistaking 1 / A for the inverse. In NumPy, 1 / A and A ** -1 are element-wise calculations that take the reciprocal of each slot. The inverse is always np.linalg.inv(A).

>>> 1 / A
array([[0.5       , 1.        ],
       [1.        , 0.33333333]])

This is completely different from the A_inv in 7-5. As with Lesson 6's * and @, always keep element-wise calculations and matrix calculations apart.

Solving a system of equations with inv. You could solve the next lesson's Ax = b as np.linalg.inv(A) @ b, but in practice you use np.linalg.solve. Lesson 8 explains why.

Exercises (6 min)

1. Work out the determinant of A = [[4, 2], [1, 3]] by hand with ad − bc, and say how many times areas grow. Check with NumPy too.
2. Find the inverse of the A from question 1 with the 2 × 2 formula, then compare it with np.linalg.inv.
3. What is the det of the matrix that sends î to (0, 2) and ĵ to (1, 0)? Drag it in the lab and check by the color of the cell.
4. If det(A) = 3 and det(B) = −2, what are det(AB) and det(A⁻¹)?

Solutions

>>> P = np.array([[4, 2], [1, 3]])
>>> np.linalg.det(P)
np.float64(10.000000000000002)
>>> np.linalg.inv(P)
array([[ 0.3, -0.2],
       [-0.1,  0.4]])
>>> np.allclose(np.linalg.inv(P), np.array([[3, -2], [-1, 4]]) / 10)
True
>>> np.linalg.det(np.array([[0, 1], [2, 0]]))
np.float64(-2.0)

For question 1: 4 × 3 − 2 × 1 = 10, so areas grow 10 times. NumPy shows 10.000000000000002, with error in the last digits.

For question 2: swap the diagonal 4 and 3, flip the signs of 2 and 1 to get [[3, −2], [−1, 4]], and divide by the det, 10.

For question 3: 0 × 0 − 1 × 2 = −2. Areas double and the orientation flips, so the cell in the lab turns yellow (it is the same matrix as in Lesson 5, exercise 3).

For question 4: det(AB) = 3 × (−2) = −6, and det(A⁻¹) = 1/3.

In the next lesson you will solve the classic place where inverses are used, the system of equations Ax = b. You will find where two lines cross using a matrix, and see in pictures why, when det is 0, there is either no solution or infinitely many.