A matrix is a rule for moving space

When a game turns a character sprite by 90°, when a paint program tilts a photo, when a map app zooms in, every point on the screen moves to a new place at once. Even with millions of points, the computer does it in an instant. That is because it does not keep a separate rule for each point: a single rule made of four numbers moves them all.

That four-number rule is a matrix. Books usually introduce a matrix first as "a rectangular table of numbers", but this course looks at it differently. A matrix is a rule for moving the whole of space, and the table of numbers is just the note where that rule is written down.

This lesson is the center of the whole course. The one picture you build here, "the columns of a matrix are where î and ĵ land", keeps coming back in matrix multiplication, determinants, inverses, eigenvalues and all the way to SVD. Take your time and play with the lab as much as you like.

5-1. What it means to move space (6 min)

A transformation is a function that takes a vector in and gives a vector out. In Python terms it is something like def T(v): return new_vector. If you think of each point on the plane as the tip of an arrow, a transformation becomes a rule that moves every point of the plane to a new place.

You can make up all sorts of transformations. You could bend the plane into waves, or suck everything into a single point. Linear algebra deals only with very well-behaved ones, called linear transformations. Picture a grid drawn on graph paper and you can check the conditions with your own eyes.

Rotating, stretching, shearing (pushing sideways at a slant) and mirror-flipping all keep these conditions. On the other hand, "move every point 1 step right" (a translation) is not a linear transformation, because the origin moves to (1, 0). Neither is anything that bends the grid lines.

In return for being so strict, linear transformations have one remarkable property. Once you know where two points go, where every other point goes is settled automatically. The next section introduces those two points.

5-2. Just follow î and ĵ (10 min)

In Lesson 4 you saw that every vector in the plane can be written as a linear combination of the standard basis $\hat{\imath} = (1, 0)$ and $\hat{\jmath} = (0, 1)$. For example, $\mathbf{v} = (2, 1)$ is "two $\hat{\imath}$'s and one $\hat{\jmath}$".

Now suppose a linear transformation $T$ moves the whole plane. Since the grid lines stay parallel and evenly spaced, the relationship "two $\hat{\imath}$'s and one $\hat{\jmath}$" survives the move unchanged. The only thing that changes is where $\hat{\imath}$ and $\hat{\jmath}$ point.

So if you know $T(\hat{\imath})$ and $T(\hat{\jmath})$, that is, where the two basis vectors land, you can move any vector at all. Each landing spot is two numbers, so four numbers in total. Write where $\hat{\imath}$ lands as the first column and where $\hat{\jmath}$ lands as the second column, both standing upright, and what you have written is a matrix.

The left column $a, c$ is column 1 (where $\hat{\imath}$ lands), and the right column $b, d$ is column 2 (where $\hat{\jmath}$ lands).

Here is the most important sentence in this course. The columns of a matrix are where î and ĵ land. In Lesson 1 we said that the reason books stack a vector's numbers vertically would become clear in Lesson 5. This is it: stand each landing spot up as a column, put the columns side by side, and you have a matrix.

In the lab below, drag the tip of the teal arrow î and the tip of the crimson arrow ĵ. You only move two arrows, yet the whole grid bends along with them, and the numbers in column 1 (teal) and column 2 (crimson) of the matrix on the right change to exactly the coordinates of the tips. The cell shaded blue is the original square cell (the one made by î and ĵ) after it has been moved.

Press the 'Ready-made matrices' buttons in turn and read off where î and ĵ go, and you will find there is nothing to memorize. You just write the landing spots down as columns.

The "Matrix" column is written the way you type it into NumPy (one row at a time). We come back to this in 5-5. Take 'Rotate 90°' as an example: î = (1, 0) turns to point up, to (0, 1), and ĵ = (0, 1) turns to point left, to (−1, 0). Stand these two up and put them side by side, and column 1 is (0, 1) and column 2 is (−1, 0).

'Play the change from I to A' starts from the do-nothing matrix I (Identity) and slowly moves the grid to the current matrix A. All the way through, the grid lines stay straight, parallel and evenly spaced. It is the best way to see the three conditions of a linear transformation with your own eyes.

5-3. Two ways to read Av (8 min)

The result of moving a vector $\mathbf{v}$ with a matrix $A$ is written $A\mathbf{v}$ and read "A times v". There are two ways to compute it, and they always give the same answer. Let's try both with the Shear matrix and $\mathbf{v} = (2, 1)$.

Way ①: combining columns. This is exactly what you saw in 5-2. $\mathbf{v}$ is "two $\hat{\imath}$'s and one $\hat{\jmath}$", so after the move it is "two of column 1 and one of column 2". It is the same calculation as M @ [a, b] being a·u + b·w in Lesson 4.

Way ②: dot products with the rows. The first component of the result is the dot product of the first row of $A$ with $\mathbf{v}$, and the second component is the dot product of the second row with $\mathbf{v}$. This is Lesson 3's "multiply matching components and add".

Write it out for a general $2 \times 2$ matrix and you can see the two ways are the same expression, just grouped in a different order.

The two ways have different uses. Way ①, combining columns, is better for picturing what happens ("chain column 1 x times, then column 2 y times"); way ②, dot products with the rows, is better for working out numbers by hand. This course uses ① wherever a picture is needed.

In the lab, press 'Shear' under 'Ready-made matrices' and turn on 'Follow v = (2, 1)'. The dashed teal arrow is column 1 taken twice, the dashed crimson arrow attached to its tip is column 2 taken once, and the thick black arrow is their sum, Av. Its tip lands on (3, 1). Drag the tips of î and ĵ to change the matrix, and Av always follows to the spot "two of column 1 + one of column 2".

Reading the book's notation

Entry numbers always go row first, column second. And as Lesson 1 said, books count from 1 and Python counts from 0. The book's $a_{12}$ is Python's A[0, 1].

5-4. The other numbers in the lab (4 min)

Besides the matrix, the right side of the lab shows a few more numbers and messages. For now just get a light sense of what they mean; later lessons go into detail.

det A and the area factor. The blue cell is the square cell of area 1 after it has been moved. How many times larger its area has become is the determinant (det). Since every grid cell changes in exactly the same way, every shape in the plane has its area multiplied by det. Press the buttons and a few things stand out.

• 'Stretch' is twice as wide and half as tall, so the area is 2 × 0.5 = 1: unchanged.
• 'Shear' also leaves the area unchanged. The cell only tilts; its base and height stay the same.
• 'Flip' has det −1, and the cell turns yellow. A negative value means "it has been flipped over".
• 'Squash' has det 0. Column 1 (1, 0.5) and column 2 (2, 1) lie on one line (linearly dependent, as in Lesson 4), so the whole plane gets pressed onto a single line.

How to compute the determinant, and why det 0 means the move cannot be undone, are covered in detail in Lesson 7.

The eigenvector dashed lines. With 'Show eigenvectors' on, you will sometimes see gray dashed lines. These are directions that do not lose their heading under the transformation and stay on the same line. With 'Stretch', the horizontal and vertical axes become dashed lines. When every direction gets turned, as with 'Rotate 90°', a message says there is no such direction. These directions are eigenvectors, the stars of Lesson 11; for now it is enough to note "so there are such things" and move on.

5-5. Working with matrices in NumPy (12 min)

In the "Check it with NumPy" panel below the lab, the very first line builds the matrix, like A = np.array([[1, 0], [0, 1]]). Let's read through the panel starting with that line.

Making a 2-D array: write it one row at a time

In NumPy a matrix is made from a list of lists. Each inner list is one row.

>>> import numpy as np
>>> A = np.array([[1, 1], [0, 1]])
>>> A
array([[1, 1],
       [0, 1]])
>>> A.shape
(2, 2)
>>> A.ndim
2

The (2, 2) in shape means "2 rows, 2 columns". The vectors of Lesson 1 had a single number like (2,) and an ndim of 1; a matrix has two axes (along the rows and along the columns), so its ndim is 2.

There is a trap here you must not skip. The lab told you to think in columns (column 1 = where î lands), but in NumPy you write rows. 'Shear' is the transformation that sends î to (1, 0) and ĵ to (1, 1), yet in NumPy you write [[1, 1], [0, 1]]. The first row [1, 1] is the top of both columns read across. Read the printed result downward and the columns appear: the left column 1, 0 is where î lands, and the right column 1, 1 is where ĵ lands.

If you want to build a matrix straight from the landing spots, np.column_stack from Lesson 4 is the safest way. As the name says, it stacks vectors as columns.

>>> i_to = np.array([1, 0])
>>> j_to = np.array([1, 1])
>>> np.column_stack([i_to, j_to])
array([[1, 1],
       [0, 1]])

Taking out entries, rows and columns

Let's tie the indexing from Lesson 4 to what a matrix means. A[row, column] is one entry, A[:, 0] is the first column, and A[1] is the second row. : means "all of that axis".

>>> A[0, 1]
np.int64(1)
>>> A[:, 0]
array([1, 0])
>>> A[:, 1]
array([1, 1])
>>> A[1]
array([0, 1])

A[:, 0] is "slot 0 of every row", that is, the first column. That is where î lands, (1, 0), and A[:, 1] is where ĵ lands, (1, 1). A[1], on the other hand, is the second row, a bundle of numbers that has nothing to do with the landing spots. When you want a column, always use A[:, column].

A @ v: moving a vector with a matrix

Multiplying a matrix by a vector uses the same @ operator as the dot product in Lesson 3. The last line of the lab's NumPy panel, A @ np.array([2, 1]), is exactly the black arrow Av from 'Follow v = (2, 1)'.

>>> v = np.array([2, 1])
>>> A @ v
array([3, 1])

Write the two ways from 5-3 directly in NumPy and you get the same answer.

>>> v[0] * A[:, 0] + v[1] * A[:, 1]
array([3, 1])
>>> np.array([A[0] @ v, A[1] @ v])
array([3, 1])

The first line is ① "add column 1 twice and column 2 once", and the second line is ② "the dot product of each row with v". In real code you never spell A @ v out like this, but checking once what @ does inside lets you trust the result.

Moving many points at once

To tilt a square sprite in a game, you have to move all four of its corners. Put the corners side by side as columns of a matrix and a single @ moves them all.

>>> square = np.array([[0, 1, 1, 0],
...                    [0, 0, 1, 1]])
>>> square.shape
(2, 4)
>>> A @ square
array([[0, 1, 2, 1],
       [0, 0, 1, 1]])

The four columns of square are the corners of the square, (0, 0), (1, 0), (1, 1), (0, 1), and the four columns of the result are where they moved to, (0, 0), (1, 0), (2, 1), (1, 1). These are the same as the four corners of the blue cell when you pressed 'Shear' in the lab. Multiplying a matrix by a matrix like this is the topic of Lesson 6.

A * v is not Av

In NumPy, * is the element-wise product you saw in Lesson 2. Multiply a matrix by a vector with * and broadcasting kicks in: v is multiplied element by element into each row.

>>> A * v
array([[2, 1],
       [0, 1]])

The result is a (2, 2) matrix, so you can tell right away that it is not Av. Each row, [1, 1] and [0, 1], was multiplied element by element by [2, 1]. Read it downward and column 1 has been doubled and column 2 kept once, so what you got are the pieces of ① combining columns "just before adding". Only after you add the two columns do you get Av.

>>> (A * v)[:, 0] + (A * v)[:, 1]
array([3, 1])

Use @ to move a vector with a matrix, and * to multiply slot by slot. Lesson 6 stresses this distinction once more.

A.T: the transpose

The matrix with its rows and columns swapped is called the transpose, written Aᵀ in books and A.T in NumPy. The first row becomes the first column, and the first column becomes the first row.

>>> A.T
array([[1, 0],
       [1, 1]])
>>> np.array([[1, 0], [1, 1]]).T
array([[1, 1],
       [0, 1]])

As the second example shows, you can also write the landing spots (1, 0) and (1, 1) as rows and then flip them with .T to get the same matrix. The transpose is used in earnest in Lesson 13 (least squares) and Lesson 15 (PCA).

The other lines in the NumPy panel

The lab's NumPy panel also has lines you have not learned yet. Just learn their names for now.

np.linalg is NumPy's toolbox for linear algebra. Most of the functions you will meet from here on live in it.

np.round(value, digits) rounds a number, or every slot of an array, to that many decimal places. The lab rounds some values this way to hide the tiny errors that creep into calculations (you will see them up close in Lessons 6 and 7), and it writes the rounding into the line so you can see it. If you follow along yourself and leave out np.round(...), the last digits may come out slightly different.

>>> np.round(3.14159, 2)
np.float64(3.14)

5-6. Common mistakes (3 min)

Writing columns as rows. Hear "î goes to (1, 0) and ĵ goes to (1, 1)" and type np.array([[1, 0], [1, 1]]), and you get not 'Shear' but its transpose, a completely different matrix. When you want to write the landing spots as they are, use np.column_stack, or get into the habit of checking column 1 with A[:, 0] after you write the matrix.

Using * instead of @. As you saw in 5-5, A * v returns something without any error, so the mistake is hard to notice. If the result's shape is (2, 2) instead of (2,), be suspicious.

Multiplying by a vector of the wrong shape. A matrix with 2 columns can only be multiplied by a vector with 2 components. You need exactly two numbers: "how many of column 1, how many of column 2".

>>> A @ np.array([1, 2, 3])
Traceback (most recent call last):
  ...
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 3 is different from 2)

The message is long, but you only need to read the end, (size 3 is different from 2). It means "the vector's length, 3, differs from the matrix's number of columns, 2".

Exercises (7 min)

1. Where does the upside-down flip (mirroring across the horizontal axis) send î and ĵ? Make the matrix as a NumPy array F, predict the result of F @ np.array([2, 1]) first, then check.
2. For B = np.array([[2, -1], [1, 1]]), where do î and ĵ land? Compute B @ np.array([1, 2]) by hand with ① combining columns and ② dot products with the rows, then check with NumPy.
3. In the lab, drag î to (0, 2) and ĵ to (1, 0). How should you write this in NumPy? Also think about why the cell turns yellow.
4. Is "stretch every point by 2, then move it 1 step up" a linear transformation?

Solutions

>>> F = np.array([[1, 0], [0, -1]])
>>> F @ np.array([2, 1])
array([ 2, -1])
>>> B = np.array([[2, -1], [1, 1]])
>>> B[:, 0], B[:, 1]
(array([2, 1]), array([-1,  1]))
>>> B @ np.array([1, 2])
array([0, 3])
>>> np.column_stack([[0, 2], [1, 0]])
array([[0, 1],
       [2, 0]])

For question 1: î = (1, 0) stays where it is and ĵ = (0, 1) goes to (0, −1). The space before the 2 in the output [ 2, -1] is there because NumPy lines the numbers up with −1.

For question 2: î goes to (2, 1) and ĵ goes to (−1, 1). ① 1·(2, 1) + 2·(−1, 1) = (0, 3), and ② (2·1 + (−1)·2, 1·1 + 1·2) = (0, 3). The output is wrapped in parentheses because we showed both columns at once as a tuple.

For question 3: the landing spots are the columns, so it is [[0, 1], [2, 0]]. Normally ĵ sits 90° counterclockwise from î, but here ĵ (pointing right) sits on the clockwise side of î (pointing up), so the orientation has flipped. That is why the cell turns yellow and det becomes −2: the area doubles and the orientation flips (Lesson 7).

For question 4: no, because the origin (0, 0) moves to (0, 1). The stretch-by-2 part on its own is the linear transformation [[2, 0], [0, 2]], but the move upward shifts the origin.

In the next lesson you will see what happens when you do two transformations one after another. How to write "rotate first, then shear" as a single matrix: that is matrix multiplication.