Changing basis

A spot that a map app describes as "300 m north, 400 m east" can be described to someone walking down the street as "500 m straight ahead". There is only one spot, but the numbers change depending on what you measure it against.

Until now we have always read the vector (1, 3) as "1 to the right, 3 up". That means we used the standard basis î, ĵ from Lesson 4 as our ruler. In this lesson you change the ruler. You will learn what numbers come out when you measure the same vector with a tilted ruler, a stretched ruler or a rotated ruler, and how to compute those numbers.

Game developers do this every day. Turning an enemy's position in world coordinates into "so far ahead, so far to the right of my character" is a change of basis. And the eigenvector story in Lessons 11 and 12 is, in the end, also a change of basis: "picking the ruler in which the matrix looks simplest".

10-1. Same vector, different coordinates (10 min)

Written out as a linear combination (Lesson 4), the coordinates (1, 3) look like this.

• (1, 3) = 1·î + 3·ĵ = 1·(1, 0) + 3·(0, 1)

The two numbers in the coordinates said "how many î and how many ĵ to use". If you use two other vectors b₁, b₂ as the ruler instead of î and ĵ, you can write the same vector v as

• v = c₁·b₁ + c₂·b₂

and then (c₁, c₂) are the coordinates in the new basis. One set of rulers for measuring coordinates is called a basis, and when the basis changes, the coordinates of the same vector change too. The vector itself (the arrow) stays put.

In the lab below, teal b₁ = (2, 1) and crimson b₂ = (−1, 1) are the new basis, and black v = (1, 3) is the vector to measure. The tilted blue grid is the "new graph paper" built from b₁ and b₂. On the right, the standard coordinates and the new coordinates appear side by side.

At the start, the new coordinates are (1.33, 1.67). The dashed lines in the picture show what that means. The teal dashed line is b₁ stretched 1.33 times, and the crimson dashed line attached to it is 1.67 times b₂. Put tip to tail, they arrive exactly at the tip of v.

You are calling the same point by two names.

Move v to a crossing of the grid and the new coordinates become integers. That is because one cell of the blue grid is one "tick" of the new ruler. On the standard grid (1, 2) is nothing special, but on the new grid it is the clean spot "one b₁, one b₂".

Put b₂ on one line with b₁ and the grid disappears, with the warning "b₁ and b₂ lie on one line, so they cannot form a basis". If two vectors lie on one line, you cannot build any v off that line (Lesson 4), so they cannot serve as a ruler. To form a basis, the two vectors must be linearly independent.

10-2. Finding the new coordinates = solving B c = v (9 min)

Let's call the matrix with the two new basis vectors stood up as its columns the basis matrix $B$. It is the $B$ shown at the bottom right of the lab.

In Lesson 5, a matrix times a vector was "a linear combination of the columns". So $c_1\mathbf{b}_1 + c_2\mathbf{b}_2$ is simply $B\mathbf{c}$.

Finding the new coordinates $\mathbf{c}$ comes down to solving the system of equations $B\mathbf{c} = \mathbf{v}$. You use exactly what you learned in Lesson 8. Solving it by hand:

Add the two equations and you get $3c_1 = 4$, so $c_1 = \tfrac{4}{3}$, and then $c_2 = 3 - \tfrac{4}{3} = \tfrac{5}{3}$. The lab's 1.33 and 1.67 are these fractions.

Some books write $\mathbf{v}_{\mathcal{B}}$ or $\mathbf{v}'$ instead of $[\mathbf{v}]_{\mathcal{B}}$. When you see a subscript or a prime mark, first check whether it means "coordinates measured in a different basis".

10-3. Going both ways: B and B⁻¹ (7 min)

There are two directions for changing basis. They are easy to mix up, so here they are in a table.

New → standard is a single multiplication, so it is easy. Standard → new is an undo, so it takes the inverse, which in real calculations means solving a system of equations (solve from Lesson 8).

It is easier to remember if you see $B$ as a transformation. In Lesson 5, the columns of a matrix were "where î and ĵ land". $B$ is the transformation that sends î to $\mathbf{b}_1$ and ĵ to $\mathbf{b}_2$, so it bends the straight graph paper into the tilted blue grid of the lab. That is why "the point at $(c_1, c_2)$ on the new grid" turns into standard coordinates after going through $B$.

You can now state the condition for a basis in one line. Saying $\mathbf{b}_1, \mathbf{b}_2$ form a basis is the same as saying:

"There is an inverse" from Lesson 7, "exactly one solution" from Lesson 8 and "rank 2" from Lesson 9 all say the same thing. The new coordinates must be pinned down uniquely for the vectors to work as a ruler.

10-4. Where it is used (5 min)

Local coordinates in games and graphics. Every character has a "forward" and a "right" direction. Use these two directions as a basis, and you can turn an enemy's position in world coordinates into numbers the character understands easily, like "4.4 ahead, 0.8 to the left". Going the other way, turning "5 steps forward" into world coordinates is a multiplication by B. This is the "local coordinates ↔ world coordinates" conversion in 3-D engines.

Rotated coordinate systems. If the basis vectors are perpendicular to each other and have length 1 (an orthonormal basis), B⁻¹ is simply the transpose Bᵀ (the .T from Lesson 5). You do not have to compute an inverse, which is very convenient. This property comes back in Lesson 13 with orthogonality and QR decomposition.

A basis that makes a matrix look simple. The same transformation can look like a different matrix depending on which ruler you measure it with. In a well-chosen basis, a matrix that looked complicated becomes a diagonal matrix that only "stretches along the axes". That special basis is made of the eigenvectors of Lesson 11, and switching to that basis to make calculations easy is the diagonalization of Lesson 12. You get a first taste at the end of 10-5.

10-5. Changing basis with NumPy (12 min)

Follow the code in the lab panel. Make the two basis vectors and v, then build B by standing the basis vectors up as columns with np.column_stack (Lesson 4).

>>> import numpy as np
>>> b1 = np.array([2, 1])
>>> b2 = np.array([-1, 1])
>>> v = np.array([1, 3])
>>> B = np.column_stack([b1, b2])
>>> B
array([[ 2, -1],
       [ 1,  1]])

In the output, the first column is (2, 1) = b₁ and the second column is (−1, 1) = b₂. Do not read it by rows. Now solve B c = v with solve from Lesson 8.

>>> c = np.linalg.solve(B, v)
>>> c
array([1.33333333, 1.66666667])
>>> c * 3
array([4., 5.])

NumPy shows fractions as decimals, so 4/3 comes out as 1.33333333. Multiply by 3 and you can see the numerators 4 and 5. To check the new coordinates, go back the other way, new → standard.

>>> B @ c
array([1., 3.])
>>> c[0] * b1 + c[1] * b2
array([1., 3.])

B @ c and c[0] * b1 + c[1] * b2 giving the same value is a direct check of Bc = c₁b₁ + c₂b₂ from 10-2. But compare with == and you get a surprise.

>>> B @ c == v
array([False,  True])
>>> np.allclose(B @ c, v)
True

The screen shows 1., but the actual value is a tiny bit off from 1. That is because 4/3 cannot be written out fully as a decimal, so it is computed in rounded form (Lesson 3). Always compare floating-point results with np.allclose.

Standard → new coordinates as one matrix

Look at B⁻¹ directly and you can see what the standard → new conversion looks like. Just as in the 2×2 formula from Lesson 7, there is a 1/det = 1/3 in front. det also comes out as 2.9999999999999996 rather than 3: that is the floating-point leftover you saw in Lesson 7.

>>> np.linalg.det(B)
np.float64(2.9999999999999996)
>>> np.linalg.inv(B) * 3
array([[ 1.,  1.],
       [-1.,  2.]])

Many points at once

In graphics you convert not one point but thousands at once. Put in a 2×n matrix with the points lined up as columns, and solve solves each column separately and returns the results in the same shape. Convert the three points you checked in the lab, (1, 3), (1, 2) and (3, 0), in one go.

>>> pts = np.column_stack([[1, 3], [1, 2], [3, 0]])
>>> pts
array([[1, 1, 3],
       [3, 2, 0]])
>>> C = np.linalg.solve(B, pts)
>>> C
array([[ 1.33333333,  1.        ,  1.        ],
       [ 1.66666667,  1.        , -1.        ]])
>>> np.allclose(B @ C, pts)
True

Each column of C holds the new coordinates of one point. The second point (1, 2) becomes (1, 1) and the third point (3, 0) becomes (1, −1), just as you saw in the lab.

A rotated basis

Try a basis that rotates both coordinate axes by 30°. The rotation matrix from Lesson 6 is the basis matrix as it is. Column 1 is î rotated by 30°, and column 2 is ĵ rotated by 30°.

>>> theta = np.radians(30)
>>> R = np.array([[np.cos(theta), -np.sin(theta)],
...               [np.sin(theta),  np.cos(theta)]])
>>> w = np.array([2, 0])
>>> np.linalg.solve(R, w)
array([ 1.73205081, -1.        ])
>>> R.T @ w
array([ 1.73205081, -1.        ])

Measured with a ruler rotated by 30°, the point (2, 0) is (1.732…, −1). The ruler turned 30° counterclockwise, so a point that sat on the original x-axis now lies below the new first axis (the second coordinate is negative), and the first coordinate gets a bit shorter than 2. And multiplying by the transpose R.T instead of using solve gives the same answer. The columns of a rotation matrix are perpendicular to each other and have length 1, so its inverse is just its transpose. Let's check that too.

>>> np.allclose(R.T @ R, np.eye(2))
True

Now compute the character example from 10-4. The character faces the direction (0.6, 0.8), and its right is (0.8, −0.6) (both have length 1 and are perpendicular). Build the basis matrix in the order [right, forward].

>>> right = np.array([0.8, -0.6])
>>> forward = np.array([0.6, 0.8])
>>> F = np.column_stack([right, forward])
>>> enemy = np.array([2, 4])
>>> F.T @ enemy
array([-0.8,  4.4])
>>> F @ np.array([0, 5])
array([3., 4.])

The enemy at world coordinates (2, 4) is at "−0.8 to the right (that is, 0.8 to the left), 4.4 ahead" from the character's point of view. Going the other way, "5 ahead" for the character is world coordinates (3, 4). Because the basis is orthonormal, F.T did the conversion directly.

A taste: matrices can be rewritten in a new basis too

Measure the transformation A = [[2, 1], [1, 2]] again in the basis (1, 1), (−1, 1). Chain "new coordinates → (with B) standard coordinates → apply A → (with B⁻¹) new coordinates" and you get B⁻¹AB. As in Lesson 6, read it from the right. Use solve instead of multiplying by B⁻¹.

>>> A = np.array([[2, 1], [1, 2]])
>>> E = np.column_stack([[1, 1], [-1, 1]])
>>> np.linalg.solve(E, A @ E)
array([[3., 0.],
       [0., 1.]])

In the new basis, A is a diagonal matrix that only does "3 times along the first axis, 1 times along the second". That is because A @ (1, 1) = (3, 3) and A @ (−1, 1) = (−1, 1): the two basis vectors keep their direction and only get 3 times and 1 times as long. Vectors like these are the eigenvectors of Lesson 11.

10-6. Common mistakes (3 min)

Stacking the basis vectors as rows. np.array([b1, b2]) stacks b1 and b2 as rows. A basis matrix needs them as columns, so use np.column_stack. Stacking them as rows gives a wrong answer without any error, which makes it more dangerous.

>>> np.array([b1, b2])
array([[ 2,  1],
       [-1,  1]])
>>> np.linalg.solve(np.array([b1, b2]), v)
array([-0.66666667,  2.33333333])

That is nothing like the right answer (1.33…, 1.67…). The fastest way to catch a mistake is always to check (does B @ c equal v?).

Using vectors on one line as a basis. As when you put b₂ on one line with b₁ in the lab, linearly dependent vectors do not pin down the coordinates, and solve raises an error.

>>> bad = np.column_stack([[2, 1], [4, 2]])
>>> np.linalg.solve(bad, v)
Traceback (most recent call last):
  ...
numpy.linalg.LinAlgError: Singular matrix

Exercises (5 min)

1. In the basis b₁ = (1, 1), b₂ = (−1, 1), predict the new coordinates of v = (2, 0), then check with the lab and NumPy.
2. In the new basis b₁ = (2, 0), b₂ = (0, 0.5), what are the standard coordinates of the point whose new coordinates are (1.5, 4)?
3. Shrink the basis from question 1 to length 1 and you get a basis rotated by 45°. Think with a picture first: what are the new coordinates of (1, 1) in that basis?

Solutions

>>> B1 = np.column_stack([[1, 1], [-1, 1]])
>>> np.linalg.solve(B1, np.array([2, 0]))
array([ 1., -1.])
>>> B2 = np.column_stack([[2, 0], [0, 0.5]])
>>> B2 @ np.array([1.5, 4])
array([3., 2.])
>>> theta = np.radians(45)
>>> R45 = np.array([[np.cos(theta), -np.sin(theta)],
...                 [np.sin(theta),  np.cos(theta)]])
>>> R45.T @ np.array([1, 1])
array([1.41421356, 0.        ])

For question 1, (2, 0) = 1·(1, 1) − 1·(−1, 1), so the new coordinates are (1, −1). Question 2 goes new → standard, so multiply by B. Add 1.5 times b₁, (3, 0), and 4 times b₂, (0, 2), to get (3, 2). For question 3, (1, 1) lies right on the first axis rotated by 45°, so the second coordinate is 0 and the first coordinate is the length of (1, 1), √2 ≈ 1.414.

In the next lesson you look for the special directions you saw at the end of 10-5: vectors whose direction does not change under a transformation. These are eigenvectors, and the factor by which their length stretches is the eigenvalue.