Vector = arrow = list of numbers: addition and scaling

In Lesson 1 you drew a vector as a point on the plane. This lesson draws the same vector as an arrow. A point shows "where something is"; an arrow shows "which way and how far it moves". The bundle of numbers stays the same; only the way you look at it changes.

Seen as arrows, adding and stretching vectors becomes very natural. If you have ever made a game, you have already done these calculations. You add a character's velocity to its position to get the next position, you double the velocity when the run button is held, and you flip the direction when it backs away. That is the whole of this lesson: vector addition and scalar multiplication.

At the end you will do the same calculations with NumPy. Along the way you will make sure to see that + on a Python list does something completely different. It is the first place people trip when they turn linear algebra into code.

2-1. From points to arrows (7 min)

To draw the vector (3, 2) as an arrow, start at the origin (0, 0) and draw an arrow that ends at the point (3, 2). The arrow means "move 3 steps right and 2 steps up".

There is one important idea here. An arrow is the same vector no matter where it starts. An arrow from (1, 1) to (4, 3) also moves 3 right and 2 up, so it is the same vector (3, 2). Moving an arrow without changing its direction or length is called translation, and all translated copies of an arrow count as the same vector.

A vector only records "how far it moved", not "where it started". So you may draw the same vector wherever you need it. Addition, in the very next section, relies on exactly this.

Whether you see a vector as a point or an arrow depends on the situation. A character's position is natural as a point, and its velocity or displacement as an arrow. Either way, it is a vector of two numbers.

2-2. Addition = tip to tail (10 min)

The picture of adding two vectors is simple. Attach the tail of the second arrow to the tip of the first, then draw a new arrow from the origin to the final tip. That new arrow is the sum.

In the lab below, the teal arrow is u and the crimson arrow is w. With 'Show u + w (tip to tail)' turned on, the dashed crimson arrow is w moved to the tip of u, and the black arrow is u + w.

With the defaults u = (2, 1) and w = (1, 2), follow the arrows: along u, 2 right and 1 up, then along w, 1 right and 2 up. Altogether that is 3 right and 3 up, so u + w = (3, 3). Written in numbers, what you did in the picture is adding the components in the same position.

You will also see a faint dashed teal arrow in the lab. This is the other route: u attached to the tip of w. That way you land on the same point u + w. Because the two routes form a parallelogram, the order of addition does not change the result (u + w = w + u).

In game terms: if a character is now at (2, 1) and its velocity for one frame is (1, 2), its position on the next frame is (2, 1) + (1, 2) = (3, 3). Add a displacement (an arrow) to a position (a point) and you get a new position (a point).

Books write addition with column vectors like this.

Since you add component by component, you can only add vectors of the same dimension. A 2-dimensional vector and a 3-dimensional vector have no matching "same positions", so they cannot be added. The rule is the same in 784 dimensions or 768.

2-3. Scalar multiplication = stretch, shrink, flip (7 min)

Multiplying a vector by a single number is called scalar multiplication. The single number is called a scalar, which just means a plain number rather than a bundle like a vector. To multiply, you multiply every component by that number.

In the picture, the arrow keeps its direction and only its length changes. Here is what each number does.

In the lab, turn on 'Show c·u (stretch or flip)' and move the slider c. The thick, faint teal arrow is c·u. However you change c, c·u only moves along the faint teal line through u. Scalar multiplication can "flip" a direction, but it can never "turn" it. This fact becomes very important in Lesson 4.

The $(0, 0)$ you get at c = 0 is called the zero vector and is written as a bold $\mathbf{0}$. As an arrow it has length 0 and no direction: a single point at the origin. Adding the zero vector to any vector leaves it unchanged: $\mathbf{u} + \mathbf{0} = \mathbf{u}$. It plays the same role as the number 0.

In a game, you multiply the velocity by 2 when the run button is held, by −1 to walk backwards, and by 0 to stop.

2-4. Subtraction = the arrow from w to u (6 min)

Subtraction is "flip, then add". $\mathbf{u} - \mathbf{w}$ is the same as $\mathbf{u} + (-1)\mathbf{w}$. In numbers, again, you just subtract component by component.

The picture has a more useful reading. u − w is the arrow from the tip of w to the tip of u. Start at w, go a further $\mathbf{u} - \mathbf{w}$, and you arrive at u: $\mathbf{w} + (\mathbf{u} - \mathbf{w}) = \mathbf{u}$. It is the same story as "end − start = how far it moved" from 2-1.

Turn on 'Show u − w' in the lab and a yellow arrow runs from the tip of crimson w to the tip of teal u. It does not start at the origin, but as 2-1 said, a translated arrow is the same vector, so that is fine. Also try putting u and w on the same point and watch the yellow arrow disappear: $\mathbf{u} - \mathbf{u} = \mathbf{0}$.

For an enemy in a game to chase the player, you compute "player position − enemy position". That is the arrow from the enemy to the player. Subtract in the wrong order and the arrow points the other way, so the enemy runs off.

Here is the new notation in one place.

2-5. Vector arithmetic with NumPy (12 min)

The "Check it with NumPy" panel under the lab shows an interactive session like this every time you let go. Follow it line by line with the default values.

>>> import numpy as np
>>> u = np.array([2, 1])
>>> w = np.array([1, 2])
>>> u + w
array([3, 3])
>>> 2 * u
array([4, 2])
>>> u - w
array([ 1, -1])

+ and - on NumPy arrays do exactly what you did in the picture: they compute position by position. This kind of calculation is called an element-wise operation. 2 * u is scalar multiplication, multiplying every component by 2. The space before the 1 in array([ 1, -1]) is just NumPy lining up the digits; it has nothing to do with the value.

Even if you do not turn on 'Show c·u (stretch or flip)' in the lab, the NumPy panel always shows a c·u line (c starts at 2). Move the slider to 0.5 and that line changes like this.

>>> 0.5 * u
array([1. , 0.5])
>>> -1 * u
array([-2, -1])
>>> -u
array([-2, -1])
>>> 0 * u
array([0, 0])

Multiplying an integer array by 0.5 gives decimals, so the dtype becomes float64 (the same rule you saw in Lesson 1). 1. is 1.0. -u is short for -1 * u.

Python lists behave completely differently

The last line of the NumPy panel is a trap, put there on purpose. Add the same numbers as Python lists and you get this.

>>> [2, 1] + [1, 2]
[2, 1, 1, 2]

+ on lists does not add components; it joins the two lists. Two numbers became four, which is something entirely different from vector addition. Multiplication is the same story.

>>> [2, 1] * 3
[2, 1, 2, 1, 2, 1]
>>> np.array([2, 1]) * 3
array([6, 3])

Multiply a list by an integer and it repeats that many times; multiply an array and every component is multiplied. Lists cannot even be subtracted from each other.

>>> [2, 1] - [1, 2]
Traceback (most recent call last):
  ...
TypeError: unsupported operand type(s) for -: 'list' and 'list'

So when you do vector arithmetic, get into the habit of converting to np.array first. If you mix an array and a list, NumPy does convert the list to an array for you. But if both sides are lists you get joining instead, so it is safer to make arrays from the start.

>>> u + [1, 2]
array([3, 3])

An array and a single number: broadcasting

Add a single number to an array and it is added to every component. NumPy "spreads" the single number out to the shape of the array before computing; this is called broadcasting. Scalar multiplication 2 * u works on the same principle.

>>> u + 10
array([12, 11])
>>> u / 2
array([1. , 0.5])

u + 10 is not an operation you will find in a linear algebra book, but it comes up all the time when working with data (for example, adding 10 points to every score). Dividing with / always gives real numbers.

Broadcasting spreads out a single number; it does not force vectors of different shapes to fit together. Try to add a 3-dimensional vector to a 2-dimensional one and you get an error.

>>> np.array([1, 2, 3]) + u
Traceback (most recent call last):
  ...
ValueError: operands could not be broadcast together with shapes (3,) (2,)

The shapes (3,) (2,) in the message tells you the cause directly. When you hit an error like this, check the shape of both sides first.

Moving a game character

Let's write a game loop in NumPy that adds the velocity to the position every frame.

>>> pos = np.array([0.0, 0.0])
>>> vel = np.array([1.5, 0.5])
>>> for frame in range(3):
...     pos = pos + vel
...     print(pos)
[1.5 0.5]
[3. 1.]
[4.5 1.5]
>>> pos + 2 * vel
array([7.5, 2.5])

An array printed with print shows just the numbers separated by spaces, without the array(...) wrapper. The last line works out "where would it be after one more frame at double speed", mixing addition and scalar multiplication.

2-6. Common mistakes (3 min)

Adding a decimal to an integer array in place. u += 0.5 means "add 0.5 to u and store the result back in u". But u is an integer (int64) array, so it cannot hold a decimal result, and you get an error.

>>> u = np.array([2, 1])
>>> u += 0.5
Traceback (most recent call last):
  ...
numpy._core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('float64') to dtype('int64') with casting rule 'same_kind'

The message is long, but it means "a float64 result cannot go into an int64 slot". Either make a new array with u = u + 0.5, or create it as real numbers from the start, like np.array([2.0, 1.0]). That is why the game example above started the position at 0.0.

>>> u = u + 0.5
>>> u
array([2.5, 1.5])

Adding lists together. As you saw in 2-5, [2, 1] + [1, 2] returns [2, 1, 1, 2] without any error. No error is exactly what makes it dangerous. If a result suddenly doubles in length, suspect this mistake.

Exercises (5 min)

1. In the lab, set u = (3, 1) and w = (−1, 2). Work out u + w and u − w by hand first, then check.
2. The player is at (1, 4) and an enemy is at (4, 0). Which direction vector should the enemy use to move toward the player? Compute it with NumPy.
3. With a = [1, 2, 3], predict the results of a * 2 and np.array(a) * 2, then check.
4. What number c turns u = (2, 1) into (−3, −1.5)?

Solutions

>>> u = np.array([3, 1])
>>> w = np.array([-1, 2])
>>> u + w
array([2, 3])
>>> u - w
array([ 4, -1])
>>> player = np.array([1, 4])
>>> enemy = np.array([4, 0])
>>> player - enemy
array([-3,  4])
>>> a = [1, 2, 3]
>>> a * 2
[1, 2, 3, 1, 2, 3]
>>> np.array(a) * 2
array([2, 4, 6])
>>> -1.5 * np.array([2, 1])
array([-3. , -1.5])

For question 2 it is "end (player) − start (enemy)", so player - enemy: an arrow going 3 left and 4 up. For question 4, both components are −1.5 times the originals, so c = −1.5. The direction flipped, so c had to be negative.

In the next lesson you will put numbers on an arrow's length and on the angle between two arrows. At the center of it is the dot product, the very calculation recommendation systems use to find "people with similar taste".