Chapter 2: Vectors

---

In this book, we denote points by capital italic letters, e.g., $A$, $B$, and $Q$. For most of the presentation in the early chapters, we will use two- and three-dimensional points, and some occasional one-dimensional points. We start with a definition of a vector.

For instance, the two line segments $\overrightarrow{AB}$ and $\overrightarrow{CD}$ in Interactive Illustration 2.2 constitute the same vector as can be seen when pushing the "forward" button.

We say that $\overrightarrow{AB}$ is a vector and that

\begin{equation} \overrightarrow{AB} = \overrightarrow{CD}. \end{equation}

(2.2)

A shorter notation for vectors is to use a single bold face characters, such as $\vc{v}$. As is shown in the illustration, $\vc{v} = \overrightarrow{AB} = \overrightarrow{CD}$. Some books make a difference between directed line segments and vectors, and reserve the short hand variant $\vc{v}$ for true vectors and the longer $\overrightarrow{AB}$ for directed line segments. While this may be mathematically more stringent, this difference is ignored for the purposes of this book, and we use vectors and directed line segments as one and the same thing.

We also use the terms tail point and tip point of a vector when this is convenient, where the tip point is where the arrowhead is, and the tail point is the other end.

A vector is completely defined by its

1. direction, and
2. its length

Note that a starting position of a vector is missing from the list above. As long as the direction and length is not changed, it is possible to move it around and have it start in any location. This is illustrated in Interactive Illustration 2.3. The length of a vector is denoted by $\ln{\overrightarrow{AB}}$, or in shorthand by $\ln{\vc{v}}$.

\begin{equation} \text{length of vector:}\spc\spc \ln{\vc{v}} \end{equation}

(2.3)

The length of a vector is a scalar, which just means that it is a regular number, such as $7.5$. The term scalar is used to emphasize that it is just a number and not a vector or a point. Exactly how the length of a vector can be calculated will be deferred to Chapter 3.

Note that the order of the points is important, i.e., if you change the order of $A$ and $B$, another vector, $\overrightarrow{BA}$, is obtained. It has opposite direction, but the same length, i.e., $\ln{\overrightarrow{AB}} = \ln{\overrightarrow{BA}}$. Even $\overrightarrow{AA}$ is a vector, which is called the zero vector, as shown in the definition below.

Two vectors, $\vc{u}$ and $\vc{v}$, are parallel if they have the same direction or opposite directions, but not necessarily the same lengths. This is shown to the right in Figure 2.4. Note how you can change the vectors in the figure, some can be changed by grabbing the tip, others by grabbing the tail. The notation

\begin{equation} \vc{u}\, ||\, \vc{v} \end{equation}

(2.4)

means that $\vc{u}$ is parallel to $\vc{v}$. The zero vector $\vc{0}$ is said to be parallel to all other vectors. Next, we will present how two vectors can be added to form a new vector, and then follows scalar vector multiplication in Section 2.3.

---

There are two fundamental vector operations in linear algebra, namely, vector addition and scalar vector multiplication, where the latter is sometimes called vector scaling. Most of the mathematics in this book build upon these two operations, and even the most complex operations often lead back to addition and scaling. Vector scaling is described in Section 2.3, while vector addition is described here. Luckily, both vector addition and vector scaling behave as we would expect them to.

Exactly how the vector sum is constructed is shown in Interactive Illustration 2.5 below.

So far, we have only illustrated the vector addition in the plane, i.e., in two dimensions. However, it can also be illustrated in three dimensions. This is done below in Interactive Illustration 2.6. Remember that you can rotate the figure by moving the mouse while right clicking or by using a two-finger swipe.

---

As we saw in the Breakout Game 2.1, the speed of the ball was increased by 50% after a while. This is an example of vector scaling, where the velocity vector simply was scaled by a factor of $1.5$. However, a scaling factor can be negative as well, and this is all summarized in the definition below, and instead of the term vector scaling, we also use the term scalar vector multiplication.

A corollary to this is that if the two vectors $\vc{u}$ and $\vc{v}$ satisfy $\vc{u} = k \vc{v}$ for some scalar $k$, then $\vc{u}$ and $\vc{v}$ are parallel.

Scalar vector multiplication is shown in Interactive Illustration 2.7 below. The reader is encouraged to play around with the illustration. Now that we can both add vectors, and scale vectors by a real number, it is rather straightforward to subtract two vectors as well. This is shown in the following example.

There are a number of different rules for using both vector addition and scalar vector multiplication. This is the topic of the next section.

---

Using vectors in calculations with vector addition and scalar vector multiplication is fairly straightforward. They behave as we might expect them to. However, rules such as $\vc{u}+(\vc{v}+\vc{w})=(\vc{u}+\vc{v})+\vc{w}$, must be proved. The rules for vector arithmetic are summarized in Theorem 2.1.

While most (or all) of the rules above feel very natural and intuitive, they must be proved nevertheless. The reader is encouraged to look at the proofs, and especially at the interactive illustrations, which can increase the feeling and intuition for many of the rules.

---

It is often useful to be able to calculate the middle point of two points. This is described in the following theorem.

---

---

Most of you are probably familiar with the concept of a coordinate system, such as in the map in the first step of Interactive Illustration 2.17 below. In this first step, the axes are perpendicular and of equal length, but this is a special case, as can be seen by pressing Forward. This section will describe the general coordinate systems, and the interaction between vectors, bases, and coordinates. Next, we define how coordinates can be described in one, two, and three dimensions. This is done with the following set of theorems.

---

Note that we say that $\vc{e}$ is a basis vector, and that $x$ is the coordinate for $\vc{v}$ in the basis of $\{\vc{e}\}$.

So far, this is not very exciting, but the next step makes this much more useful.

---

Note that we say that $\vc{e}_1$ and $\vc{e}_2$ are basis vectors, and that $x$ and $y$ are the coordinates for $\vc{v}$ in the basis of $\{\vc{e}_1,\vc{e}_2\}$.

Next, we will extend this to three dimensions as well.

---

Similarly as before, we say that $\vc{e}_1$, $\vc{e}_2$, and $\vc{e}_3$ are basis vectors, and that $x$, $y$, and $z$ are the coordinates for $\vc{v}$ in the basis of $\{\vc{e}_1,\vc{e}_2,\vc{e}_3\}$.

Now, we can finally see where the vector representation using coordinates comes from. If we assume that a certain basis, $\{\vc{e}_1, \vc{e}_2, \vc{e}_3\}$, is used, then we can write a three-dimensional vector, $\vc{v}$, as

\begin{equation} \vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3= \begin{pmatrix} v_x \\ v_y \\ v_z \end{pmatrix}, \end{equation}

(2.37)

where we have used $v_x$ instead of $x$, $v_y$ instead of $y$, and $v_z$ instead of $z$. This is to make it simpler to mix several different vectors, and still be able to access the individual components. Note that the right-hand expression shows the vector as a column of three numbers, the $x$-coordinate on top, the $y$-coordinate in the middle, and the $z$-coordinate at the bottom. This is such an important notation, so we have summarized it into the following definition:

We also use a more compact way of writing vectors, which is convenient when writing vectors in text, for example: $\vc{w} = \bigl(w_1,w_2,w_3\bigr)$, which means the same as above (notice the commas between the vector elements).

Column vectors, per the definition above, is the type of vectors that we use mostly throughout this book. Hence, when we say "vector", we mean a "column vector". However, there is also another type of vectors, namely, row vectors. As can be deduced from the name, it is simply a row of scalar values, instead of a column of scalar values. An example of a row vector is:

\begin{equation} \bigl(1\spc 2\spc 5 \bigr). \end{equation}

(2.39)

Any vector, be it row or column, can be transposed, which means that a row vector turns into a column vector, and a column vector turns into a row vector. The notation for a transposed vector is: $\vc{v}^T$. An example is shown below:

\begin{equation} \vc{v} = \begin{pmatrix} 1\\ 2\\ 5 \end{pmatrix}, \spc\spc\spc \vc{v}^\T = \bigl(1\spc 2\spc 5 \bigr). \end{equation}

(2.40)

We summarize the transposing of a vector in the following definition:

Note that with this definition, we can transpose a vector twice, and get back the same vector, i.e., $\bigl(\vc{v}^T\bigr)^T = \vc{v}$. Next, we also summarize the row vector definition below:

Notice that a row vector never has any commas between the vector elements. This is reserved for the compact notation for column vector (see Definition 2.5).

Now, let us assume that we have two vectors, $\vc{u}$ and $\vc{v}$, in the same basis, i.e.,

\begin{equation} \vc{u} = u_x \vc{e}_1 + u_y \vc{e}_2 + u_z \vc{e}_3= \begin{pmatrix} u_x \\ u_y \\ u_z \end{pmatrix} \spc\spc \text{and} \spc\spc \vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3= \begin{pmatrix} v_x \\ v_y \\ v_z \end{pmatrix}. \end{equation}

(2.42)

The addition, $\vc{u}+\vc{v}$, becomes:

\begin{align} \vc{u}+\vc{v} &= u_x \vc{e}_1 + u_y \vc{e}_2 + u_z \vc{e}_3 + v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3 \\ &=(u_x+v_x)\vc{e}_1 + (u_y+v_y)\vc{e}_2 + (u_z+v_z)\vc{e}_3 \\ &= \begin{pmatrix} u_x+v_x \\ u_y+v_y \\ u_z+v_z \end{pmatrix}. \end{align}

(2.43)

As can be seen, the vector addition boils down to simple component-wise scalar addition. For scalar vector multiplication, $k\vc{v}$, we have:

\begin{align} k\vc{v} &= k (v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3) \\ &= (k v_x) \vc{e}_1 + (k v_y) \vc{e}_2 + (k v_z) \vc{e}_3\\ &= \begin{pmatrix} k v_x \\ k v_y \\ k v_z \end{pmatrix}, \end{align}

(2.44)

and here we see that each component of the vector is multiplied by $k$.

In many calculations, one uses a simple and intuitive basis called the standard basis, which is defined as follows.

In Chapter 3, we will discuss different types of bases, and we will see that the standard basis is, in fact, an orthonormal basis (Section 3.3).

Next, two intuitive examples will be given on the topics of coordinate systems, basis vectors, uniqueness, and coordinates.

---

In this chapter, we have introduced the notion of vectors. The definitions of a vector in Section 2.5 and the basic operations, such as vector addition (Definition 2.3) and multiplication with a scalar (Definition 2.4), have been defined geometrically. We then showed that these two operations fulfill a number of properties in Theorem 2.1. This definition works for $\R^1$, $\R^2$, and $\R^3$. For higher dimensions, it is difficult for us to use the geometric definition. The notion of a two-dimensional and three-dimensional vector is in itself very useful, but geometric vectors are also a stepping stone for understanding general linear spaces or vector spaces. This more general theory is extremely useful for modeling and understanding problems when we have more than three unknown parameters. The reader may want to skip the following section, and revisit it later depending on his/her needs.

In this section, we will first give a definition of a $\R^n$.

Note that using these definitions for vector addition and scalar-vector multiplication, the properties of Theorem 2.1 all hold.

2.6.1 The General Definition

---

We will now present an abstract definition of a vector space. Then we will show that any finite-dimensional vector space over $\R$ is in fact 'the same as' $\R^n$ that we defined earlier in Definition 2.9.

A vector space consists of a set $V$ of objects. As we shall see in one example, the vector space is the set of images of size $m \times n$ pixels. In another example, the vector space is a set of polynomials up to degree $5$. The elements of the field $F$ are called scalars. A field is a set of objects where addition, subtraction, multiplication and division is well defined and follows the usual properties. Most often the field used is the set of real numbers $\R$ or the set of complex numbers $\mathbb{C}$, but one could use more exotic fields, such as integers modulo a prime number, e.g., $\mathbb{Z}_3$.

The number, $m$, of basis vectors is said to be the dimension of the vector space. We will later show that this is a well-defined number for a given vector space.

In this way, one can loosely say that each $m$-dimensional vector space over $\R$ is 'the same thing' as $\R^m$.

---

The vector concept has been treated in this chapter, and the vector addition and scalar vector multiplication operations have been introduced. In addition, we have seen that these operations behave pretty much as expected, i.e., similar to how we calculate with real numbers. To make the vectors a bit more practical, the basis concept was introduced, and we saw how a, e.g., three-dimensional vector can be represented by three scalar numbers with respect to a certain basis. Finally, we also introduced the concept of a higher-dimensional vector space $\R^n$ very briefly. In Chapter 3, the dot product operation will be introduced. It is useful when measuring length and angles.