Loading and building chapter...

One of the most important and fundamental concepts in linear algebra is the

In this book, we denote points by capital italic letters, e.g., $A$, $B$, and $Q$. For most of the presentation in the early chapters, we will use two- and three-dimensional points, and some occasional one-dimensional points. We start with a definition of a vector.

Definition 2.1:
Vector

Let $A$ and $B$ be two points. A directed line segment from $A$ to $B$ is denoted by:

This directed line segment constitutes a
*vector*. If you can move the line segment to another line
segment with the same direction and length, they constitute the
same vector.

For instance, the two line segments
$\overrightarrow{AB}$ and $\overrightarrow{CD}$
in Interactive Illustration 2.2 constitute
the same vector as can be seen when pushing the "forward" button.
Let $A$ and $B$ be two points. A directed line segment from $A$ to $B$ is denoted by:

\begin{equation} \overrightarrow{AB}. \end{equation} | (2.1) |

We say that $\overrightarrow{AB}$ is a vector and that

\begin{equation} \overrightarrow{AB} = \overrightarrow{CD}. \end{equation} | (2.2) |

We also use the terms

A vector is completely defined by its

- direction, and
- its length

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

$\vc{u}$

\begin{equation} \text{length of vector:}\spc\spc \ln{\vc{v}} \end{equation} | (2.3) |

Note that the order of the points is important, i.e., if you change the order of $A$ and $B$, another vector, $\overrightarrow{BA}$, is obtained. It has opposite direction, but the same length, i.e., $\ln{\overrightarrow{AB}} = \ln{\overrightarrow{BA}}$. Even $\overrightarrow{AA}$ is a vector, which is called the

Definition 2.2:
Zero Vector

The zero vector is denoted by $\vc{0}$, and can be created using a directed line segment using the same point twice, i.e., $\vc{0}=\overrightarrow{AA}$. Note that $\ln{\vc{0}}=0$, i.e., the length of the zero vector is zero.

Two vectors, $\vc{u}$ and $\vc{v}$, are parallel if they have the same direction or opposite directions, but not necessarily
the same lengths. This is shown to the right in Figure 2.4. Note how you can change the vectors in the figure, some can be changed by grabbing the tip, others by grabbing the tail.
The notation
The zero vector is denoted by $\vc{0}$, and can be created using a directed line segment using the same point twice, i.e., $\vc{0}=\overrightarrow{AA}$. Note that $\ln{\vc{0}}=0$, i.e., the length of the zero vector is zero.

\begin{equation} \vc{u}\, ||\, \vc{v} \end{equation} | (2.4) |

There are two fundamental vector operations in linear algebra, namely,

Definition 2.3:
Vector Addition

The sum, $\vc{u}+\vc{v}$, of two vectors, $\vc{u}$ and $\vc{v}$, is constructed by placing $\vc{u}$, at some arbitrary location, and then placing $\vc{v}$ such that $\vc{v}$'s tail point coincides with $\vc{u}$'s tip point, and $\vc{u}+\vc{v}$ is the vector that starts at $\vc{u}$'s tail point, and ends at $\vc{v}$'s tip point.

Exactly how the vector sum is constructed is shown in Interactive Illustration 2.5 below.
The sum, $\vc{u}+\vc{v}$, of two vectors, $\vc{u}$ and $\vc{v}$, is constructed by placing $\vc{u}$, at some arbitrary location, and then placing $\vc{v}$ such that $\vc{v}$'s tail point coincides with $\vc{u}$'s tip point, and $\vc{u}+\vc{v}$ is the vector that starts at $\vc{u}$'s tail point, and ends at $\vc{v}$'s tip point.

So far, we have only illustrated the vector addition in the plane, i.e., in two dimensions. However, it can also be illustrated in three dimensions. This is done below in Interactive Illustration 2.6. Remember that you can rotate the figure by moving the mouse while right clicking or by using a two-finger swipe.

As we saw in the Breakout Game 2.1, the speed of the ball was increased by 50% after a while. This is an example of vector scaling, where the velocity vector simply was scaled by a factor of $1.5$. However, a scaling factor can be negative as well, and this is all summarized in the definition below, and instead of the term vector scaling, we also use the term scalar vector multiplication.

Definition 2.4:
Scalar Vector Multiplication

When a vector, $\vc{v}$, is multiplied by a scalar, $k$, the vector $k\vc{v}$ is obtained, which is parallel to $\vc{v}$ and its length is $\abs{k}\,\ln{v}$. The direction of $k\vc{v}$ is opposite $\vc{v}$ if $k$ is negative, and otherwise it has the same direction as $\vc{v}$. If $k=0$, then $k\vc{v}=\vc{0}$.

A corollary to this is that if the two vectors $\vc{u}$ and $\vc{v}$ satisfy $\vc{u} = k \vc{v}$ for some scalar $k$, then $\vc{u}$ and $\vc{v}$ are parallel.
When a vector, $\vc{v}$, is multiplied by a scalar, $k$, the vector $k\vc{v}$ is obtained, which is parallel to $\vc{v}$ and its length is $\abs{k}\,\ln{v}$. The direction of $k\vc{v}$ is opposite $\vc{v}$ if $k$ is negative, and otherwise it has the same direction as $\vc{v}$. If $k=0$, then $k\vc{v}=\vc{0}$.

Scalar vector multiplication is shown in Interactive Illustration 2.7 below. The reader is encouraged to play around with the illustration. Now that we can both add vectors, and scale vectors by a real number, it is rather straightforward to subtract two vectors as well. This is shown in the following example.

Example 2.1:
Vector Subtraction

Note that by using vector addition (Definition 2.3) and scalar vector multiplication (Definition 2.4) by $-1$, we can subtract one vector, $\vc{v}$, from another, $\vc{u}$ according to

where we have introduced the shorthand notation, $\vc{u}-\vc{v}$,
for the expression to the left of the equal sign. Vector subtraction is illustrated below.

Note that by using vector addition (Definition 2.3) and scalar vector multiplication (Definition 2.4) by $-1$, we can subtract one vector, $\vc{v}$, from another, $\vc{u}$ according to

\begin{equation} \underbrace{\vc{u} + (\underbrace{-1\vc{v}}_{\text{scaling}})}_{\text{addition}} = \vc{u}-\vc{v}, \end{equation} | (2.5) |

Example 2.2:
Box

In this example, we will see how a box can be created by using three vectors that all make a right angle with each other.

There are a number of different rules for using both vector addition and scalar vector multiplication.
This is the topic of the next section.
In this example, we will see how a box can be created by using three vectors that all make a right angle with each other.

Using vectors in calculations with vector addition and scalar vector multiplication is fairly straightforward. They behave as we might expect them to. However, rules such as $\vc{u}+(\vc{v}+\vc{w})=(\vc{u}+\vc{v})+\vc{w}$, must be proved. The rules for vector arithmetic are summarized in Theorem 2.1.

Theorem 2.1:
Properties of Vector Arithmetic

Assuming that $\vc{u}$, $\vc{v}$, and $\vc{w}$ are vectors of the same size, and that $k$ and $l$ are scalars, then the following rules hold:

While most (or all) of the rules above feel very natural and intuitive, they must be proved nevertheless.
The reader is encouraged to look at the proofs, and especially at the interactive illustrations,
which can increase the feeling and intuition for many of the rules.
Assuming that $\vc{u}$, $\vc{v}$, and $\vc{w}$ are vectors of the same size, and that $k$ and $l$ are scalars, then the following rules hold:

\begin{gather} \begin{array}{llr} (i) & \vc{u}+\vc{v} = \vc{v}+\vc{u} & \spc\text{(commutativity)} \\ (ii) & (\vc{u}+\vc{v})+\vc{w} = \vc{u}+(\vc{v}+\vc{w}) & \spc\text{(associativity)} \\ (iii) & \vc{v}+\vc{0} = \vc{v} & \spc\text{(zero existence)} \\ (iv) & \vc{v}+ (-\vc{v}) = \vc{0} & \spc\text{(negative vector existence)} \\ (v) & k(l\vc{v}) = (kl)\vc{v} & \spc\text{(associativity)}\\ (vi) & 1\vc{v} = \vc{v} & \spc\text{(multiplicative one)} \\ (vii) & 0\vc{v} = \vc{0} & \spc\text{(multiplicative zero)} \\ (viii) & k\vc{0} = \vc{0} & \spc\text{(multiplicative zero vector)} \\ (ix) & k(\vc{u}+\vc{v}) = k\vc{u}+k\vc{v} & \spc\text{(distributivity 1)} \\ (x) & (k+l)\vc{v} = k\vc{v}+l\vc{v} & \spc\text{(distributivity 2)} \\ \end{array} \end{gather} | (2.6) |

$(i)$ This rule (commutativity) has already been proved in the figure in Definition 2.3. Another way to prove this rule is shown below in Interactive Illustration 2.10.

$\textcolor{#aa0000}{\vc{u}}$

$\textcolor{#aa0000}{\vc{u}}$

$\textcolor{#aa0000}{\vc{u}}$

$\textcolor{#aa0000}{\vc{u}}$

$\textcolor{#00aa00}{\vc{v}}$

$\textcolor{#00aa00}{\vc{v}}$

$\textcolor{#00aa00}{\vc{v}}$

$\textcolor{#00aa00}{\vc{v}}$

$\textcolor{#0000aa}{\vc{u}+\vc{v}}$

$\textcolor{#0000aa}{\vc{v}+\vc{u}}$

$\textcolor{#aa0000}{\vc{u}}$

$\textcolor{#00aa00}{\vc{v}}$

$\textcolor{#0000aa}{\vc{w}}$

$\textcolor{#00aaaa}{\vc{v}+\vc{w}}$

$\vc{u}+(\vc{v}+\vc{w})$

$\textcolor{#aaaa00}{\vc{u}+\vc{v}}$

$(\vc{u}+\vc{v})+\vc{w}$

$\vc{u}+\vc{v}+\vc{w}$

$(iv)$ Since $-\vc{v}$ is exactly $\vc{v}$ with opposite direction, the sum will be zero.

$(v)$ The approach to this proof is to start with the left hand side of the equal sign, and find out what the direction and length is. Then the same is done for the right hand side of the equal sign. The details are left as an exercise for the reader.

$(vi)$ Since $1$ is a positive number, we know that $1\vc{v}$ and $\vc{v}$ have the same direction, so it only remains to control that they have the same length. The length of the left hand side of the equal sign is $\abs{1}\,\ln{\vc{v}}=\vc{v}$, and for the right hand side it is $\ln{\vc{v}}$, i.e., they are the same, which proves the rule.

$(vii)$ and $(viii)$ First, note the difference between these. In $(vii)$, we have a scalar zero times $\vc{v}$ equals a zero vector, and in $(viii)$, we have a scalar, $k$ times a zero vector, which is equals the zero vector. $(vii)$ is actually defined in Definition 2.4, so only $(viii)$ needs to be proved. The length of both $k\vc{0}$ and $\vc{0}$ are zero, which proves the rule.

$(ix)$ First, we refer the reader to Interactive Illustration 2.12. Be sure to press

$k=$

$\vc{u}$

$\vc{v}$

$\vc{u} + \vc{v}$

$k\vc{u}$

$k\vc{v}$

$k(\vc{u} + \vc{v})$

$O$

$A_1$

$A_2$

$B_1$

$B_2$

\begin{align} \ln{k\vc{u}} &= \abs{k}\,\ln{\vc{u}}, \\ \ln{k\vc{v}} &= \abs{k}\,\ln{\vc{v}}, \end{align} | (2.7) |

\begin{equation} \ln{k(\vc{u}+\vc{v})} = \abs{k}\,\ln{\vc{u}+\vc{v}}. \end{equation} | (2.8) |

$(x)$ This is somewhat similar to $(ix)$, but simpler, and so is left for the reader.

This concludes the proofs for Theorem 2.1.

$\square$

Example 2.3:
Vector Addition of Three Vectors

To get an understanding of how vector addition works for more than two vectors, Interactive Illustration 2.13 below shows the addition of three vectors. Recall that vector addition is associative, so we may write $\vc{u}+\vc{v}+\vc{w}$ without any parenthesis.

It is often useful to be able to calculate the middle point of two points. This is described in the following theorem.
To get an understanding of how vector addition works for more than two vectors, Interactive Illustration 2.13 below shows the addition of three vectors. Recall that vector addition is associative, so we may write $\vc{u}+\vc{v}+\vc{w}$ without any parenthesis.

Theorem 2.2:
The Middle Point Formula

Assume that $M$ is the middle point of the line segment that goes between $A$ and $B$ as shown in the illustration to the right. Assume $O$ is another point. The vector $\overrightarrow{OM}$, i.e., from $O$ to $M$, can be written as

Assume that $M$ is the middle point of the line segment that goes between $A$ and $B$ as shown in the illustration to the right. Assume $O$ is another point. The vector $\overrightarrow{OM}$, i.e., from $O$ to $M$, can be written as

\begin{equation} \overrightarrow{OM} = \frac{1}{2}(\overrightarrow{OA} + \overrightarrow{OB}). \end{equation} | (2.9) |

The vector $\overrightarrow{OM}$ is the sum of $\overrightarrow{OA}$ and $\overrightarrow{AM}$

\begin{equation} \overrightarrow{OM} = \overrightarrow{OA} + \overrightarrow{AM}. \end{equation} | (2.10) |

By going via $B$ instead we get

\begin{equation} \overrightarrow{OM} = \overrightarrow{OB} + \overrightarrow{BM}. \end{equation} | (2.11) |

\begin{equation} 2\overrightarrow{OM} = \overrightarrow{OA} + \overrightarrow{OB} + \overrightarrow{AM} + \overrightarrow{BM}. \end{equation} | (2.12) |

\begin{equation} \overrightarrow{OM} = \frac{1}{2}(\overrightarrow{OA} + \overrightarrow{OB}). \end{equation} | (2.13) |

\begin{equation} M = \frac{1}{2}(A + B) \end{equation} | (2.14) |

$\square$

Example 2.4:
Sierpinski Triangles using the Middle Point Theorem

We will now show how the middle point formula can be used to generate a geometrical figure called the Sierpinski triangle. Assume we have a triangle consisting of three points, $A$, $B$, and $C$. Using Theorem 2.2, the midpoints of each edge can now be computed. These midpoints can be connected to form four new triangles, where the center triangle is empty. If this process is repeated for each new non-empty triangle, then we arrive at the Sierpinski triangle. This is shown in Interactive Illustration 2.15 below.

We will now show how the middle point formula can be used to generate a geometrical figure called the Sierpinski triangle. Assume we have a triangle consisting of three points, $A$, $B$, and $C$. Using Theorem 2.2, the midpoints of each edge can now be computed. These midpoints can be connected to form four new triangles, where the center triangle is empty. If this process is repeated for each new non-empty triangle, then we arrive at the Sierpinski triangle. This is shown in Interactive Illustration 2.15 below.

Example 2.5:
Center of Mass Formula

In the triangle $ABC$, the point $A'$ is on the mipoint between $B$ and $C$. The line segment from $A$ to $A'$ is called the median of $A$. Let $M$ be the point that divides the median of $A$ in a proportion 2 to 1, shown in the illustration to the right.

The center of mass formula states that

This formula can be proved as follows.
We can go from $O$ to $M$ either directly or via $A$, hence

It is also possible to arrive at $M$ via $A’$. This gives

One of the assumptions is that $\pvec{A'M}$ is half the length of $\pvec{AM}$ and of opposite direction,
and therefore, it holds that $\pvec{A'M} = -\frac{1}{2}\pvec{AM}$. Inserting this in the equation above gives

We can eliminate $\pvec{AM}$ by adding Equation (2.16) to two times Equation (2.18)

Since $A'$ is the mid point of $B$ and $C$, we known from the middle point formula that $\pvec{OA'} = \frac{1}{2}(\pvec{OB} + \pvec{OC})$.
Inserting this in the equation above gives

which simplifies as

This completes the proof.

Note that since the formula is symmetric, it works just as well on the median to $B$. The same point, $M$, will divide the median going from $B$ to $B'$ in proportion $2:1$. This can be seen by pressing forward in the interactive illustration.

This point is also called the center of mass. If the triangle was cut out of cardboard, this is the point where it would balance of the top of a pencil, which exaplains the name "center for mass". Also, if equal point masses were placed in the points $A$, $B$ and $C$, $M$ is the point where they would balance out.

In the triangle $ABC$, the point $A'$ is on the mipoint between $B$ and $C$. The line segment from $A$ to $A'$ is called the median of $A$. Let $M$ be the point that divides the median of $A$ in a proportion 2 to 1, shown in the illustration to the right.

The center of mass formula states that

\begin{equation} \pvec{OM} = \frac{1}{3}(\pvec{OA} + \pvec{OB} + \pvec{OC}). \end{equation} | (2.15) |

\begin{equation} \pvec{OM} = \pvec{OA} + \pvec{AM}. \end{equation} | (2.16) |

\begin{equation} \pvec{OM} = \pvec{OA'} + \pvec{A'M}. \end{equation} | (2.17) |

\begin{equation} \pvec{OM} = \pvec{OA'} -\frac{1}{2}\pvec{AM}. \end{equation} | (2.18) |

\begin{equation} \pvec{3OM} = \pvec{OA} + 2\pvec{OA'}. \end{equation} | (2.19) |

\begin{equation} \pvec{3OM} = \pvec{OA} + 2\frac{1}{2}(\pvec{OB} + \pvec{OC}), \end{equation} | (2.20) |

\begin{equation} \pvec{OM} = \frac{1}{3}(\pvec{OA} + \pvec{OB} + \pvec{OC}). \end{equation} | (2.21) |

Note that since the formula is symmetric, it works just as well on the median to $B$. The same point, $M$, will divide the median going from $B$ to $B'$ in proportion $2:1$. This can be seen by pressing forward in the interactive illustration.

This point is also called the center of mass. If the triangle was cut out of cardboard, this is the point where it would balance of the top of a pencil, which exaplains the name "center for mass". Also, if equal point masses were placed in the points $A$, $B$ and $C$, $M$ is the point where they would balance out.

Most of you are probably familiar with the concept of a coordinate system, such as in the map in the first step of Interactive Illustration 2.17 below. In this first step, the axes are perpendicular and of equal length, but this is a special case, as can be seen by pressing

Theorem 2.3:
Coordinate in One Dimension

Let $\vc{e}$ be a non-zero vector on a straight line. For each vector, $\vc{v}$, on the line, there is only one number, $x$, such that

(The vector, $\vc{v}$, in the figure to the right can be moved around.)

Let $\vc{e}$ be a non-zero vector on a straight line. For each vector, $\vc{v}$, on the line, there is only one number, $x$, such that

\begin{equation} \vc{v} = x \vc{e}. \end{equation} | (2.22) |

If $\vc{e}$ and $\vc{v}$ have the same direction, then choose $x=\ln{\vc{v}}/\ln{\vc{e}}$, and if $\vc{e}$ and $\vc{v}$ have opposite directions, then set $x=-\ln{\vc{v}}/\ln{\vc{e}}$. Finally, if $\vc{v}=\vc{0}$, then $x=0$. It follows from the definition of scalar vector multiplication, $x\vc{e}$, that $x$ is the only number that fulfils $ \vc{v} = x\vc{e}$.

$\square$

Note that we say that $\vc{e}$ is a

So far, this is not very exciting, but the next step makes this much more useful.

Theorem 2.4:
Coordinates in Two Dimensions

Let $\vc{e}_1$ and $\vc{e}_2$ be two non-parallel vectors (which both lie in a plane). For every vector, $\vc{v}$, in this plane, there is a single coordinate pair, $(x,y)$, such that

(The vectors $\vc{v}$, $\vc{e}_1$, and $\vc{e}_2$ can be moved around in the figure.)

Let $\vc{e}_1$ and $\vc{e}_2$ be two non-parallel vectors (which both lie in a plane). For every vector, $\vc{v}$, in this plane, there is a single coordinate pair, $(x,y)$, such that

\begin{equation} \vc{v} = x\vc{e}_1 + y\vc{e}_2. \end{equation} | (2.23) |

For this proof, we will use Interactive Illustration 2.19. As can be seen, $P_1$ was obtained by drawing a line, parallel to $\vc{e}_2$, from the tip point of $\vc{v}$ until it collides with the line going through $\vc{e}_1$. Similarly, $P_2$ is obtained by drawing a line, parallel to $\vc{e}_1$, from the tip point of $\vc{v}$ until it collides with the line going through $\vc{e}_2$. It is clear that

\begin{equation} \vc{v} = \overrightarrow{O P_1} + \overrightarrow{O P_2}. \end{equation} | (2.24) |

\begin{equation} \vc{v} = \vc{u} + \vc{w} = x \vc{e}_1 + y \vc{e}_2. \end{equation} | (2.25) |

\begin{equation} \vc{v} = x' \vc{e}_1 + y' \vc{e}_2. \end{equation} | (2.26) |

\begin{gather} x \vc{e}_1 + y \vc{e}_2= x' \vc{e}_1 + y' \vc{e}_2 \\ \Longleftrightarrow \\ (x-x') \vc{e}_1 = (y'-y) \vc{e}_2. \end{gather} | (2.27) |

\begin{gather} \vc{e}_1 = \frac{(y'-y)}{(x-x')} \vc{e}_2, \end{gather} | (2.28) |

$\square$

Note that we say that $\vc{e}_1$ and $\vc{e}_2$ are

Next, we will extend this to three dimensions as well.

Theorem 2.5:
Coordinates in Three Dimensions

Let $\vc{e}_1$, $\vc{e}_2$, and $\vc{e}_3$ be three non-zero basis vectors, and that there is no plane that is parallel with all three vectors. For every vector, $\vc{v}$, in the three-dimensional space, there is a single coordinate triplet, $(x,y,z)$, such that

Let $\vc{e}_1$, $\vc{e}_2$, and $\vc{e}_3$ be three non-zero basis vectors, and that there is no plane that is parallel with all three vectors. For every vector, $\vc{v}$, in the three-dimensional space, there is a single coordinate triplet, $(x,y,z)$, such that

\begin{equation} \vc{v} = x\vc{e}_1 + y\vc{e}_2 + z\vc{e}_3. \end{equation} | (2.29) |

Start by placing all the vectors $\vc{v}$, $\vc{e}_1$, $\vc{e}_2$ and $\vc{e}_3$ so that they start in the origin according to Interactive Illustration 2.20. Let $\pi_{12}$ be the plane through $O$ that contains $\vc{e}_1$ and $\vc{e}_2$, and let $P$ be the point at the tip of $\vc{v}$, i.e., $\vc{v} = \overrightarrow{OP}$.

Draw a line from $P$ parallel with $\vc{e}_3$ that intersects the plane $\pi_{12}$ in the point $P_{12}$. It is now clear that we can write $\vc{v}$ as the sum

\begin{equation} \vc{v} = \overrightarrow{OP} = \overrightarrow{OP_{12}} + \overrightarrow{P_{12}P}. \end{equation} | (2.30) |

\begin{equation} \vc{v} = x \vc{e}_1 + y \vc{e}_2 + z \vc{e}_3. \end{equation} | (2.31) |

\begin{equation} \vc{v} = x' \vc{e}_1 + y' \vc{e}_2 + z' \vc{e}_3. \end{equation} | (2.32) |

\begin{equation} x \vc{e}_1 + y \vc{e}_2 + z \vc{e}_3 = x' \vc{e}_1 + y' \vc{e}_2 + z' \vc{e}_3. \end{equation} | (2.33) |

\begin{equation} (x-x') \vc{e}_1 + (y-y') \vc{e}_2 + (z-z') \vc{e}_3 = 0. \end{equation} | (2.34) |

\begin{equation} \vc{e}_1 = - \frac{(y-y')}{(x-x')} \vc{e}_2 - \frac{(z-z')}{(x-x')}\vc{e}_3, \end{equation} | (2.35) |

\begin{equation} \vc{e}_1 = \alpha \vc{e}_2 + \beta \vc{e}_3, \end{equation} | (2.36) |

$\square$

Similarly as before, we say that $\vc{e}_1$, $\vc{e}_2$, and $\vc{e}_3$ are

Now, we can finally see where the vector representation using coordinates comes from. If we assume that a certain basis, $\{\vc{e}_1, \vc{e}_2, \vc{e}_3\}$, is used, then we can write a three-dimensional vector, $\vc{v}$, as

\begin{equation} \vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3= \begin{pmatrix} v_x \\ v_y \\ v_z \end{pmatrix}, \end{equation} | (2.37) |

Definition 2.5:
Column Vector Notation

Given a basis, a column vector, $\vc{v}$, in $n$ dimensions (we have used $n\in [1,2,3]$) is a column of $n$ scalar values. These scalar components, sometimes called vector elements, of the vector can either be numbered, i.e., $v_1$, $v_2$, and $v_3$, or we can use $x$, $y$, and $z$ as subscripts when that is more convenient. The notation is:

where $\vc{u} = u_x \vc{e}_1$, $\vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2$,
and $\vc{w} = w_x \vc{e}_1 + w_y \vc{e}_2 + w_z \vc{e}_3$.

We also use a more compact way of writing vectors, which is convenient when writing
vectors in text, for example: $\vc{w} = \bigl(w_1,w_2,w_3\bigr)$, which means the same as above (notice
the commas between the vector elements).
Given a basis, a column vector, $\vc{v}$, in $n$ dimensions (we have used $n\in [1,2,3]$) is a column of $n$ scalar values. These scalar components, sometimes called vector elements, of the vector can either be numbered, i.e., $v_1$, $v_2$, and $v_3$, or we can use $x$, $y$, and $z$ as subscripts when that is more convenient. The notation is:

\begin{gather} \underbrace{ \vc{u} = \begin{pmatrix} u_x \end{pmatrix} = \begin{pmatrix} u_1 \end{pmatrix}}_{\text{1D vector}}, \spc\spc \underbrace{ \vc{v} = \begin{pmatrix} v_x \\ v_y \end{pmatrix} = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix}}_{\text{2D vector}}, \spc\spc \\ \underbrace{ \vc{w} = \begin{pmatrix} w_x \\ w_y \\ w_z \end{pmatrix} = \begin{pmatrix} w_1 \\ w_2 \\ w_3 \end{pmatrix}}_{\text{3D vector}}, \end{gather} | (2.38) |

Column vectors, per the definition above, is the type of vectors that we use mostly throughout this book. Hence, when we say "vector", we mean a "column vector". However, there is also another type of vectors, namely, row vectors. As can be deduced from the name, it is simply a row of scalar values, instead of a column of scalar values. An example of a row vector is:

\begin{equation} \bigl(1\spc 2\spc 5 \bigr). \end{equation} | (2.39) |

\begin{equation} \vc{v} = \begin{pmatrix} 1\\ 2\\ 5 \end{pmatrix}, \spc\spc\spc \vc{v}^\T = \bigl(1\spc 2\spc 5 \bigr). \end{equation} | (2.40) |

Definition 2.6:
Transpose of a Vector

The transpose of a vector, $\vc{v}$, is denoted by $\vc{v}^\T$, and turns a column vector into a row vector, and a row vector into a column vector. The order of the vector components is preserved.

Note that with this definition, we can transpose a vector twice, and get back the
same vector, i.e., $\bigl(\vc{v}^T\bigr)^T = \vc{v}$.
Next, we also summarize the row vector definition below:
The transpose of a vector, $\vc{v}$, is denoted by $\vc{v}^\T$, and turns a column vector into a row vector, and a row vector into a column vector. The order of the vector components is preserved.

Definition 2.7:
Row Vector Notation

A row vector is expressed as a transposed column vector, as shown below:

Notice that a row vector never has any commas between the vector elements. This is reserved for
the compact notation for column vector (see Definition 2.5).
A row vector is expressed as a transposed column vector, as shown below:

\begin{equation} \underbrace{ \vc{v}^\T = \bigl( v_x \spc v_y \bigr) }_{\text{2D row vector}}, \spc \spc \underbrace{ \vc{w}^\T = \bigl( w_x \spc w_y \spc w_z \bigr) }_{\text{3D row vector}}. \end{equation} | (2.41) |

Now, let us assume that we have two vectors, $\vc{u}$ and $\vc{v}$, in the same basis, i.e.,

\begin{equation} \vc{u} = u_x \vc{e}_1 + u_y \vc{e}_2 + u_z \vc{e}_3= \begin{pmatrix} u_x \\ u_y \\ u_z \end{pmatrix} \spc\spc \text{and} \spc\spc \vc{v} = v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3= \begin{pmatrix} v_x \\ v_y \\ v_z \end{pmatrix}. \end{equation} | (2.42) |

\begin{align} \vc{u}+\vc{v} &= u_x \vc{e}_1 + u_y \vc{e}_2 + u_z \vc{e}_3 + v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3 \\ &=(u_x+v_x)\vc{e}_1 + (u_y+v_y)\vc{e}_2 + (u_z+v_z)\vc{e}_3 \\ &= \begin{pmatrix} u_x+v_x \\ u_y+v_y \\ u_z+v_z \end{pmatrix}. \end{align} | (2.43) |

\begin{align} k\vc{v} &= k (v_x \vc{e}_1 + v_y \vc{e}_2 + v_z \vc{e}_3) \\ &= (k v_x) \vc{e}_1 + (k v_y) \vc{e}_2 + (k v_z) \vc{e}_3\\ &= \begin{pmatrix} k v_x \\ k v_y \\ k v_z \end{pmatrix}, \end{align} | (2.44) |

Example 2.6:
Vector Addition and Scalar Multiplication using Coordinates

Assume we have the following vectors in the same basis:

and that we now want to evaluate $\vc{u} + \vc{v} - 2\vc{w}$.
As we have seen above, vector addition is simply a matter of adding the vector elements:

We can also scale a vector by a scalar value, e.g., $k=2$:

which means that $\vc{u} + \vc{v} - 2\vc{w} = \vc{0}$.

In many calculations, one uses a simple and intuitive basis called the Assume we have the following vectors in the same basis:

\begin{equation} \vc{u} = \left( \begin{array}{r} 3 \\ -4 \\ 7 \end{array} \right), \spc \spc \vc{v} = \left( \begin{array}{r} 1 \\ 2 \\ 5 \end{array} \right), \spc \spc \text{and} \spc \spc \vc{w} = \left( \begin{array}{r} 2 \\ -1 \\ 6 \end{array} \right), \end{equation} | (2.45) |

\begin{equation} \vc{u}+\vc{v} = \left( \begin{array}{r} 3 \\ -4 \\ 7 \end{array} \right) + \left( \begin{array}{r} 1 \\ 2 \\ 5 \end{array} \right) = \left( \begin{array}{r} 3+1 \\ -4+2 \\ 7+5 \end{array} \right) = \left( \begin{array}{r} 4 \\ -2 \\ 12 \end{array} \right). \end{equation} | (2.46) |

\begin{equation} 2\vc{w} = 2 \left( \begin{array}{r} 2 \\ -1 \\ 6 \end{array} \right) = \left( \begin{array}{c} 2\cdot 2 \\ 2\cdot (-1) \\ 2\cdot 6 \end{array} \right) = \left( \begin{array}{r} 4 \\ -2 \\ 12 \end{array} \right), \end{equation} | (2.47) |

Definition 2.8:
Standard Basis

The standard basis in this book is as follows for two and three dimensions, that is,

In general, for an $n$-dimensional standard basis, the basis vectors, $\vc{e}_i$ have vector elements which
are all zeroes, except the $i$:th element, which is a one.

In Chapter 3, we will discuss different types of bases, and we will see that the standard basis is, in fact,
an orthonormal basis (Section 3.3).
The standard basis in this book is as follows for two and three dimensions, that is,

\begin{gather} \underbrace{ \vc{e}_1 = \begin{pmatrix} 1 \\ 0 \end{pmatrix},\ \vc{e}_2 = \begin{pmatrix} 0 \\ 1 \end{pmatrix} }_{\mathrm{two-dimensional\ standard\ basis}} \ \ \mathrm{and} \\ \ \\ \underbrace{ \vc{e}_1 = \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix},\ \vc{e}_2 = \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix},\ \vc{e}_3 = \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix}. }_{\mathrm{three-dimensional\ standard\ basis}} \end{gather} | (2.48) |

Example 2.7:
Addition in the Standard Basis

In this example, we will illustrate how vector addition is done in the standard basis in order to increase the reader's intuition about addition. See Interactive Illustration 2.21. Recall that the standard basis vector in two dimensions are $\vc{e}_1=(1,0)$ and $\vc{e}_2=(0,1)$.

Next, two intuitive examples will be given on the topics of coordinate systems, basis vectors, uniqueness, and coordinates.
In this example, we will illustrate how vector addition is done in the standard basis in order to increase the reader's intuition about addition. See Interactive Illustration 2.21. Recall that the standard basis vector in two dimensions are $\vc{e}_1=(1,0)$ and $\vc{e}_2=(0,1)$.

Example 2.8:
Same Point Expressed In Different Bases

Note that the same point will have different coordinates when different basis vectors are used, as shown in Interactive Illustration 2.22. Note in the illustration that when the basis vectors change, the coordinates change too, but the point stays at the same place all the time.

Note that the same point will have different coordinates when different basis vectors are used, as shown in Interactive Illustration 2.22. Note in the illustration that when the basis vectors change, the coordinates change too, but the point stays at the same place all the time.

Example 2.9:
Intuition about Uniqueness

Going back to stage two in Interactive Illustration 2.22, it is obvious that adding the two basis vectors together brings us the vector $\overrightarrow{OP}$ exactly, so $\overrightarrow{OP} = 1.0 \vc{e}_1 + 1.0 \vc{e}_2$ must hold. Thus, $(1, 1)$ is a valid coordinate pair for the point $P$. However, one may ask whether there are any other coordinates that will also describe the point $P$, now that the basis vectors no longer need to make a right angle. The answer to this is no, as we have seen in the proof to Theorem 2.4. A bit more intuition about why this is so, can be obtained from Interactive Illustration 2.23.

Going back to stage two in Interactive Illustration 2.22, it is obvious that adding the two basis vectors together brings us the vector $\overrightarrow{OP}$ exactly, so $\overrightarrow{OP} = 1.0 \vc{e}_1 + 1.0 \vc{e}_2$ must hold. Thus, $(1, 1)$ is a valid coordinate pair for the point $P$. However, one may ask whether there are any other coordinates that will also describe the point $P$, now that the basis vectors no longer need to make a right angle. The answer to this is no, as we have seen in the proof to Theorem 2.4. A bit more intuition about why this is so, can be obtained from Interactive Illustration 2.23.

In this chapter, we have introduced the notion of vectors. The definitions of a vector in Section 2.5 and the basic operations, such as vector addition (Definition 2.3) and multiplication with a scalar (Definition 2.4), have been defined geometrically. We then showed that these two operations fulfill a number of properties in Theorem 2.1. This definition works for $\R^1$, $\R^2$, and $\R^3$. For higher dimensions, it is difficult for us to use the geometric definition. The notion of a two-dimensional and three-dimensional vector is in itself very useful, but geometric vectors are also a stepping stone for understanding general linear spaces or vector spaces. This more general theory is extremely useful for modeling and understanding problems when we have more than three unknown parameters. The reader may want to skip the following section, and revisit it later depending on his/her needs.

In this section, we will first give a definition of a $\R^n$.

Definition 2.9:
Real Coordinate Space

The vector space $\R^n$ is defined as $n$-tuples $\vc{u} = (u_1, u_2, \ldots, u_n)$, where each $u_i$ is a real number. It is a vector space over the real numbers $\R$, where vector addition $\vc{u}+\vc{v}$ is defined as $\vc{u}+\vc{v} = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n)$ and scalar-vector multiplications is defined as $k\vc{v} = (k v_1, k v_2, \ldots, k v_n)$, where $k\in \R$.

Note that using these definitions for vector addition and scalar-vector multiplication, the properties of Theorem 2.1 all hold.
The vector space $\R^n$ is defined as $n$-tuples $\vc{u} = (u_1, u_2, \ldots, u_n)$, where each $u_i$ is a real number. It is a vector space over the real numbers $\R$, where vector addition $\vc{u}+\vc{v}$ is defined as $\vc{u}+\vc{v} = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n)$ and scalar-vector multiplications is defined as $k\vc{v} = (k v_1, k v_2, \ldots, k v_n)$, where $k\in \R$.

Example 2.10:

Let $\vc{u}=(1,2,3,4,5)$ and $\vc{v}=(5,4,3,2,1)$ be two vectors in $\R^5$. What is $\vc{u}+\vc{v}$, $3\vc{u}$ and $3\vc{u}+3\vc{v}$?

In this last step, the result ($\vc{u}+\vc{v}$) from Equation (2.49) was used.

Let $\vc{u}=(1,2,3,4,5)$ and $\vc{v}=(5,4,3,2,1)$ be two vectors in $\R^5$. What is $\vc{u}+\vc{v}$, $3\vc{u}$ and $3\vc{u}+3\vc{v}$?

\begin{equation} \vc{u}+\vc{v} = (1,2,3,4,5) + (5,4,3,2,1) = (1+5,2+4,3+3,4+2,5+1) = (6,6,6,6,6) \end{equation} | (2.49) |

\begin{equation} 3\vc{u}= 3(1,2,3,4,5) = (3 \cdot 1,3 \cdot 2,3 \cdot 3,3 \cdot 4,3 \cdot 5) = (3,6,9,12,15) \end{equation} | (2.50) |

\begin{equation} 3\vc{u}+3\vc{v} = 3 (\vc{u}+\vc{v}) = 3 (6,6,6,6,6) = (18,18,18,18,18) \end{equation} | (2.51) |

Definition 2.10:
Basis in $\R^n$

A basis in $\R^n$ is a set of vectors $\{\vc{e}_1, \ldots, \vc{e}_m\}$ in so that for every vector $\vc{u}\in\R^n$, there is a unique set of*coordinates* $(u_1, \ldots, u_m)$ so that

A basis in $\R^n$ is a set of vectors $\{\vc{e}_1, \ldots, \vc{e}_m\}$ in so that for every vector $\vc{u}\in\R^n$, there is a unique set of

\begin{equation} \vc{u} = \sum_{i_1}^m u_i \vc{e}_i. \end{equation} | (2.52) |

Example 2.11:
Canonical Basis in $\R^n$

The canonical basis in $\R^n$ is the following set of basis vectors

The canonical basis in $\R^n$ is the following set of basis vectors

\begin{equation} \begin{cases} \begin{array}{ll} \vc{e}_1 &= (1, 0, \ldots, 0), \\ \vc{e}_2 &= (0, 1, \ldots, 0), \\ \vdots & \\ \vc{e}_n &= (0, 0, \ldots, 1). \end{array} \end{cases} \end{equation} | (2.53) |

We will now present an abstract definition of a vector space. Then we will show that any finite-dimensional vector space over $\R$ is in fact 'the same as' $\R^n$ that we defined earlier in Definition 2.9.

Definition 2.11:
Vector space

A vector space consists of a set $V$ of objects (called vectors) and a field $F$, together with a definition of vector addition and multiplication of a scalar with a vector, in such a way that the properties of Theorem 2.1 holds.

A vector space consists of a set $V$ of objects. As we shall see in one example, the vector space is the set of images of size $m \times n$ pixels. In another example, the vector space is a set of polynomials up to degree $5$.
The elements of the field $F$ are called scalars. A field is a set of objects where addition, subtraction, multiplication and division is well defined and follows the usual properties. Most often the field used is the set of real numbers $\R$ or the set of complex numbers $\mathbb{C}$, but one could use more exotic fields, such as integers modulu a prime number, e.g., $\mathbb{Z}_3$.
A vector space consists of a set $V$ of objects (called vectors) and a field $F$, together with a definition of vector addition and multiplication of a scalar with a vector, in such a way that the properties of Theorem 2.1 holds.

Example 2.12:
Polynomials up to degree 2

Polynomials in $x$ up to degree 2 with real coefficients is a vector space over $\R$. Here if $u = u_0 + u_1 x + u_2 x^2$ and $v = v_0 + v_1 x + v_2 x^2$, where each coefficient $u_i$ and $v_i$ is a real number. Here vector addition $u+v$ is defined as $ u+v = (u_0+v_0) + (u_1+v_1) x + (u_2+v_2) x^2 $ and scalar-vector multiplications is defined as $ku = k u_0 + k u_1 x + k u_2 x^2$.

Polynomials in $x$ up to degree 2 with real coefficients is a vector space over $\R$. Here if $u = u_0 + u_1 x + u_2 x^2$ and $v = v_0 + v_1 x + v_2 x^2$, where each coefficient $u_i$ and $v_i$ is a real number. Here vector addition $u+v$ is defined as $ u+v = (u_0+v_0) + (u_1+v_1) x + (u_2+v_2) x^2 $ and scalar-vector multiplications is defined as $ku = k u_0 + k u_1 x + k u_2 x^2$.

Example 2.13:
Gray-scale images

Gray-scale images, where each pixel intensity is a real number is a vector space over $\R$. Here if the pixel of the image $u$ at position $(i,j)$ has intensity $u_{i,j}$ and similarily if the pixel of the image $v$ at position $(i,j)$ has intensity $v_{i,j}$, then vector addition is defined as an image $u+v$ where the intensity of the pixel at position $(i,j)$ is $u_{i,j}+v_{i,j}$. The scalar-vector multiplications is defined as the image $ku$ where the pixel at position $(i,j)$ has intensity $k u_{i,j}$,

Gray-scale images, where each pixel intensity is a real number is a vector space over $\R$. Here if the pixel of the image $u$ at position $(i,j)$ has intensity $u_{i,j}$ and similarily if the pixel of the image $v$ at position $(i,j)$ has intensity $v_{i,j}$, then vector addition is defined as an image $u+v$ where the intensity of the pixel at position $(i,j)$ is $u_{i,j}+v_{i,j}$. The scalar-vector multiplications is defined as the image $ku$ where the pixel at position $(i,j)$ has intensity $k u_{i,j}$,

Example 2.14:
$\mathbb{Z}_3$ Coordinate Space

The vector space $\mathbb{Z}_3^n$ is defined as n-tuples $ \vc{u} = (u_1, u_2, \ldots, u_n)$, where each $u_i$ is one of the integers $0$, $1$, or $2$. It is a vector space over the integers $0$, $1$, and $2$. Vector addition $\vc{u}+\vc{v}$ is defined as $ \vc{u}+\vc{v} = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n)$ and scalar-vector multiplications is defined as $k\vc{v} = (k u_1, k u_2, \ldots, k u_n)$. Here the multiplications and additions of two scalars are done moduli 3.

The vector space $\mathbb{Z}_3^n$ is defined as n-tuples $ \vc{u} = (u_1, u_2, \ldots, u_n)$, where each $u_i$ is one of the integers $0$, $1$, or $2$. It is a vector space over the integers $0$, $1$, and $2$. Vector addition $\vc{u}+\vc{v}$ is defined as $ \vc{u}+\vc{v} = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n)$ and scalar-vector multiplications is defined as $k\vc{v} = (k u_1, k u_2, \ldots, k u_n)$. Here the multiplications and additions of two scalars are done moduli 3.

Definition 2.12:
Basis in Vector Space

A basis in a finite dimensional vector space $V$ over $F$ is a set of vectors $\{\vc{e}_1, \ldots, \vc{e}_m\}$ in so that for every vector $\vc{u} \in V$, there is a unique set of*coordinates* $(u_1, \ldots, u_m)$ with $u_i \in F$, so that

The number, $m$, of basis vectors is said to be the dimension of the vector space. We will later show that this is a well-defined number for a given vector space.
A basis in a finite dimensional vector space $V$ over $F$ is a set of vectors $\{\vc{e}_1, \ldots, \vc{e}_m\}$ in so that for every vector $\vc{u} \in V$, there is a unique set of

\begin{equation} \vc{u} = \sum_{i_1}^m u_i \vc{e}_i. \end{equation} | (2.54) |

Theorem 2.6:
Vector in Vector Space

Let $V$ be a vector space over $\R^m$ and let $\{\vc{e}_1, \ldots, \vc{e}_m\}$ be a basis. Then each vector $\vc{u}$ can be identified with its coordinates $(u_1, \ldots, u_m)$.

In this way, one can loosely say that each $m$-dimensional vector space over $\R$ is
'the same thing' as $\R^m$.
Let $V$ be a vector space over $\R^m$ and let $\{\vc{e}_1, \ldots, \vc{e}_m\}$ be a basis. Then each vector $\vc{u}$ can be identified with its coordinates $(u_1, \ldots, u_m)$.

The vector concept has been treated in this chapter, and the vector addition and scalar vector multiplication operations have been introduced. In addition, we have seen that these operations behave pretty much as expected, i.e., similar to how we calculate with real numbers. To make the vectors a bit more practical, the basis concept was introduced, and we saw how a, e.g., three-dimensional vector can be represented by three scalar numbers with respect to a certain basis. Finally, we also introduced the concept of a higher-dimensional vector space $\R^n$ very briefly. In Chapter 3, the dot product operation will be introduced. It is useful when measuring length and angles.