Chapter 6: The Matrix

---

Matrices are a very powerful tool to manipulate data with. As can be seen in the example shown in Interactive Illustration 6.1, matrices can be used to transform images in different ways. After the theory has been presented, the text will connect back to this example.

The Matrix, $\mx{M}$:

In Section 6.10, we will connect back to this introductory example. Now, let us start with a definition of the matrix, and then see how they can be used.

---

As we saw in Chapter 5, a typical linear system of equations can look like

\begin{equation} \begin{cases} \begin{array}{rrrl} 2 & \!\!\!\!\!\! x_1 + 4 &\!\!\!\!\!\!\!x_2 - 2 &\!\!\!\!\!\!x_3 = \hid{-}16, \\ - & \!\!\!\!\!\! x_1 - 7 &\!\!\!\!\!\!x_2 + 2 &\!\!\!\!\!\!x_3 = -27, \\ & 3 &\!\!\!\!\!\!x_2 - 6 &\!\!\!\!\!\!x_3 = -21. \\ \end{array} \end{cases} \end{equation}

(6.1)

To the left of the equal signs, there are a number of constants being multiplied by either $x_1$, $x_2$, or $x_3$. All these constants can be taken out and inserted into a structure called a matrix. For Equation (6.1) this results in

\begin{equation} \left(\begin{array}{rrr} 2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array}\right) . \end{equation}

(6.2)

As can be seen, there are large parenthesis surrounding the array of numbers. This size of the matrix above is $3\times 3$, i.e., three rows and three columns. However, in general, a matrix can have any size. This leads to the following definition.

Note that the notation for a matrix is upper-case bold letters, e.g., $\mx{A}$, and as usual, all scalars are lower-case italic letters, $a_{ij}$., where $i$ is the row and $j$ is the column of the matrix element. Sometimes, it is convenient to extract out either a particular column of scalars, or a particular row. Note that for a $r \times c$ matrix, $\mx{A}$, there are $r$ different row vectors and $c$ different column vectors. The column vectors for a matrix, $\mx{A}$, are

\begin{align} \mx{A}=& \left( \begin{array}{rrr} 2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) = \left( \begin{array}{ccc} \vert & \vert & \vert \\ \vc{a}_{,1} & \vc{a}_{,2} & \vc{a}_{,3} \\ \vert & \vert & \vert \end{array} \right), \\ &\\ &\\ &\text{where } \vc{a}_{,1} = \left( \begin{array}{rrr} 2 \\ -1\\ 0 \end{array} \right), \ \ \vc{a}_{,2} = \left( \begin{array}{rrr} 4 \\ -7\\ 3 \end{array} \right), \ \ \text{and } \vc{a}_{,3} = \left( \begin{array}{rrr} -2 \\ 2\\ -6 \end{array} \right). \end{align}

(6.4)

Note that the vertical lines ($\vert$) are there to illustrate that $\vc{a}_{,i}$ are column vectors that extend both upwards and downwards in the matrix. The corresponding row vectors are

\begin{align} \mx{A}=& \left( \begin{array}{rrr} 2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) = \left( \begin{array}{c} -\,\,\, \vc{a}_{1,}^\T - \\ -\,\,\, \vc{a}_{2,}^\T - \\ -\,\,\, \vc{a}_{3,}^\T - \end{array} \right), \\ &\text{where } \vc{a}_{1,} = \left( \begin{array}{rrr} 2 \\ 4\\ -2 \end{array} \right), \ \ \vc{a}_{2,} = \left( \begin{array}{rrr} -1 \\ -7\\ 2 \end{array} \right), \ \ \text{and } \vc{a}_{3,} = \left( \begin{array}{rrr} 0 \\ 3\\ -6 \end{array} \right). \end{align}

(6.5)

Recall that vectors are by default column vectors in this book, and therefore, we have transposed the $\vc{a}_{i,}$ vectors above, in order to turn them into row vectors. The horizontal lines ($-$) are there to illustrate that $\vc{a}_{i,}^\T$ are row vectors that extend both to the left and to the right in the matrix. As can be seen, this is consistent with our notation for vectors, which use lower-case bold letters. This notation is summarized in the following definition.

In the definition above, we have omitted vertical and horizontal lines (as used in (6.5) and (6.4)) in Equation (6.6). Note that the row vector is denoted $\vc{a}_{i,}^\T$, i.e., it is a column vector ($\vc{a}_{i,}$), which has been transposed into a row vector.

There are also two special constant matrices called the identity matrix, which is denoted by $\mx{I}$, and the zero matrix, which is denoted by $\mx{O}$. The former plays a role which is similar to the number 1 in plain algebra, and the latter plays the role of the 0.

Hence, a $2 \times 2$ identity matrix is $\mx{I} =\bigl( \begin{smallmatrix} 1 & 0\\ 0 & 1\end{smallmatrix} \bigr)$, and a $3\times 3$ identity matrix is $\mx{I} =\Bigl( \begin{smallmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1\end{smallmatrix} \Bigr)$. As can be seen, we have used $\mx{I}$ for both these matrices. Next follows the definition of the zero matrix.

In most cases, the size of $\mx{I}$ and $\mx{O}$ can be determined from the context in which they are used, and otherwise, we will mention what the size is.

Note also that if the number of columns of a matrix, $\mx{A}$, is 1, i.e., $c=1$, then we have a column vector, and we may denote it as a vector instead, i.e., $\vc{a}$. Furthermore, if the number of rows is 1, i.e., $r=1$, in a matrix, $\mx{B}$, then we have a row vector, and may we denote it by a transposed column vector instead, i.e., $\vc{b}^\T$. Below, we show a $3\times 1$ matrix (column vector) and a $1\times 3$ matrix (row vector), which is written out as a transposed column vector.

\begin{equation} \underbrace{ \mx{A} }_{3\times 1} = \left( \begin{array}{c} 3 \\ 2 \\ 6 \end{array} \right) = \underbrace{\vc{a}}_{\begin{array}{c} \text{column} \\ \text{vector} \end{array}} ,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\, \underbrace{ \mx{B} }_{1\times 3} = \bigl(5\,\,\, 1 \,\,\, 4\bigr) = \underbrace{ \vc{b}^\T }_{\begin{array}{c} \text{transposed} \\ \text{column} \\ \text{vector} \end{array}} \end{equation}

(6.8)

It should be pointed out that sometimes it is more natural to use row vectors just as a row instead of a transposed column. This is really up to the reader.

As we have already seen in Chapter 2, vectors can be transposed. This means that a column vector becomes a row vector and vice versa. A matrix can also be transposed as defined below.

A square matrix may also be symmetric if it can be reflected along the main diagonal (from upper left down to lower right) while remaining the same. This is summarized in the following definition.

Next follows some examples of matrix transposing.

With these definitions, it is time to attempt to visualize a matrix with geometry. This is not done in any books that we have seen, however, it makes for a deeper understanding in some cases. See Interactive Illustration 6.2. Next, a handful of matrix operations are presented.

---

There are three fundamental operations on matrices. These are

• matrix multiplication by a scalar,
• matrix addition, and
• matrix-matrix multiplication.

These are are presented in the following subsections.

6.3.1 Matrix Multiplication by a Scalar

---

Matrix multiplication by a scalar is quite similar to vector multiplication by a scalar (Section 2.3), as can be seen in the following definition.

A short example on the scalar multiplication by a matrix follows.

6.3.2 Matrix Addition

---

Matrix addition is also similar to vector addition (Section 2.2).

With the help from Definition 6.7 (matrix multiplication by a scalar), and with the definition of matrix addition, we can easily subtract two matrices. The difference $\mx{D}$ between $\mx{A}$ and $\mx{B}$ becomes

\begin{gather} \mx{D} = \mx{A} + (-1)\mx{B} = \mx{A} - \mx{B} \\ \Longleftrightarrow \\ [d_{ij}] = [a_{ij}]+ (-1)[b_{ij}] = [a_{ij}]+ [-b_{ij}] = [a_{ij} - b_{ij}], \end{gather}

(6.16)

where we on the first row have used scalar multiplication (by $-1$) and matrix addition.

A short example on matrix addition follows.

6.3.3 Matrix-Matrix Multiplication

---

While matrix multiplication by a scalar and matrix addition are rather straightforward, the matrix-matrix multiplication may not be so at first. However, as we will see, it is an extremely powerful tool. The definition is below.

The size of the product may be remembered more easily with this rule,

\begin{equation} (r \times \bcancel{s})\, (\bcancel{s} \times t) \longrightarrow (r \times t), \end{equation}

(6.20)

i.e., only the row size ($r$) of the first operand and the column size ($t$) of the second operand remains, and the column size ($s$) of the first operand and the row size ($s$) of the second operand must be equal.

Note that the sum in the product, $\sum_{k=1}^s a_{ik} b_{kj}$, reminds us of a dot product in an orthonormal basis (Definition 3.4). Hence, by using Definition 6.2, we can express the matrix-matrix multiplication in terms of the row vectors of $\mx{A}$ and the column vectors of $\mx{B}$ as

\begin{align} \mx{P} = \mx{A}\mx{B} &= \left( \begin{array}{c} -\,\,\, \vc{a}_{1,}^\T - \\ \textcolor{#cc0000}{-}\,\,\, \textcolor{#cc0000}{\vc{a}_{2,}^\T} \textcolor{#cc0000}{-} \\ \vdots \\ -\,\,\, \vc{a}_{r,}^\T - \end{array} \right) \left( \begin{array}{ccccc} \vert & \vert & \textcolor{#cc0000}{\vert} & & \vert \\ \vc{b}_{,1} & \vc{b}_{,2} & \textcolor{#cc0000}{\vc{b}_{,3}} & \dots & \vc{b}_{,t} \\ \vert & \vert & \textcolor{#cc0000}{\vert} & & \vert \end{array} \right) \\ &\\ &= \left( \begin{array}{ccccc} \vc{a}_{1,}\cdot \vc{b}_{,1} & \vc{a}_{1,}\cdot \vc{b}_{,2} & \vc{a}_{1,}\cdot \vc{b}_{,3} & \dots & \vc{a}_{1,}\cdot \vc{b}_{,t} \\ \vc{a}_{2,}\cdot \vc{b}_{,1} & \vc{a}_{2,}\cdot \vc{b}_{,2} & \textcolor{#cc0000}{\vc{a}_{2,}\cdot \vc{b}_{,3}} & \dots & \vc{a}_{2,}\cdot \vc{b}_{,t} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ \vc{a}_{r,}\cdot \vc{b}_{,1} & \vc{a}_{r,}\cdot \vc{b}_{,2} & \vc{a}_{r,}\cdot \vc{b}_{,3} & \dots & \vc{a}_{r,}\cdot \vc{b}_{,t} \\ \end{array} \right). \end{align}

(6.21)

As can be seen on the red vectors above, you can produce a matrix element on the $i$:th row and $j$:th column in $\mx{P}$ by taking the dot product between the $i$:th row vector in $\mx{A}$ and the $j$:th column vector in $\mx{B}$, i.e.,

\begin{equation} [p_{ij}] = \Biggl[\sum_{k=1}^s a_{ik} b_{kj}\Biggr] = \bigl[ \vc{a}_{i,} \cdot \vc{b}_{,j} \bigr], \end{equation}

(6.22)

in shorthand notation. Note that if we relax the notion of the matrix, so that $\vc{a}_{i,}^\T$ is considered as a $1\times s$ matrix, and $\vc{b}_{,j}$ is an $s\times 1$ matrix, then we can rewrite Equation (6.22) as a matrix-matrix multiplication, that is,

\begin{equation} [p_{ij}] = \Biggl[ \sum_{k=1}^s a_{ik} b_{kj} \Biggr] = \biggl[ \vc{a}_{i,}^\T \vc{b}_{,j} \biggr] = \Biggl[ \Bigl(a_1\spc a_2 \dots a_s \Bigr) \left( \begin{array}{c} b_1\\ b_2\\ \vdots\\ b_s \end{array} \right) \Biggr]. \end{equation}

(6.23)

Per Definition 6.9, we know that the matrix-matrix multiplication, $\mx{M}\mx{N}$, is only defined if $\mx{M}$ is $r \times s$ and $\mx{N}$ is $s\times t$. This means the number of columns ($s$) in $\mx{M}$ must be equal to the number of rows ($s$) in $\mx{N}$. Hence, $r$ and $t$ can be arbitrary values as long as they are $\geq 1$. If $t=1$ then $\mx{N}$ has only one column, and we do not really have a matrix, but rather a column vector, as exemplified in (6.8). This means that matrix-vector multiplication is a subset of the matrix-matrix multiplication. An example is given below.

Note that the example in (6.1) is a linear system of equations that can be expressed using a matrix and two vectors, i.e.,

\begin{gather} \begin{cases} \begin{array}{rrrl} 2 & \!\!\!\!\!\! x_1 + 4 &\!\!\!\!\!\!\!x_2 - 2 &\!\!\!\!\!\!x_3 = \hid{-}16 \\ - & \!\!\!\!\!\! x_1 - 7 &\!\!\!\!\!\!x_2 + 2 &\!\!\!\!\!\!x_3 = -27 \\ & 3 &\!\!\!\!\!\!x_2 - 6 &\!\!\!\!\!\!x_3 = -21 \\ \end{array} \end{cases} \\ \Longleftrightarrow \\ \underbrace{ \left( \begin{array}{rrr} -2 & 4 & -2 \\ -1 & -7 & 2 \\ 0 & 3 & -6 \end{array} \right) }_{\mx{A}} \, \underbrace{ \left( \begin{array}{c} x_1\\ x_2\\ x_3 \end{array} \right) }_{\vc{x}} = \underbrace{ \left( \begin{array}{r} 16\\ -27\\ -21 \end{array} \right) }_{\vc{b}} \\ \Longleftrightarrow \\ \mx{A} \vc{x} = \vc{b}. \end{gather}

(6.26)

Now that we know this much, we can de-mystify where the definition of the matrix-matrix multiplication (Definition 6.9) really comes from. Let us assume that we have two systems of linear equations,

\begin{gather} \begin{cases} \begin{array}{r} z_1 = a_{11} y_1 + a_{12} y_2 \\ z_2 = a_{21} y_1 + a_{22} y_2 \end{array} \end{cases} \,\,\,\,\,\,\,\,\text{and}\,\,\,\,\,\,\,\, \begin{cases} \begin{array}{r} y_1 = b_{11} x_1 + b_{12} x_2 \\ y_2 = b_{21} x_1 + b_{22} x_2 \end{array} \end{cases}. \end{gather}

(6.27)

Now, what would the result be if we wanted to express $z_1$ and $z_2$ in terms of $x_1$ and $x_2$ instead of in $y_1$ and $y_2$. This is done below as

\begin{gather} \begin{cases} \begin{array}{r} z_1 = a_{11} (b_{11} x_1 + b_{12} x_2) + a_{12} (b_{21} x_1 + b_{22} x_2) \\ z_2 = a_{21} (b_{11} x_1 + b_{12} x_2) + a_{22} (b_{21} x_1 + b_{22} x_2) \end{array} \end{cases} \\ \Longleftrightarrow \\ \begin{cases} \begin{array}{r} z_1 = (a_{11} b_{11} + a_{12} b_{21}) x_1 + (a_{11} b_{12} + a_{12} b_{22}) x_2) \\ z_2 = (a_{21} b_{11} + a_{22} b_{21}) x_1 + (a_{21} b_{12} + a_{22} b_{22}) x_2) \end{array} \end{cases}. \end{gather}

(6.28)

As can be seen, the terms before $x_1$ and $x_2$ are exactly the terms that we would get if we multiply two matrices, $\mx{A}$ and $\mx{B}$. Both Equation (6.27) and (6.28) can be expressed on matrix/vector form as

\begin{gather} \vc{z}=\mx{A}\vc{y} \spc\spc\text{and}\spc\spc \vc{y}=\mx{B}\vc{x} \\ \Longleftrightarrow \\ \vc{z}=\mx{A}\mx{B}\vc{x}, \end{gather}

(6.29)

and this motivates why the matrix-matrix multiplication is defined the way it is.

As we saw in Equation (6.21), the first operand in the matrix-matrix multiplication, $\mx{A}\mx{B}$, can be seen as a set of row vectors, while the second operand can be seen as a set of column vectors. Now, we have just seen that a matrix times a vector produces a vector (if the sizes match). Hence, we can think of the second operand $(\mx{B})$ as a set of column vectors that are transformed by the first operand, namely, $\mx{A}$. This can be expressed as

\begin{align} \mx{P} = \mx{A}\mx{B} = \mx{A} \left( \begin{array}{ccc} \vert & & \vert \\ \vc{b}_{,1} & \dots & \vc{b}_{,t} \\ \vert & & \vert \end{array} \right) = \left( \begin{array}{ccc} \vert & & \vert \\ \mx{A}\vc{b}_{,1} & \dots & \mx{A}\vc{b}_{,t} \\ \vert & & \vert \end{array} \right). \end{align}

(6.30)

That is, we can see the matrix-matrix multiplication like this: we start with $t$ column vectors, $\vc{b}_{,1}\dots \vc{b}_{,t}$ (in this case), and each of these are transformed by $\mx{A}$, and inserted as column vectors in the product matrix.

Next, we present a simple example where the dot product is expressed as matrix-matrix multiplication.

---

In this section, a number of useful matrices will be presented. This includes matrices for rotation, scaling, and shearing in both two and three dimensions. It should be noted that these can be generalized to higher dimensions as well.

6.4.1 Two Dimensions

---

There are many cases when it is desirable to apply a rotation. One may want to align a vector with another vector or simply rotate an object in order to animate it. The rotation matrix in two dimensions is simple to derive. A point $\vc{p}=(p_x,p_y)$ can be parameterized using a radius, $r$, and an angle, $\theta$, as $\vc{p}=(p_x,p_y)=(r\cos\theta, r\sin\theta)$. Rotating $\vc{p}$ by $\phi$ radians will produce a new vector $\vc{q}=(r\cos(\theta+\phi), r\sin(\theta+\phi))$, which can be rewritten as

\begin{align} \vc{q}=& \begin{pmatrix} r\cos(\theta+\phi) \\ r\sin(\theta+\phi) \end{pmatrix} = \begin{pmatrix} r (\cos\theta \cos\phi - \sin\theta \sin\phi) \\ r (\sin\theta \cos\phi + \cos\theta \sin\phi) \end{pmatrix} \\ =& \underbrace{ \left(\begin{array}{rr} \cos\phi & -\sin\phi \\ \sin\phi & \cos\phi \end{array}\right) }_{\mx{R}(\phi)} \underbrace{ \begin{pmatrix} r\cos\theta \\ r\sin\theta \end{pmatrix} }_{\vc{p}}, \end{align}

(6.39)

where we have used the angle sum relations $cos(\theta+\phi)=\cos\theta \cos\phi - \sin\theta \sin\phi$ and $\sin(\theta+\phi) = \sin\theta \cos\phi + \cos\theta \sin\phi$. Note that we separated out a $2\times 2$ matrix $\mx{R}(\phi)$ and that the rotated vector $\vc{q}$ is that matrix multiplied by the vector $\vc{p}$, i.e.,

\begin{align} \vc{q} = \mx{R}(\phi) \vc{p}. \end{align}

(6.40)

This leads to the following definition.

In Interactive Illustration 6.3, a rotation matrix is applied to the vertices of a rectangle, and the reader can change the angle, $\phi$, of the rotation matrix, $\mx{R}(\phi)$. A scaling matrix is very simple in that it has zeroes everywhere except in the diagonal elements. Hence, each diagonal element will be applied as a multiplicative factor to its respective dimension.

An example of how a scaling matrix applied to a rectangle can be seen in Interactive Illustration 6.4. The effect of a shear matrix is best seen before it is described in detail, so we recommend that the reader explores Interactive Illustration 6.5 first, and then a formal definition will be provided. Given the figure above, one can come to the conclusion that shearing is done using an identity matrix, where one of the zeroes have been replaced by a non-zero factor, $s$.

As an example, $\mx{H}_{xy}(s)$ means that the $x$-coordinate will be sheared using $s$ times the $y$-coordinate. This is, in fact, exactly what is shown in Interactive Illustration 6.5.

6.4.2 Three Dimensions

---

In three dimensions, rotation, scaling, and shearing behave in pretty much the same way as in two dimensions. However, there are more ways to perform each of these. For example, rotation in two dimensions occurred in the plane, but in three dimensions, it is possible to rotate around each axis. Let us start with rotation matrices.

Note that a rotation matrix around an axis, $i$, that is, using $\mx{R}_i(\phi)$, leaves the $i$-coordinates unaffected while the two remaining coordinates are rotated around axis $i$. It should be noted that it is possible to create rotation matrices around any arbitrary axis as well.

Recall that we use right-handed coordinate systems, if nothing else is mentioned. It is worth noting that $\mx{R}_x$ and $\mx{R}_z$ are similar to the two-dimensional rotation matrix (Definition 6.10), but $\mx{R}_y$ has the signs on the $\sin\phi$-terms flipped. This is so because we want to use positive orientations for $\phi$ for all three rotation matrices. For $\mx{R}_x$, for example, imagine that you are looking down the negative $x$-axis, which means you will see the $y$- and $z$-axes positively oriented. This is not the case, for the $x$- and $z$-axes, when looking down the negative $y$-axis for $\mx{R}_y$. Note that since $\cos(-\phi)=\cos\phi$ and $\sin(-\phi)=-\sin\phi$, rotating in the negative direction will flip the signs only on the $\sin\phi$-terms.

Scaling matrices in three dimensions are simpler since they only add scaling along the $z$-axis compared to a two-dimensional scaling matrix. This leads to the following definition.

In contrast to scaling in three dimensions, shearing can be done in several different ways. However, the matrices are still quite similar as shown in the following definition.

Sometimes, it is useful to have shear matrices with two parameters, which is a simple extension of the shear matrices with one parameter.

Some of the matrices in this chapter will be used to illustrate some of the properties of matrix arithmetic, which is the topic of the next section. For example, we will show that rotating first and then shearing is not the same as shearing first and then rotating. Hence, matrix multiplication is not commutative.

---

---

Note that $(v)$ and $(ix)$ are particularly convenient, since we can write both $\mx{A}+\mx{B}+\mx{C}$ and $\mx{A}\mx{B}\mx{C}$ (i.e., without any parenthesis), since the order does not matter. This is similar to how $1+2+3$ and $5\cdot 3 \cdot 2$ do not need any parenthesis.

Next, we present an example with $\mx{A}\mx{B}=\mx{O}$ (zero matrix defined in Definition 6.4) without either of $\mx{A}$ or $\mx{B}$ being $\mx{O}$.

It is also worth noting that Theorem 6.1 does not contain the rule $\mx{A}\mx{B}=\mx{B}\mx{A}$ and the reason is that this is not true in most cases. This is shown in the following example.

In Interactive Illustration 6.6, we show an example of how the order of two matrices in a matrix multiplication affects a rectangle when its vertices are interpreted as vectors and multiplied by a matrix,

---

Now that we have seen that matrix addition, matrix multiplication by a scalar, and matrix-matrix multiplication exist, it is reasonable to ask whether there also is a division-like operator. That is, how can we solve for $\mx{X}$ in

\begin{equation} \mx{A}\mx{X} = \mx{B}. \end{equation}

(6.54)

Now, let us take a step back, and start with a simpler expression using scalars only, i.e.,

\begin{equation} ax = b. \end{equation}

(6.55)

From algebra, the solution is trivially

\begin{equation} x = \frac{b}{a} = a^{-1}b, \end{equation}

(6.56)

Note the right hand side, where $a^{-1}$ is used as an operator solving Equation (6.55), is only valid if $a \neq 0$. This is the same notation that will be used for the matrix inverse, as we will see. However, if $a=0$ then all values of $x$ solves the equation if also $b=0$.

We will focus only on square matrix inverses, i.e., if $\mx{A}$ is square, find a solution to Equation (6.54), such that

\begin{equation} \mx{X} = \mx{A}^{-1}\mx{B}. \end{equation}

(6.57)

Only square matrices can have an inverse. Non-square matrices can have something similar to an inverse called a pseudo-inverse, but they are often less useful, and if nothing else is said, then we talk about square matrices when discussing matrix inverses. The definition of the matrix inverse follows below.

In the following, we will present a theorem that shows that if a matrix is invertible, then there is only one such matrix, and it works as a left-side and a right-side matrix.

---

Now that this theorem has been proved, we can omit the left-side and right-side matrix notation, and simply use $\mx{A}^{-1}$ as the matrix inverse notation. Next, a matrix inverse example follows.

In Example 6.10, we saw that for rotation, scaling, and shear matrices, it is straightforward to obtain the corresponding matrix inverse. However, so far, nothing has been said about how this can be done for general matrices. There are several different ways to do this. For two-dimensional matrices, it is particularly simple, as shown in the following theorem.

---

In Chapter 7, which is about determinants, it will become clear that the denominator in Theorem 6.3 is, in fact, the determinant of the $2\times 2$ matrix.

Next, we will show how the inverse can be computed using Gaussian elimination (Chapter 5).

Note that Chapter 7 will present other ways to compute the inverse as well.

---

---

We saw in Chapter 5 that to test if a set of vectors $\{\vc{u}_1, \vc{u}_2, \ldots, \vc{u}_q\}$ are independent or if they span $\R^p$, we have to study a set of $p$ equations in $q$ unknowns. In this chapter, we saw that matrices can be used to conveniently express system of linear equations.

---

---

---

---

---

---

Section 5.10 has already outlined how change of basis can be done, where the second basis $\{\hat{\vc{e}}_1,\hat{\vc{e}}_2\}$ can be expressed in terms of a first basis $\{\vc{e}_1,\vc{e}_2\}$, i.e.,

\begin{align} \hat{\vc{e}}_1 = b_{11} \vc{e}_1 + b_{21} \vc{e}_2,\\ \hat{\vc{e}}_2 = b_{12} \vc{e}_1 + b_{22} \vc{e}_2, \end{align}

(6.78)

where we have changed from $x_{ij}$ to $b_{ij}$ above, compared to Equation (5.102). As we will see, this is much more powerful when expressed in matrix form, since you then can get the transform in the other direction by using the inverse of the corresponding matrix. In the following theorem, the change of basis is generalized to any dimension and written in matrix form.

---

Note that per Equation (6.81), we have $\vc{v} = \mx{B}\vc{\hat{v}}$, which also means that $\vc{\hat{v}} = \mx{B}^{-1} \vc{v}$, assuming that $\mx{B}$ is invertible. It is often this expression that one is interested in.

---

A special set of matrices are the so called orthogonal matrices, and they have the convenient property that the inverse can be obtained by just taking the transpose. They are important because, for example, they can describe change of basis between two orthonormal bases. We start by the following definition.

Note that since the column vector constitute an orthonormal basis (Definition 3.3), it would make more sense to call the matrix orthonormal, but the term "orthogonal matrix" has a lot of legacy, so we will use it here as well. Given this short definition, the following theorem can be proved.

---

This means that the inverse of an orthogonal matrix is simply its transpose, that is, $\mx{A}^{-1} = \mx{A}^{\T}$, which is very convenient since the transpose is trivial to compute, while the inverse of an arbitrary square matrix usually is not.

---

---

---

---

In Section 6.1, we had one image to the left (original) and another image to the right. The right image is the left image manipulated in a certain way using a matrix. A TV or computer display contains of a number (often millions) of pixels (picture elements) and each pixel has a red, green, and a blue component. For each, pixel we can put these into a vector, i.e.,

\begin{equation} \vc{p} = \begin{pmatrix} r\\ g\\ b \end{pmatrix}, \end{equation}

(6.100)

where $r$ is the red component of the pixel, $g$ is the green component, and $b$ is the blue component. In the example, we also had a $3\times 3$ matrix $\mx{M}$ that was applied to each pixel. This was done as

\begin{align} \vc{p}' = \begin{pmatrix} r'\\ g'\\ b' \end{pmatrix}= \mx{M}\vc{p} = \begin{pmatrix} m_{11} && m_{12} && m_{13} \\ m_{21} && m_{22} && m_{23} \\ m_{31} && m_{32} && m_{23} \end{pmatrix} \begin{pmatrix} r\\ g\\ b \end{pmatrix}, \end{align}

(6.101)

Note that we have used $r$, $g$, $b$ for the vector components in this example instead of $x$, $y$, $z$ or $p_x$, $p_y$, $p_z$. This is to make it clearer what we are manipulating. Using the rules for matrix-vector multiplication, we see, for example, that $r' = m_{11} r+ m_{12}g+ m_{13}b$, etc. Hence, if we use the identity matrix, $\mx{I}$, the original image is obtained and using $\mx{M} = \left( \begin{smallmatrix} 1 & 1 & 1 \\ 0 & 0 & 0\\ 0 & 0 & 0 \end{smallmatrix} \right)$, we see that $r' = r+g+b$, while $g'=b'=0$, which results in a red image. Finally, if all rows in $\mx{M}$ are identical, we will obtain $r'=g'=b'$, i.e., a gray image. There are many more ways to manipulate images with, but matrices can take you a long way.