Chapter 11: Matrix Factorizations

Example 11.1: Structure from Motion using Matrix Factorization
An approximative model for the image projection of a 3D point with coordinates $(x,y,z)$ to an image point with coordinates $(u,v)$ is the so called affine camera model

\begin{equation} \begin{pmatrix} u\\ v \end{pmatrix} = \begin{pmatrix} p_{11} & p_{12} & p_{13} & p_{14} \\ p_{21} & p_{22} & p_{23} & p_{24} \end{pmatrix} \begin{pmatrix} x\\y\\z\\1 \end{pmatrix} , \end{equation}

(11.1)

which we can write (using matrix terminology) as

\begin{equation} \vc{u} = \mx{P} \vc{x}. \end{equation}

(11.2)

Assume that $n$ points with column vectors $\vc{x}_1, \ldots, \vc{x}_n$ are all seen in $m$ images and denote by $\vc{u}_{i,j}$ the column vector $\vc{u}$ for point $j$ seen in image $i$. Let the camera matrix for the $m$ images be $\mx{P}_1, \ldots, \mx{P}_m$.

The $mn$ constraints can now be written as one big matrix equation

\begin{equation} \begin{pmatrix} \vc{u}_{1,1} & \ldots & \vc{u}_{1,n} \\ \hdots & \hdots & \hdots \\ \vc{u}_{m,1} & \ldots & \vc{u}_{m,n} \\ \end{pmatrix} = \begin{pmatrix} \mx{P}_1\\ \hdots \\ \mx{P}_m \end{pmatrix} \begin{pmatrix} \vc{x}_1 & \ldots & \vc{x}_n \end{pmatrix} . \end{equation}

(11.3)

This can be written

\begin{equation} \mx{U} = \mx{M} \mx{X}. \end{equation}

(11.4)

The matrix to the left $\mx{U}$ contain the measured (known) image coordinates that have been detected in the images. To the right we have the product of two matrices $ \mx{M}$ and $\mx{X}$ that contain the parameters that we would like to estimate. If we only could factorize or decompose $\mx{U}$ as $ \mx{M} \mx{X}$, this would solve our problem.