Image formation

Image formation involves the projection of points in (the world) to points in (the image plane). The perspective projection equations with which we are familiar,

$$\begin{eqnarray*}x = -f\frac{X}{Z} \\ y = -f\frac{Y}{Z} \end{eqnarray*}$$

where the point (X,Y,Z) in the world is projected to the point (x,y)on the image plane, are inherently nonlinear. Converting to homogeneous coordinates, however, makes them linear:

$$\ensuremath{{\bf p}} ' = T_{perspective}\ensuremath{{\bf p}} ,$$

where $\ensuremath{{\bf p}} ' = [x,y,w]^T$, $\ensuremath{{\bf p}} = [X,Y,Z,W]^T$, and the perspective projection matrix T is given by:

$$T_{perspective} = \left[\matrix{-f & 0 & 0 & 0 \cr 0 & -f & 0 & 0 \cr 0 & 0 & 1 & 0}\right].$$

The entire image formation process includes perspective projection, along with matrices for internal and external calibration:

  $$\begin{eqnarray*}\ensuremath{{\tilde P}} = T_{internal} T_{perspective} T_{exter... ...atrix{R & \ensuremath{{\bf t}} }\right] \nonumber \\ & = & A D, \end{eqnarray*}$$ (4)

where $\alpha_u$ and $\alpha_v$ are the scale factors of the image plane (in units of the focal length f), $\theta$ is the skew ( $\theta=\pi/2$ for most real cameras), the point (u₀,v₀) is the principal point, R is the $3 \times 3$ rotation matrix, and $\ensuremath{{\bf t}}$ is the $3 \times 1$ translation vector. The matrix A contains the internal parameters and perspective projection, while D contains the external parameters.

It is sometimes convenient to decompose the $3 \times 4$ projection matrix $\ensuremath{{\tilde P}}$ into a $3 \times 3$ matrix P and a $3 \times 1$ vector p

$$\ensuremath{{\tilde P}} = \left[\matrix{P & p}\right]$$

so that

$$P = AR \; \; \mbox{and} \; \; p = A \ensuremath{{\bf t}} .$$ (5)