Preliminaries

All points and vectors are expressed in homogeneous coordinates. In 2D, points and vectors are represented as [x,y,1]^T, and [x,y,0]^T, respectively. 3D points and vectors may be represented with an extra depth (z) component, or with a disparity component; 3D transformations must be computed accordingly. For disparity maps, a rigid object motion can be simulated with a rigid camera motion (see Appendix B). The presentation in the remainder of this paper applies equally to 2D (planar) or 3D transformations.

We represent the appearance of a body segment as a matrix called an image list: $A=\mbox{$\left [ \begin{array} {c} {S} \\ {X}\end{array} \right ]$}=\left [ \b... ...ay} {cccc} p_1 & p_2 & p_3 & ... \\ t_1 & t_2 & t_3 & ... \end{array} \right ]$, where each t_i=[r_i,g_i,b_i]^T contains texture information. This representation is similar to a layered depth image [22], but is more convenient for expressing point correspondences. (Layered depth images allow efficient incremental computation during reprojection.)