This report addresses the problem of modeling the appearance of human bodies. Current computer graphics techniques for photorealistic human modeling are difficult and expensive, and are thus only feasible for high-end production, such as [14,19]. Furthermore, it is nearly impossible to produce truly convincing images that do not appear computer-generated -- imagine trying to generate convincing animation of actors based on old footage. For many applications, such as teleconferencing and video games, an efficient and effective modeling technique is desirable.
Our method is based on ideas from image-based rendering. The figure is conceptually modeled as a kinematic chain, used to parameterize the rigid figure motion. Each body segment is rendered by non-rigid morphing of the example images. These deformations are accurately represented, because the morphing is done in an aligned frame of reference, similar to the rectification phase of View Morphing [20].
The idea of this work is to combine the advantages of both rigid and non-rigid transformations. The body is modelled as a rigid, articulated figure. However, the deformation of each segment is modeled with a Radial Basis Function (RBF). We rigidly transform each example for a segment to the same orientation, and then do RBF interpolation. (RBF interpolation can be thought of simply as morphing, with particular choices for the interpolation coefficients.) This means that the RBFs only model non-rigid shape/texture changes, and do not attempt to model rigid changes in orientation or position. The appeal of this method is to use each technique for what it is good at: rigid transformations model the overall rigid transformations of the segments, and RBFs pick up the slack by modeling the additional non-rigid transformations. This method also has the potential to properly decouple unrelated body segments for interpolation (i.e. moving the arm doesn't cause the leg to move).
All of the image analysis in this work were done by hand. However, many of the subproblems (segmentation, articulated tracking, dense correspondence) are areas of continuing work in computer vision (e.g. [7,3,4]). Many of the steps of the analysis could be done automatically or by an assisted system.