Image Analysis

The input images must first be analyzed and annotated. There are several steps to the analysis.

1.

Correspondence: Dense correspondence between the image points on the figure is computed between each frame.

2.

Segmentation: The figure is separated into kinematic links. The segmentation must be compatible with the correspondence (i.e. the same segmentation is used for each example image).

3.

Linkages: The corresponding axes for each segment is identified in each frame. Each axis consists of a pivot point and a direction. The direction is indicated by an end point, although the absolute position of the end point is only important if the end point is used as the pivot point for another link.

In our experiments, these steps were done with two input images which were annotated by hand-labelled contour correspondences and split into two segments.

Using the contour points, dense correspondence between contours is generated in a standard way. A Delaunay triangulation of the contours is generated for each segment, followed by a mapping from each pixel in one triangle to the corresponding pixel in the corresponding triangle in the other image. Note that some contours may have a different Delaunay triangulation than the corresponding contour; we pick the triangulation that has better aspect ratios in both images. Triangle correspondence is computed as follows: We begin with a triangle (p₀, p₁, p₂) in image 1, and the corresponding triangle in image 2 (q₀, q₁, q₂) expressed in homogeneous coordinates. To find the corresponce q in image 2 for a point p in image one, we compute the barycentric coordinates of p in triangle one, and the image coordinates of q in triangle 2: q = [ q₀, q₁, q₂ ] [ p₀, p₁, p₂ ]⁺ p. The pseudoinverse is used in case of degenerate triangles.

We then annotate the input image with articulated segments in the plane. A fist, elbow, and shoulder point were each selected from the image contours. These points are in correspondence, and the elbow point represents the pivot point of the forearm. All of the image points in the forearm half of the segmentation belong to the fist-elbow axis, and all of the remaining points belong to the elbow-shoulder axis.