If the camera (or human eye) moves in the 3-D scene, the resulting apparent motion in the image is called the optical flow. The optical flow describes the direction and the speed of motion of the features in the image. We wil now show how to compute the optical flow from sequence of images and then derive an equations relating the 3-D structure of the world to our 2-D measurements.
An example of the optical flow pattern from the sequence of images of a rotating Rubik's cube shown in Figure 2.
Figure 2: A Rubik's cube on a rotating turntable, taken from Russell and Norvig, ``AI, A Modern Approach'', Prentice Hall, 1995, Figure 24.8, pg. 736.
is given in Figure 3.
Figure 3: Flow vectors calculated from comparing the two images of a Rubik's cube, taken from Russell and Norvig, ``AI, A Modern Approach'', Prentice Hall, 1995, Figure 24.9, pg. 737.
The optical flow vector has two components and describing the motion of a point feature in and direction in the image plane respectively. In order to be able to measure optical flow we need to find corresponding points between two frames. One possibility would be to exploit the similarity between the image patches surrounding the individual points. There are two measures of similarity that are commonly used for optical flow vectors. One is sum of squared differences (SSD) between an image patch centered at a point , which is a location at time and various other candidate locations where that patch could have moved between two frames at time and The goal here is to find a displacement in the image plane , which minimizes the SSD criterion:
where the summation ranges over the image patch centered at the feature of interest. The optical flow of the chosen point feature is
. An alternative measure of similarity would be to maximize the cross-correlation between the neighboring patches in the images expressed as
It is important to note that the optical flow is not uniquely determined by the local information. For example consider two local patches of the images observed in two consecutive times (see Figure 4).
Figure 4: In spite of the fact that the dark square moved between the two consecutive frames, observing purely the cut-out patch we cannot observe any change, or we may assume that the observed pattern moved arbitrarily along the direction of the edge.
In spite of the fact that the dark square moved between the two consecutive frames, observing purely the cut-out patch we cannot observe any change, or we may assume that the observed pattern moved arbitrarily along the direction of the edge. The fact that one cannot determine the optical flow along the direction of the brightness pattern is known as aperture problem. Have you ever thought about what might cause the barber pole illusion, where it appears that a barber shop pole is spiralling up and out. Indeed, full optical flow is best determined at the corners.