Implicit, view-invariant modelling of 3D non-rigid objects
This thesis describes and evaluates the Integrated Shape and Pose Model (ISPM), a novel technique for modelling the geometry of a 3D non-rigid object, such as a face, via images captured from various viewpoints. The ISPM can be trained on almost any set of images since it does not require images captured simultaneously from more than one view. This is advantageous over conventional techniques that impose such constraints on the training data. The ISPM is built by transferring the object's intrinsic shape from the view of each training image, to two basis views. This is achieved by first computing the Centred Affine Trifocal Tensor (CATT) between the view of each given image and the basis views, which implicitly encodes the 3D pose of the object. The object's intrinsic shape is then transferred to the basis views by enforcing the epipolar constraints provided by the CATT followed by an affine alignment. This process (the Implicit Pose Alignment (IPA) algorithm) requires the mean basis view images, which are not initially known. Therefore, the generalized Procrustes alignment algorithm is extended, by employing the IPA algorithm to perform the alignment steps. The extended Procrustes alignment algorithm simultaneously generates the mean basis view images and achieves the required intrinsic shape transfer. The key benefit of our approach is illustrated by the significant improvement in view-invariance and consistency in the ISPM's modelling errors as well as its specificity, in comparison to those of conventional image-based models. The ISPM is evaluated on four databases of real and synthetic face images containing variations in identity, expression and pose. Its various algorithms are also individually evaluated and their performance critically assessed. Future work on incorporating grey-level values may also be possible and, is briefly explored. Our approach may also be of relevance to theories of human vision.