Using 3D morphable models for face recognition in video


Rootseler, R.T.A. van and Spreeuwers, L.J. and Veldhuis, R.N.J. (2012) Using 3D morphable models for face recognition in video. In: 33rd WIC Symposium on Information Theory in the Benelux, 24-25 May 2012, Boekelo, the Netherlands (pp. pp. 235-242).

open access
Abstract:The 3D Morphable Face Model (3DMM) has been used for over a decade for creating 3D models from single images of faces. This model is based on a PCA model of the 3D shape and texture generated from a limited number of 3D scans. The goal of fitting a 3DMM to an image is to find the model coefficients, the lighting and other imaging variables from which we can remodel that image as accurately as possible. The model coefficients consist of texture and of shape descriptors, and can without further processing be used in verification and recognition experiments. Until now little research has been performed into the influence of the diverse parameters of the 3DMM on the recognition performance. In this paper we will introduce a Bayesian-based method for texture backmapping from multiple images. Using the information from multiple (non-frontal) views we construct a frontal view which can be used as input to 2D face recognition software. We also show how the number of triangles used in the fitting proces influences the recognition performance using the shape descriptors. The verification results of the 3DMM are compared to state-of-the-art 2D face recognition software on the MultiPIE dataset. The 2D FR software outperforms the Morphable Model, but the Morphable Model can be useful as a preprocesser to synthesize a frontal view from a non-frontal view and also combine images with multiple views to a single frontal view. We show results for this preprocessing technique by using an average face shape, a fitted face shape, with a MM texture, with the original texture and with a hybrid texture. The preprocessor has improved the verification results significantly on the dataset.
Item Type:Conference or Workshop Item
Copyright:© 2012 WIC
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:
Proceedings URL:
Organisation URL:
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page