Research Outcome
Face Alignment
Let the Shape Speak - Discriminative Face Alignment using Conjugate Priors
This work presents a novel Bayesian formulation for aligning faces in unseen images. Our approach is closely related to Constrained Local Models (CLM) [2] and Active Shape Models (ASM) [6], where an ensemble of local feature detectors are constrained to lie within the subspace spanned by a Point Distribution Model (PDM).
Fitting a model to an image typically involves two steps: a local search using a detector, obtaining response maps for each landmark (like- lihood term) and a global optimization that finds the PDM parameters that jointly maximize all the detections. The global optimization can be seen as a Bayesian inference problem, where the posterior distribution of the PDM parameters (and pose) can be inferred in a maximum a posteriori (MAP) sense. We present a novel Bayesian global optimization strategy, where the prior is used to encode the dynamic transitions of the PDM parameters. Using recursive Bayesian estimation we model the prior dis- tribution of the data as being Gaussian. The mean and covariance were assumed to be unknown and treated as random variables.
(click to enlarge)
Discriminative Bayesian Active Shape Models
This work presents a simple and very efficient solution to align facial parts in unseen images. Our solution relies on a Point Distri- bution Model (PDM) face model and a set of discriminant local detectors, one for each facial landmark. The patch responses can be embedded into a Bayesian inference problem, where the posterior distribution of the global warp is inferred in a maximum a posteriori (MAP) sense. How- ever, previous formulations do not model explicitly the covariance of the latent variables, which represents the confidence in the current solution. In our Discriminative Bayesian Active Shape Model (DBASM) formu- lation, the MAP global alignment is inferred by a Linear Dynamical System (LDS) that takes this information into account.
(click to enlarge)
Face Alignment Through 2.5D Active Appearance Modelst
3D face models from single view through 2.5D Active Appeance Models.
The 2.5D AAM combine a 3D Point Distribution Model (PDM) and 2D appearance whose control points are defined by full perspective projections of the PDM.
Two fitting algorithms - Simultaneous Forwards Additive (SFA) and Normalization Forwards Additive (NFA) - and their computationally efficient approximations are proposed, based on the Lucas and Kanade framework. Expanded solutions for the SFA and NFA are also proposed taking into account head self occu.
(click to enlarge)
Towards Generic Fitting Using Multiple Features Discriminative Active Appearance Models
Solution for discriminative based Active Appearance Models (AAM).
The model consists in a set of descriptors which are covariances of multiple features evaluated over the neighborhood of landmarks whose locations are governed by a Point Distribution Model (PDM). The covariances are a special set of tensors that lie into a Riemannian Manifold. Is possible to mesure the dissimilarity and to update them, imposing temporal appearance consistency.
The fitting method uses a combination of exustive local search, finding modes with mean-shfit and clustering for each landmark independently. The global optimization then constrains each landmark location update by the PDM.
(click to enlarge)
Parametric Face Alignment: Generative and Discriminative Approaches
PhD Thesis - Pedro Martins
Faculty of Sciences and Technology - University of Coimbra
(click to enlarge)
Identity and Facial Expression Recognition
Identity and Expression Recognition on Low Dimensional Manifolds
Face geometry extracted using Active Appearance Models (AAM).
Low dimensional manifolds were then derived using Laplacian EigenMaps resulting in two types of manifolds, one for model identity and the other for expression. The recognition is composed by a two step cascade, where first the identity is predicted and then its associated expression model is used to predict the facial expression.
The identity overall recognition rate was 96.8%.Facial expression results are identity dependent, the most expressive individual achieves 81.2% of overall recognition rate.
(click to enlarge)
Facial Dynamics for Identity and Expression Recognition
Psychological studies indicate that facial dynamics is a biometric, i.e., it can be used for identity recognition. Based on this information, the present thesis attempts to demonstrate that facial motion alone is sufficient for performing person identification, through a series of experiments. As opposed to static analysis, research related to facial motion is a relatively new area of study. Besides identity recognition, expression recognition is also performed, using several techniques. The work developed includes the usage of different facial descriptors which make use of shape information, as is the case with Active Shape Models (ASM), and texture information, as in Local Binary Patterns (LBP). However, since it is desirable to analyse only the dynamic components, techniques are used for removing shape and texture information, such as image subtraction and optical flow fields.
(click to enlarge)
Gaze Estimation
”Are you looking at me?” An Eye2Eye Human Interaction Evaluation
Eye tracking systems are currently becoming more and more useful and practical. In several areas, such as security, medicine, ergonomics, etc...the study of the human eye using technological equipment is a big advantage. The existing eye gaze trackers either are intrusive methods (using IR illumination) or require an exhaustive personal calibration. Besides, most of gaze estimators assume a fixed head pose so that they can ignore the 6 degrees of freedom that free head motion adds to the problem. What is presented in this thesis is a visual contact evaluator for conversations between two users. This evaluator functionality is based on the gaze direction of the users, under free head movements, using a pair of calibrated cameras. The head pose estimation is based on rigid head models combined with Active Appearance Models (AAM) for head tracking. The gaze direction is estimated based on a 3D geometric model of the eyeball. Finally, it is considered that there are visual contact when both users are looking at each other’s eyes.
(click to enlarge)
DEMOS
Demonstration videos