A Multi-layer Hybrid Framework for Dimensional Emotion Classification

Nicolaou, Mihalis A. and Gunes , Hatice and Pantic, Maja (2011) A Multi-layer Hybrid Framework for Dimensional Emotion Classification. In: 19th ACM International Conference on Multimedia, MM 2011, 28 November - 1 December 2011, Scottsdale, USA.

PDF
Restricted to UT campus only: Request a copy
819Kb

Abstract:	This paper investigates dimensional emotion prediction and classification from naturalistic facial expressions. Similarly to many pattern recognition problems, dimensional emotion classification requires generating multi-dimensional outputs. To date, classification for valence and arousal dimensions has been done separately, assuming that they are independent. However, various psychological findings suggest that these dimensions are correlated. We therefore propose a novel, multi-layer hybrid framework for emotion classification that is able to model inter-dimensional correlations. Firstly, we derive a novel geometric feature set based on the (a)symmetric spatio-temporal characteristics of facial expressions. Subsequently, we use the proposed feature set to train a multi-layer hybrid framework composed of a tem- poral regression layer for predicting emotion dimensions, a graphical model layer for modeling valence-arousal correlations, and a final classification and fusion layer exploiting informative statistics extracted from the lower layers. This framework (i) introduces the Auto-Regressive Coupled HMM (ACHMM), a graphical model specifically tailored to accommodate not only inter-dimensional correlations but also to exploit the internal dynamics of the actual observations, and (ii) replaces the commonly used Maximum Likelihood principle with a more robust final classification and fusion layer. Subject-independent experimental validation, performed on a naturalistic set of facial expressions, demonstrates the effectiveness of the derived feature set, and the robustness and flexibility of the proposed framework.
Item Type:	Conference or Workshop Item
Copyright:	© 2011 ACM
Faculty:	Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:	Human Media Interaction (HMI)
Link to this item:	http://purl.utwente.nl/publications/79394
Official URL:	http://dx.doi.org/10.1145/2072298.2071906
Export this item as:	BibTeX EndNote HTML Citation Reference Manager

Show download statistics for this publication

Repository Staff Only: item control page