Speech-based recognition of self-reported and observed emotion in a dimensional space


Truong, Khiet P. and Leeuwen, David A. van and Jong, Franciska M.G. de (2012) Speech-based recognition of self-reported and observed emotion in a dimensional space. Speech communication, 54 (9). pp. 1049-1063. ISSN 0167-6393

[img] PDF
Restricted to UT campus only
: Request a copy
Abstract:The differences between self-reported and observed emotion have only marginally been investigated in the context of speech-based automatic emotion recognition. We address this issue by comparing self-reported emotion ratings to observed emotion ratings and look at how differences between these two types of ratings affect the development and performance of automatic emotion recognizers developed with these ratings. A dimensional approach to emotion modeling is adopted: the ratings are based on continuous arousal and valence scales. We describe the TNO-Gaming Corpus that contains spontaneous vocal and facial expressions elicited via a multiplayer videogame and that includes emotion annotations obtained via self-report and observation by outside observers. Comparisons show that there are discrepancies between self-reported and observed emotion ratings which are also reflected in the performance of the emotion recognizers developed. Using Support Vector Regression in combination with acoustic and textual features, recognizers of arousal and valence are developed that can predict points in a 2-dimensional arousal-valence space. The results of these recognizers show that the self-reported emotion is much harder to recognize than the observed emotion, and that averaging ratings from multiple observers improves performance.
Item Type:Article
Copyright:© Elsevier
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/80909
Official URL:https://doi.org/10.1016/j.specom.2012.04.006
Export this item as:BibTeX
HTML Citation
Reference Manager


Repository Staff Only: item control page