Classifying laughter and speech using audio-visual feature prediction


Share/Save/Bookmark

Petridis, Stavros and Asghar, Ali and Pantic, Maja (2010) Classifying laughter and speech using audio-visual feature prediction. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2010, 14-19 March 2010, Dallas, TX, USA (pp. pp. 5254-5257).

open access
[img]
Preview
PDF
87kB
Abstract:In this study, a system that discriminates laughter from speech by modelling the relationship between audio and visual features is presented. The underlying assumption is that this relationship is different between speech and laughter. Neural networks are trained which learn the audio-to-visual and visual-to-audio features mapping for both classes. Classification of a new frame is performed via prediction. All the networks produce a prediction of the expected audio/visual features and the network with the best prediction, i.e., the model which best describes the audiovisual feature relationship, provides its label to the input frame. When trained on a simple dataset and tested on a hard dataset, the proposed approach outperforms audiovisual feature-level fusion, resulting in a 10.9% and 6.4% absolute increase in the F1 rate for laughter and classification rate, respectively. This indicates that classification based on prediction can produce a good model even when the available dataset is not challenging enough.

Item Type:Conference or Workshop Item
Copyright:© 2010 IEEE
Faculty:
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/75891
Official URL:http://dx.doi.org/10.1109/ICASSP.2010.5494992
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page