A probabilistic multimodal approach for predicting listener backchannels

Morency, Louis-Philippe and Kok de, Iwan and Gratch, Jonathan (2009) A probabilistic multimodal approach for predicting listener backchannels. Autonomous Agents and Multi-Agent Systems, 20 (1). pp. 70-84. ISSN 1387-2532

PDF
Restricted to UT campus only: Request a copy
346Kb

Abstract:	During face-to-face interactions, listeners use backchannel feedback such as head nods as a signal to the speaker that the communication is working and that they should continue speaking. Predicting these backchannel opportunities is an important milestone for building engaging and natural virtual humans. In this paper we show how sequential probabilistic models (e.g., Hidden Markov Model or Conditional Random Fields) can automatically learn from a database of human-to-human interactions to predict listener backchannels using the speaker multimodal output features (e.g., prosody, spoken words and eye gaze). The main challenges addressed in this paper are automatic selection of the relevant features and optimal feature representation for probabilistic models. For prediction of visual backchannel cues (i.e., head nods), our prediction model shows a statistically significant improvement over a previously published approach based on hand-crafted rules.
Item Type:	Article
Copyright:	© 2009 Springer
Faculty:	Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:	Human Media Interaction (HMI)
Link to this item:	http://purl.utwente.nl/publications/68958
Official URL:	http://dx.doi.org/10.1007/s10458-009-9092-y
Export this item as:	BibTeX EndNote HTML Citation Reference Manager

Show download statistics for this publication

Repository Staff Only: item control page

Metis ID: 270701