Context-based Recognition during Human Interactions: Automatic Feature Selection and Encoding Dictionary


Share/Save/Bookmark

Morency, Louis-Philippe and Kok, Iwan de and Gratch, Jonathan (2008) Context-based Recognition during Human Interactions: Automatic Feature Selection and Encoding Dictionary. In: 10th International Conference on Multimodal Interfaces, ICMI, October 20 - 22, 2008, Chania, Crete, Greece (pp. pp. 181-188).

[img] PDF
Restricted to UT campus only
: Request a copy
539kB
Abstract:During face-to-face conversation, people use visual feedback such as head nods to communicate relevant information and to synchronize rhythm between participants. In this paper we describe how contextual information from other partic- ipants can be used to predict visual feedback and improve recognition of head gestures in human-human interactions. For example, in a dyadic interaction, the speaker contextual cues such as gaze shifts or changes in prosody will in uence listener backchannel feedback (e.g., head nod). To auto- matically learn how to integrate this contextual information into the listener gesture recognition framework, this paper addresses two main challenges: optimal feature representa- tion using an encoding dictionary and automatic selection of optimal feature-encoding pairs. Multimodal integration between context and visual observations is performed using a discriminative sequential model (Latent-Dynamic Condi- tional Random Fields) trained on previous interactions. In our experiments involving 38 storytelling dyads, our context- based recognizer signicantly improved head gesture recog- nition performance over a vision-only recognizer.
Item Type:Conference or Workshop Item
Copyright:© 2008 ACM
Faculty:
Electrical Engineering, Mathematics and Computer Science (EEMCS)
Research Group:
Link to this item:http://purl.utwente.nl/publications/69012
Official URL:http://dx.doi.org/10.1145/1452392.1452426
Export this item as:BibTeX
EndNote
HTML Citation
Reference Manager

 

Repository Staff Only: item control page