Louis-Philippe Morency: Contextual Recognition of Head Gestures
Louis-Philippe Morency
Date: Friday, March 17, 2006
Time: 11 AM
Location: 6th floor amphitheater
Host:
Head pose and gesture offer several key conversational grounding cues and are used extensively inface-to-face interaction among people. In this talk, we investigate how dialog context from an embodied conversational agent (ECA) can improve visual recognition of user gestures. We present a recognition framework which (1) extracts contextual features from an ECA’s dialog manager, (2) computes a prediction of head nod and head shakes, and (3) integrates the contextual predictions with the visual observation of a vision-based head gesture recognizer. We found a subset of lexical, punctuation and timing features that are easily available in most ECA architectures and can be used to learn how to predict user feedback. Using a discriminative approach to contextual prediction and multi-modal integration, we were able to improve the performance of head gesture detection even when the topic of the test set was significantly different than the training set.