Jina Lee, Stacy Marsella: “Learning a Model of Speaker Head Nods using Gesture Corpora”

May 10, 2009 | Budapest, Hungary

Speaker: Jina Lee, Stacy Marsella
Host: 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2009)

During face-to-face conversation, the speaker’s head is continually in motion. These movements serve a variety of important communicative functions, and may also be influ- enced by our emotions. The goal for this work is to build a domain-independent model of speaker’s head movements and investigate the effect of using affective information dur- ing the learning process. Once the model is learned, it can later be used to generate head movements for virtual agents. In this paper, we describe our machine-learning approach to predict speaker’s head nods using an annotated corpora of face-to-face human interaction and emotion labels gener- ated by an affect recognition model. We describe the feature selection process, training process, and the comparison of results of the learned models under varying conditions. The results show that using affective information can help pre- dict head nods better than when no affective information is used.