Research Assistant Intern 0922
January 08, 2009
Project Name
Nonverbal Behavior Understanding
Project Description
Nonverbal behavior understanding - During face-to-face conversation, people use nonverbal behaviors to communicate relevant information and to synchronize communicative rhythm between participants. Our research aims at bringing the human ability to recognize nonverbal behaviors to novel embodied interfaces such as robots and virtual humans. When recognizing nonverbal behaviors, people use more than their visual perception; knowledge about the current topic and expectations from previous utterances help guide recognition of visual cues such as head nodding and facial expressions. Since many of these visual gestures are subtle, the use of contextual information helps people to disambiguate between voluntary and non-voluntary gestures.
Job Description
Natural conversation is fluid and highly interactive. Participants seem tightly enmeshed in something like a dance, rapidly detecting and responding, not only to each other’s words, but to speech prosody, gesture, gaze, posture, and facial expression movements. The goal of this internship is to automatically learn predictive models of human verbal and nonverbal behaviors using a probabilistic multimodal approach. These predictive models can improve performance of nonverbal behaviors recognition and also be used to generate nonverbal behaviors of a robot or virtual human.
Building on top of previous work that showed the importance of these predictive models during human-robot interactions [AI journal 2007] as well as human-human interactions [IVA 2008], this internship will focus on training multimodal predictive models of head gestures (head nod, head tilt), eye gestures (gaze aversion, deictic gestures), facial expressions (smiling, confusion) and arm gestures (beating gestures, pointing gestures). The intern will focus on (1) determining the optimal feature representation for the multimodal context, (2) develop algorithms to automatically select relevant features, and (3) experiment on multiple pre-existing datasets (virtual rapport and AMI meeting datasets) to show the generalization among different domains.
Skills:
• Experience with human behavior analysis, more specifically nonverbal behaviors
• Machine learning experience (e.g., HMM, SVM)
• Linear algebra and signal processing (e.g., FFT, MFCC)
• Good programming knowledge of Matlab and C++