Detection and Computational Analysis of Psychological Signals of Distress Using a Virtual Human Interviewing Agent

November 6, 2015 | New Orleans, Louisiana

Speaker: Stefan Scherer
Host: International Society for Traumatic Stress Studies (ISTSS) 31st Annual Meeting

Facial expressions, body gestures, and vocal parameters play an important role in human communication and the signaling of emotion. Advances in low cost computer vision and behavioral sensing technologies (webcams, depth-sensing cameras, and 3D microphones) can now be applied for capturing human behavior and subsequently providing probabilistic inferences regarding the psychological state of a person when they interact with a live or virtual human interviewer. The SimSensei Virtual Human (VH) interviewing system provides participants with the opportunity to engage in a private conversational interview with a VH character who asks an interviewee open ended questions that are responded to with natural language. During the course of the interview, cameras and a microphone capture the user’s facial/body and vocal communication “signals” for quantification and interpretation. We will present SimSensei interview data from a National Guard unit (n=25) obtained prior to and immediately following a combat deployment to Afghanistan. Initial analyses suggest differences in pre to post interviews were found on automatic measures of facial expression related to displays of “joy” and “fear”. Interviewees were also more likely to endorse the presence of adverse psychological symptoms to the virtual human (Ellie) compared to their responses on the PDHA questionnaire.