Awards presentation held at UNESCO headquarters in Paris on March 26.
The USC Institute for Creative Technologies’ SimSensei prototype was named one of the year’s top ten most promising digital initiatives by the NetExplo Forum, a partnership with UNESCO to recognize technologies that are shaping the future in areas including education, health and communication.
SimSensei is a virtual human interviewer that can be used to identify signals of depression and other mental health issues. In recognizing ICT’s innovation, NetExplo’s event organizer’s noted SimSensei’s potential as, “a state-of-the-art tool that health care providers can use for screening and monitoring patients.”
SimSensei leverages ICT’s advances in developing interactive virtual humans – computer-generated characters that use language, have appropriate gestures, show emotion, and react to verbal and non-verbal stimuli. It also incorporates ICT’s MultiSense technology which provides real-time tracking and analysis of non-verbal behaviors, including facial expressions, eye gaze, body posture and voice intonation. From these signals, SimSensei can engage a user in conversation, follow up with appropriate questions based on an individual’s answers and body language, and use this data to infer signs of emotional distress. SimSensei is not designed for therapy or medical diagnosis, but is intended as a support tool for clinicians and healthcare providers.
“Think about SimSensei as a diagnostic tool, like a blood sample,” said Louis-Philippe Morency, director of ICT’s MultiComp Lab, who co-leads the project with Skip Rizzo, director for medical virtual reality at ICT. “You send a blood sample to the lab and you get the result. The people doing the diagnosis are the clinicians, but they use these objective measures to make the diagnosis.”
The SimSensei research and development effort is funded as part of DARPA’s Detection and Computational Analysis of Psychological Signals (DCAPS) project. This effort aims to address the high numbers of soldiers affected with post-traumatic stress by developing new tools to assess their mental health and enable them to seek timely help.
“Many people suffer in silence because they fear the stigma that may come from seeking help through traditional channels or because they simply don’t know where to turn,” said Rizzo. “Computer‐mediated care offers anonymity and access that may help reach these service men and women who need it most and could help support them in deciding to seek help with a live provider.”
Additional ICT projects are using the underlying MultiSense technology to enable richer human-computer interactions. These include an interactive virtual human audience that can give feedback to people looking to improve their public speaking skills and virtual human role players that can help people to prepare for job interviews.
About the USC Institute for Creative Technologies
At the University of Southern California Institute for Creative Technologies (ICT) leaders in artificial intelligence, graphics, virtual reality and narrative advance low-cost immersive techniques and technologies to solve problems facing service members, students and society. Established in 1999, ICT is a DoD-sponsored University Affiliated Research Center working in collaboration with the U.S. Army Research Laboratory. ICT brings film and game industry artists together with computer and social scientists to study and develop immersive media for military training, health therapies, education and more.