Institute for Creative Technologies » Jennifer Wohlner

Celso de Melo: “Model of the Social Effects of Emotion in Decision-Making in Multiagent Systems”

Jennifer Wohlner — Fri, 08 Jun 2012 00:00:55 +0000

Research in the behavioral sciences suggests that emotion can serve important social functions and that, more than a simple manifestation of internal experience, emotion displays communicate one’s beliefs, desires and intentions. In a recent study we have shown that, when engaged in the iterated prisoner’s dilemma with agents that display emotion, people infer, from the emotion displays, how the agent is appraising the ongoing interaction (e.g., is the situation favorable to the agent? Does it blame me for the current state-of-affairs?). From these appraisals people, then, infer whether the agent is likely to cooperate in the future. In this paper we propose a Bayesian model that captures this social function of emotion. The model supports probabilistic predictions, from emotion displays, about how the counterpart is appraising the interaction which, in turn, lead to predictions about the counterpart’s intentions. The model’s parameters were learnt using data from the empirical study. Our evaluation indicated that considering emotion displays improved the model’s ability to predict the counterpart’s intentions, in particular, how likely it was to cooperate in a social dilemma. Using data from another empirical study where people made inferences about the counterpart’s likelihood of cooperation in the absence of emotion displays, we also showed that the model could, from information about appraisals alone, make appropriate inferences about the counterpart’s intentions. Overall, the paper suggests that appraisals are valuable for computational models of emotion interpretation. The relevance of these results for the design of multiagent systems where agents, human or not, can convey or recognize emotion is discussed.

Matthew Jensen Hays, H. Chad Lane, Daniel Auerbach: “Clear and presence danger: Feedback in serious games.”

Jennifer Wohlner — Fri, 01 Jun 2012 00:00:14 +0000

Serious games are generally designed with two goals in mind. First, they are designed to promote learning. Second, like most games, they seek to create compelling and engaging experiences, often referred to as a sense of presence. Presence itself is believed to promote learning, but serious games often incorporate additional features to increase pedagogical value. One such feature is the use of an intelligent tutoring system (ITS) to provide feedback during gameplay. Because feedback from an ITS is usually extrinsic (i.e., it operates outside of the primary game mechanic), attending to it may disrupt players’ sense of presence, thereby hindering learning. To avoid this potential disruption, ITS feedback has been removed from some serious games, and omitted from the design of others. However, the most beneficial conditions of instruction and practice are often counterintuitive; in this paper, we challenge the assumption that feedback during learning hinders sense of presence. Across three experiments, we examined how an ITS that provided extrinsic feedback during a serious game affected presence. Across different modalities and conditions, we found that feedback not affect presence (although manipulations of visual richness did). Presence was also unaffected when we manipulated different features of the ITS, such as the participants’ ability to control feedback delivery, or even their displeasure with feedback frequency. Our results suggest that it is possible to provide extrinsic feedback in a serious game without detracting from the immersive power of the game itself.

Kallirroi Georgila, Alan W. Black, Kenji Sagae, David Traum: “Practical Evaluation of Human and Synthesized Speech for Virtual Human Dialogue Systems”

Jennifer Wohlner — Fri, 25 May 2012 00:00:25 +0000

The current practice in virtual human dialogue systems is to use professional human recordings or limited-domain speech synthesis. Both approaches lead to good performance but at a high cost. To determine the best trade-off between performance and cost, we perform a systematic evaluation of human and synthesized voices with regard to naturalness, conversational aspect, and likability. We also vary the type (in-domain vs. out-of-domain), length, and content of utterances, and take into account the age and native language of raters as well as their familiarity with speech synthesis. We present detailed results from two studies, a pilot one and one run on Amazon’s Mechanical Turk. Our results suggest that a professional human voice can supersede both an amateur human voice and synthesized voices. Also, a high-quality general-purpose voice or a good limited-domain voice can perform better than amateur human recordings. We do not find any significant differences between the performance of a high-quality general-purpose voice and a limited-domain voice, both trained with speech recorded by actors. As expected, in most cases, the high-quality general-purpose voice is rated higher than the limited-domain voice for out-of-domain sentences and lower for in-domain sentences. There is also a not statistically significant trend for long or negative-content utterances to receive lower ratings.

Priti Aggarwal, Ron Artstein, Jillian Gerten, Athanasios Katsamanis, Shrikanth Narayanan, Angela Nazarian, David Traum: “The Twins Corpus of Museum Visitor Questions”

Jennifer Wohlner — Thu, 24 May 2012 00:00:27 +0000

The Twins corpus is a collection of utterances spoken in interactions with two virtual characters who serve as guides at the Museum of Science in Boston. The corpus contains about 200,000 spoken utterances from museum visitors (primarily children) as well as from trained handlers who work at the museum. In addition to speech recordings, the corpus contains the outputs of speech recognition performed at the time of utterance as well as the system interpretation of the utterances. Parts of the corpus have been manually transcribed and annotated for question interpretation. The corpus has been used for improving performance of the museum characters and for a variety of research projects, such as phonetic-based Natural Language Understanding, creation of conversational characters from text resources, dialogue policy learning, and research on patterns of user interaction. It has the potential to be used for research on children’s speech and on language used when talking to a virtual human.

Paul Debevec: “From Spider-Man to Avatar, Emily and Benjamin: Achieving Photoreal Digital Actors”

Jennifer Wohlner — Tue, 15 May 2012 00:00:20 +0000

Somewhere between “Final Fantasy” in 2001 and “The Curious Case of Benjamin Button” in 2008, digital actors crossed the “Uncanny Valley” from looking strangely synthetic to believably real. This talk describes some of the technological advances that have enabled this achievement. For an in-depth example, the talk describes how high-resolution face scanning, advanced character rigging, and performance-driven facial animation were combined to create “Digital Emily”, a collaboration between the USC ICT Graphics Laboratory and Image Metrics. Actress Emily O’Brien was scanned in Light Stage 5 in 33 facial poses at the resolution of skin pores and fine wrinkles. These scans were assembled into a rigged face model driven by Image Metrics’ video-based animation software, and the resulting photoreal facial animation premiered at SIGGRAPH 2008. The talk also presents techniques which may allow digital characters to leap from the movie screen and into the space around us, including a 3D teleconferencing system that uses live facial scanning and an autostereoscopic display to transmit a person’s face in 3D and make eye contact with remote collaborators.

Paul Debevec: “Avatar and Beyond: Lighting Hollywood’s Real and Virtual Actors”

Jennifer Wohlner — Tue, 08 May 2012 18:33:27 +0000

Photoreal digital actors have become a practical reality in the last decade and are poised to revolutionize the entertainment industry. Paul Debevec from USC’s Institute for Creative Technologies will explain the technical progression and application of his lab’s LED-based “Light Stage” facial scanning systems, which have helped produce photoreal digital actors for movies such as Spider-Man 2, The Curious Case of Benjamin Button and Avatar.

The Huffington Post Showcases Gunslinger, ICT’s Virtual Human Shootout

Jennifer Wohlner — Fri, 04 May 2012 00:00:15 +0000

Cara Santa Maria from the Huffington Post’s Talk Nerdy to Me science series, visited ICT and experienced our Wild West mixed-reality experience Gunslinger. Santa Maria proved to be a quick study as she learned about the technology behind the demonstration, which features interaction with three different virtual characters, and also proved to be a sharp shooter, as she took on our virtual villain Rio Laine.

Naval Service Training Command Article about INOTS

Jennifer Wohlner — Fri, 27 Apr 2012 00:00:37 +0000

The Naval Service Training Command published an article about ICT’s Immersive Naval Officer Training System (INOTS). Read the full article here.

ICT Virtual Humans Promote Science Education at USA Science and Engineering Festival

Jennifer Wohlner — Fri, 27 Apr 2012 00:00:15 +0000

ICT’s virtual human museum guides, along with several of our real researchers, made the trip to Washington D.C. to teach kids about science as part of the National Science Foundation booth at the USA Science and Engineering Festival. ICT’s virtual humans Coach Mike and Ada and Grace, who were all created in collaboration with the Boston Museum of Science as part of an NSF-funded STEM education effort, were selected by the NSF to be showcased in their booth at the weekend event promoting science to school children.

Priti Aggarwal: “Ada and Grace: Engaging virtual human museum guides”

Jennifer Wohlner — Fri, 27 Apr 2012 00:00:09 +0000

Presenting the Twins and Coach Mike @ USA Science & Engineering Festival Expo with NSF