Institute for Creative Technologies

As an increasing part of the Army’s mission involves establishing rapport with diverse populations, training interpersonal skills becomes critically important. Here we describe a “Rapport Agent” that senses and responds to a speaker’s nonverbal behavior and provide empirical evidence that it increases speaker fluency and engagement. We argue such agent technology has potential, both as a training system to enhance communication skills, and to assess the key factors that influence rapport in face-to-face interactions. We conclude by discussing ways the nonverbal correlates of rapport vary between Arabic and English speakers and discuss the potential of such technology to advance research and training into rapport in cross-cultural settings.

In this paper, we introduce a methodology for analyzing judgment opinions. We define a judgment opinion as consisting of a valence, a holder, and a topic. We decompose the task of opinion analysis into four parts: 1) recognizing the opinion; 2) identifying the valence; 3) identifying the holder; and 4) identifying the topic. In this paper, we address the first three parts and evaluate our methodology using both intrinsic and extrinsic measures.

This paper describes the virtual humans developed as part of the Mission Rehearsal Exercise project, a virtual reality-based training system. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. We describe the key capabilities of the virtual humans, including task representation and reasoning, natural language dialogue, and emotion reasoning, and show how these capabilities are integrated to provide more human-level intelligence than would otherwise be possible.

The objective of this study was to test the efficacy and suitability of virtual reality (VR) as apain distraction for pediatric intravenous (IV) placement. Twenty children (12 boys, 8 girls) requiring IV placement for a magnetic resonance imaging/computed tomography (MRI/CT) scan were randomly assigned to two conditions: (1) VR distraction using Street Luge(5DT), presented via a head-mounted display, or (2) standard of care (topical anesthetic) with no distraction. Children, their parents, and nurses completed self-report questionnaires that assessed numerous health-related outcomes. Responses from the Faces Pain Scale--Revisedindicated a fourfold increase in affective pain within the control condition; by contrast, nosignificant differences were detected within the VR condition. Significant associations between multiple measures of anticipatory anxiety, affective pain, IV pain intensity, and measures of past procedural pain provided support for the complex interplay of a multimodalassessment of pain perception. There was also a sufficient amount of evidence supportingthe efficacy of Street Luge as a pediatric pain distraction tool during IV placement: an adequate level of presence, no simulator sickness, and significantly more child-, parent-, and nurse-reported satisfaction with pain management. VR pain distraction was positively endorsed by all reporters and is a promising tool for decreasing pain, and anxiety in childrenundergoing acute medical interventions. However, further research with larger sample sizesand other routine medical procedures is warranted.

This paper details the design and implementation of an embedded environmental framework that introduces cultural and social influences into a simulation agent’s decision-making process. We describe the current limitations associated with accurately representing culture in virtual environments and military simulations, and how recent research in other academic fields have enabled computational techniques to begin incorporating the effects of culture into AI and behavior subsystems. The technical approach is presented that describes the design and implementation of a hierarchical data model, as well as the software techniques for embedding culturally-specific information inside of a virtual environment. Finally, future work is discussed for developing a more comprehensive and standardized approach for embedding this culturally-specific information inside of the virtual domain.

Intelligent agents are typically situated in a social environment and must reason about social cause and effect. Such reasoning is qualitatively different from physical causal reasoning that underlies most intelligent systems. Modeling social causal reasoning can enrich the capabilities of multi-agent systems and intelligent user interfaces. In this paper, we empirically evaluate a computational model of social causality and responsibility against human social judgments. Results from our experimental studies show that in general, the model’s predictions of internal variables and inference process are consistent with human responses, though they also suggest some possible refinement to the computational model.

The authoring of fictional stories is considered a creative process.  The purpose of most story authoring is not to invent a new style or genre of story that will be accepted by the population but to invent a single narrative that is novel enough to be tellable.  Computational story generation systems are more limited than human authors in the space of narratives that can be considered because it is often the case that story generation systems are constrained to operate within a fixed representation of the story world.  These limitations can impact whether a story generation system is considered creative or not.  In this paper, we describe a story planning system, Fabulist.  Fabulist however is constrained by the world model input by the system user.  We present two algorithms that enable story planning systems such as Fabulist to break outside the bounds of the initial world model in order to search a larger space of narratives.

Video-capture virtual reality (VR) systems are gaining popularity as intervention tools. Todate, these platforms offer visual and audio feedback but do not provide haptic feedback. Wecontend that adding haptic feedback may enhance the quality of intervention for various theoretical and empirical reasons. This study aims to integrate haptic-tactile feedback into avideo capture system (GX VR), which is currently applied for rehabilitation. The proposedmulti-modal system can deliver audio-visual as well as vibrotactile feedback. The latter isprovided via small vibratory discs attached to the patient’s limbs. This paper describes thesystem, the guidelines of its design, and the ongoing usability study.

Post Traumatic Stress Disorder is reported to be caused by traumatic events that are outside the range of usual human experiences including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that 1 out of 6 Iraq War veterans are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) exposure treatment has been used in previous treatments of PTSD patients with reports of positive outcomes. The aim of the current paper is to specify the rationale, design and development of a Virtual Iraq PTSD VR application that has been created from the virtual assets that were initially developed for a combat tactical training simulation, which then served as the inspiration for the X-Box game entitled Full Spectrum Warrior.

Post Traumatic Stress Disorder (PTSD) is reported to be caused by traumatic events that are outside the range of usual human experiences including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that 1 out of 6 Iraq War veterans are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) exposure treatment has been used in previous treatments of PTSD patients with reports of positive outcomes. The aim of the current paper is to present the rationale, technical specifications, application features and user-centered design process for the development of a Virtual Iraq PTSD VR therapy application. The VR treatment environment is being created via the recycling of virtual graphic assets that were initially built for the U.S. Army-funded combat tactical simulation scenario and commercially successful X-Box game, Full Spectrum Warrior, in addition to other available and newly created assets. Thus far we have created a series of customizable virtual scenarios designed to represent relevant contexts for exposure therapy to be conducted in VR, including a city and desert road convoy environment. User-Centered tests with the application are currently underway at the Naval Medical Center--San Diego and within an Army Combat Stress Control Team in Iraq with clinical trials scheduled to commence in February 2006.

The University of Southern California and its collaborative research partner, Fakespace Labs, are participating in a number of research programs to invent and implement new forms of display technologies for immersive and semi-immersive applications. This paper briefly describes three of these technologies and highlights a few emerging results from those efforts. The first system is a rear projected 300 degree field of view cylindrical display. It is driven by 11 projectors with geometry correction and edge blending hardware. A full scale prototype will be completed in March 2006. The second system is a 14 screen projected panoramic room environment used as an advanced teaching and meeting space. It can be driven by a cluster of personal computers or low-cost DVD players, or driven by a single personal computer. The third is a prototype stereoscopic head mounted display that can be worn in a fashion similar to standard dust protection goggles. It provides a field of view in excess of 150 degrees.

This paper presents an activity-based framework for empirical discourse analysis of mitigation in public environments such as Swedish and Bulgarian courtroom examinations. Mitigation is defined as a pragmatic, cognitive and linguistic behavior the main purpose of which is reduction of vulnerability. The suggested framework consists of mitigation processes, which involve mitigating argumentation lines, defense moves, and communicative acts. The functions of mitigation are described in terms of the participants’ actions and goals separately from politeness strategies. The conclusions and observations address two things: issues related to the pragmatic theory of communication especially mitigation and issues related to the trial as a social activity. For instance, non-turn-taking confirmations by examiners are often followed by volunteered utterances, which in some cases may be examples of ‘rehearsed’ testimonies. At the same time the witnesses’ tendency to volunteer information even on the behalf of their own credibility indicates that they also favor pro-party testimonies. Despite the objective judicial role of the prosecutor or judge and/or despite the examiners accommodating style the verbal behavior of the witnesses exhibits constant anticipation of danger.

Contemporary technology urges us to believe that it improves communication. One result of such a belief is increased negativity towards “failure” in communication, which is cured only by more communication or by force. A second consequence is a focus on the view that language is a vehicle for transmission of thoughts and a dislocation of the view that language and communication offer opportunity for ethical manifestation. This article proposes that ethics emerges through and in language: beyond the contents delivered and the linguistic structure it enforces, language inspires the fundamental response-ability between self and Other. It is exactly in the breakdown of communication that we communicate through the ethics of a caress. The contemporary dichotomies describing communication and meaning are due to a conceptual fusion/confusion, which in its turn is due to the post-world-wars’ breathless desire for speedy progress and recovery.

Treatment and rehabilitation of the cognitive, psychological, and motor sequelae of central nervous system dysfunction often relies on assessment instruments to inform diagnosis and to track changes in clinical status. Typically, these assessments employ paper-and-pencil psychometrics, hands-on analog/computer tests, and rating of behavior within the context of real-world functional environments. Virtual reality offers the option to produce and distribute identical “standard” simulation environments in which performance can be measured and rehabilitated. Within such digital scenarios, normative data can be accumulated for performance comparisons needed for assessment/diagnosis and for treatment/rehabilitation purposes. In this manner, reusable archetypic virtual environments constructed for one purpose can also be applied for applications addressing other clinical targets. This article will provide a review of such a retooling approach using a virtual classroom simulation that was originally developed as a controlled stimulus environment in which attention processes could be systematically assessed in children with attention-deficit/hyperactivity disorder. This system is now being applied to other clinical targets including the development of tests.

This article summarizes recent progress in developing a validated computational account of the cognitive antecedents and consequences of emotion. We describe the potential of this work to impact a variety of AI problem domains.

Intelligent virtual agents are typically embedded in a social environment and must reason about social cause and effect. Social causal reasoning is qualitatively different from physical causal reasoning that underlies most current intelligent systems. Besides physical causality, the assessments of social cause emphasize epistemic variables including intentions, foreknowledge and perceived coercion. Modeling the process and inferences of social causality can enrich believability and cognitive capabilities of social intelligent agents. In this report, we present a general computational model of social causality and responsibility, and empirical results of a preliminary evaluation of the model in comparison with several other approaches.

Intelligent virtual agents are typically embedded in a social environment and must reason about social cause and effect.  Social causal reasoning is qualitatively different from physical causal reasoning that underlies most current intelligent systems. Besides physical causality, the assessments of social cause emphasize epistemic variables including intentions, foreknowledge and perceived coercion. Modeling the process and inferences of social causality can enrich the believability and the cognitive capabilities of social intelligent agents. In this paper, we present a general computational model of social causality and responsibility, and empirically evaluate and compare the model with several other approaches.