Publications
Search
41.
Talbot, Thomas B.; Rizzo, Albert Skip
Virtual Standardized Patients for Interactive Conversational Training: A Grand Experiment and New Approach Book Section
In: Exploring the Cognitive, Social, Cultural, and Psychological Aspects of Gaming and Simulations:, IGI Global, Hershey, PA, 2019, ISBN: 978-1-5225-7461-3 978-1-5225-7462-0.
@incollection{talbot_virtual_2019,
title = {Virtual Standardized Patients for Interactive Conversational Training: A Grand Experiment and New Approach},
author = {Thomas B. Talbot and Albert Skip Rizzo},
url = {http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-5225-7461-3},
doi = {10.4018/978-1-5225-7461-3},
isbn = {978-1-5225-7461-3 978-1-5225-7462-0},
year = {2019},
date = {2019-01-01},
booktitle = {Exploring the Cognitive, Social, Cultural, and Psychological Aspects of Gaming and Simulations:},
publisher = {IGI Global},
address = {Hershey, PA},
series = {Advances in Game-Based Learning},
abstract = {The USC Standard Patient is a virtual human-based conversational agent serving in the role of a simulated medical patient, also known as a virtual standardized patient (VSP). This research identified deficiencies of extant VSP systems, defined a robust set of requirements, and successfully achieved nearly all of them. Markedly impressive advancements were made in virtual human technology, techniques to apply natural language processing, automated assessment artificial intelligence, and pedagogical design. The effort succeeded with performance parameters of high conversational performance, accurate assessment, and strongly demonstrated user training effect. Although working well within its confined are of expertise, the ability for computers to create authentic mixed initiative conversations remains elusive. This effort leaves behind many lessons for interactive serious games, clinical virtual humans, and conversational virtual human training applications.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
The USC Standard Patient is a virtual human-based conversational agent serving in the role of a simulated medical patient, also known as a virtual standardized patient (VSP). This research identified deficiencies of extant VSP systems, defined a robust set of requirements, and successfully achieved nearly all of them. Markedly impressive advancements were made in virtual human technology, techniques to apply natural language processing, automated assessment artificial intelligence, and pedagogical design. The effort succeeded with performance parameters of high conversational performance, accurate assessment, and strongly demonstrated user training effect. Although working well within its confined are of expertise, the ability for computers to create authentic mixed initiative conversations remains elusive. This effort leaves behind many lessons for interactive serious games, clinical virtual humans, and conversational virtual human training applications.
42.
Gordon, Andrew S.; Spierling, Ulrike
Playing Story Creation Games with Logical Abduction Book Section
In: Interactive Storytelling, vol. 11318, pp. 478–482, Springer International Publishing, Cham, Switzerland, 2018, ISBN: 978-3-030-04027-7 978-3-030-04028-4.
@incollection{gordon_playing_2018,
title = {Playing Story Creation Games with Logical Abduction},
author = {Andrew S. Gordon and Ulrike Spierling},
url = {http://link.springer.com/10.1007/978-3-030-04028-4_55},
doi = {10.1007/978-3-030-04028-4_55},
isbn = {978-3-030-04027-7 978-3-030-04028-4},
year = {2018},
date = {2018-11-01},
booktitle = {Interactive Storytelling},
volume = {11318},
pages = {478–482},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Story Creation Games, such as Rory’s Story Cubes and the Tell Tale card game, require players to invent creative and coherent narratives from a set of unconnected elements assembled by random chance, e.g., the throw of a die or the draw of a card. We model this human ability as a process of logical abduction, where the reasoning task is to identify a set of assumptions about a fictional world that logically entail the elements depicted on the dice or on the cards. We demonstrate the feasibility of this approach by hand-authoring a knowledge base of axioms that is sufficient to generate eight creative narratives each related to three Tell Tale cards, depicting a baseball player, a heart, and a train.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Story Creation Games, such as Rory’s Story Cubes and the Tell Tale card game, require players to invent creative and coherent narratives from a set of unconnected elements assembled by random chance, e.g., the throw of a die or the draw of a card. We model this human ability as a process of logical abduction, where the reasoning task is to identify a set of assumptions about a fictional world that logically entail the elements depicted on the dice or on the cards. We demonstrate the feasibility of this approach by hand-authoring a knowledge base of axioms that is sufficient to generate eight creative narratives each related to three Tell Tale cards, depicting a baseball player, a heart, and a train.
43.
Goldberg, Benjamin; Nye, Benjamin; Lane, H Chad; Guadagnoli, Mark
Team Assessment and Pedagogy as Informed by Sports Coaching and Assessment Book Section
In: Design Recommendations for Intelligent Tutoring Systems: Volume 6-Team Modeling, pp. 105–119, US Army Research Laboratory (ARL), Orlando, Florida, 2018, ISBN: 978-0-9977257-4-2.
@incollection{goldberg_team_2018,
title = {Team Assessment and Pedagogy as Informed by Sports Coaching and Assessment},
author = {Benjamin Goldberg and Benjamin Nye and H Chad Lane and Mark Guadagnoli},
url = {https://gifttutoring.org/attachments/download/3029/Design%20Recommendations%20for%20ITS_Volume%206%20-%20Team%20Tutoring_final.pdf},
isbn = {978-0-9977257-4-2},
year = {2018},
date = {2018-08-01},
booktitle = {Design Recommendations for Intelligent Tutoring Systems: Volume 6-Team Modeling},
pages = {105–119},
publisher = {US Army Research Laboratory (ARL)},
address = {Orlando, Florida},
abstract = {In this chapter, we consider pedagogical insights offered by three different sources of information from sports coaching and assessment: published reports of sports training, first-hand accounts of team training, and a review of assessment approaches for measuring team performance. These issues are considered in the context of an integrated taxonomy of feedback that considers when feedback was given, who it was given to (e.g., individual vs. team), the type of feedback (e.g., positive vs. negative), and the specificity of feedback (e.g., detailed issues vs. brief note). The goal of this work is to consider how these patterns might generalize to a wider range of learning tasks, to improve both learning and assessment of team performance.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
In this chapter, we consider pedagogical insights offered by three different sources of information from sports coaching and assessment: published reports of sports training, first-hand accounts of team training, and a review of assessment approaches for measuring team performance. These issues are considered in the context of an integrated taxonomy of feedback that considers when feedback was given, who it was given to (e.g., individual vs. team), the type of feedback (e.g., positive vs. negative), and the specificity of feedback (e.g., detailed issues vs. brief note). The goal of this work is to consider how these patterns might generalize to a wider range of learning tasks, to improve both learning and assessment of team performance.
44.
Pynadath, David V.; Barnes, Michael J.; Wang, Ning; Chen, Jessie Y. C.
Transparency Communication for Machine Learning in Human-Automation Interaction Book Section
In: Human and Machine Learning, pp. 75–90, Springer International Publishing, Cham, Switzerland, 2018, ISBN: 978-3-319-90402-3 978-3-319-90403-0.
@incollection{pynadath_transparency_2018,
title = {Transparency Communication for Machine Learning in Human-Automation Interaction},
author = {David V. Pynadath and Michael J. Barnes and Ning Wang and Jessie Y. C. Chen},
url = {http://link.springer.com/10.1007/978-3-319-90403-0_5},
doi = {10.1007/978-3-319-90403-0_5},
isbn = {978-3-319-90402-3 978-3-319-90403-0},
year = {2018},
date = {2018-06-01},
booktitle = {Human and Machine Learning},
pages = {75–90},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Technological advances offer the promise of autonomous systems to form human-machine teams that are more capable than their individual members. Understanding the inner workings of the autonomous systems, especially as machine-learning (ML) methods are being widely applied to the design of such systems, has become increasingly challenging for the humans working with them. The “black-box” nature of quantitative ML approaches poses an impediment to people’s situation awareness (SA) of these ML-based systems, often resulting in either disuse or over-reliance of autonomous systems employing such algorithms. Research in human-automation interaction has shown that transparency communication can improve teammates’ SA, foster the trust relationship, and boost the human-automation team’s performance. In this chapter, we will examine the implications of an agent transparency model for human interactions with ML-based agents using automated explanations. We will discuss the application of a particular ML method, reinforcement learning (RL), in Partially Observable Markov Decision Process (POMDP)-based agents, and the design of explanation algorithms for RL in POMDPs.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Technological advances offer the promise of autonomous systems to form human-machine teams that are more capable than their individual members. Understanding the inner workings of the autonomous systems, especially as machine-learning (ML) methods are being widely applied to the design of such systems, has become increasingly challenging for the humans working with them. The “black-box” nature of quantitative ML approaches poses an impediment to people’s situation awareness (SA) of these ML-based systems, often resulting in either disuse or over-reliance of autonomous systems employing such algorithms. Research in human-automation interaction has shown that transparency communication can improve teammates’ SA, foster the trust relationship, and boost the human-automation team’s performance. In this chapter, we will examine the implications of an agent transparency model for human interactions with ML-based agents using automated explanations. We will discuss the application of a particular ML method, reinforcement learning (RL), in Partially Observable Markov Decision Process (POMDP)-based agents, and the design of explanation algorithms for RL in POMDPs.
45.
Monahan, Shannon; Johnson, Emmanuel; Lucas, Gale; Finch, James; Gratch, Jonathan
Autonomous Agent that Provides Automated Feedback Improves Negotiation Skills Book Section
In: Artificial Intelligence in Education, vol. 10948, pp. 225–229, Springer International Publishing, Cham, Switzerland, 2018, ISBN: 978-3-319-93845-5 978-3-319-93846-2.
@incollection{monahan_autonomous_2018,
title = {Autonomous Agent that Provides Automated Feedback Improves Negotiation Skills},
author = {Shannon Monahan and Emmanuel Johnson and Gale Lucas and James Finch and Jonathan Gratch},
url = {http://link.springer.com/10.1007/978-3-319-93846-2_41},
doi = {10.1007/978-3-319-93846-2_41},
isbn = {978-3-319-93845-5 978-3-319-93846-2},
year = {2018},
date = {2018-06-01},
booktitle = {Artificial Intelligence in Education},
volume = {10948},
pages = {225–229},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Research has found that individuals can improve their negotiation abilities by practicing with virtual agents [1, 2]. For these pedagogical agents to become more “intelligent,” the system should be able to give feedback on negotiation performance [3, 4]. In this study, we examined the impact of providing such individualized feedback. Participants first engaged in a negotiation with a virtual agent. After this negotiation, participants were either given automated individualized feedback or not. Feedback was based on negotiation principles [4], which were quantified using a validated approach [5]. Participants then completed a second, parallel negotiation. Our results show that, compared to the control condition, participants who received such feedback after the first negotiation showed a significantly greater improvement in the strength of their first offer, concession curve, and thus their final outcome in the negotiation.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Research has found that individuals can improve their negotiation abilities by practicing with virtual agents [1, 2]. For these pedagogical agents to become more “intelligent,” the system should be able to give feedback on negotiation performance [3, 4]. In this study, we examined the impact of providing such individualized feedback. Participants first engaged in a negotiation with a virtual agent. After this negotiation, participants were either given automated individualized feedback or not. Feedback was based on negotiation principles [4], which were quantified using a validated approach [5]. Participants then completed a second, parallel negotiation. Our results show that, compared to the control condition, participants who received such feedback after the first negotiation showed a significantly greater improvement in the strength of their first offer, concession curve, and thus their final outcome in the negotiation.
46.
Wang, Ning; Shapiro, Ari; Feng, Andrew; Zhuang, Cindy; Merchant, Chirag; Schwartz, David; Goldberg, Stephen L.
Learning by Explaining to a Digital Doppelganger Book Section
In: Intelligent Tutoring Systems, vol. 10858, pp. 256–264, Springer International Publishing, Cham, Switzerland, 2018, ISBN: 978-3-319-91463-3 978-3-319-91464-0.
@incollection{wang_learning_2018,
title = {Learning by Explaining to a Digital Doppelganger},
author = {Ning Wang and Ari Shapiro and Andrew Feng and Cindy Zhuang and Chirag Merchant and David Schwartz and Stephen L. Goldberg},
url = {http://link.springer.com/10.1007/978-3-319-91464-0_25},
doi = {10.1007/978-3-319-91464-0_25},
isbn = {978-3-319-91463-3 978-3-319-91464-0},
year = {2018},
date = {2018-05-01},
booktitle = {Intelligent Tutoring Systems},
volume = {10858},
pages = {256–264},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Digital doppelgangers are virtual humans that highly resemble the real self but behave independently. An emerging computer animation technology makes the creation of digital doppelgangers an accessible reality. This allows researchers in pedagogical agents to explore previously unexplorable research questions, such as how does increasing the similarity in appearance between the agent and the student impact learning. This paper discusses the design and evaluation of a digital doppelganger as a virtual listener in a learning-by-explaining paradigm. Results offer insight into the promise and limitation of this novel technology.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Digital doppelgangers are virtual humans that highly resemble the real self but behave independently. An emerging computer animation technology makes the creation of digital doppelgangers an accessible reality. This allows researchers in pedagogical agents to explore previously unexplorable research questions, such as how does increasing the similarity in appearance between the agent and the student impact learning. This paper discusses the design and evaluation of a digital doppelganger as a virtual listener in a learning-by-explaining paradigm. Results offer insight into the promise and limitation of this novel technology.
47.
Wang, Ning; Pynadath, David V.; Rovira, Ericka; Barnes, Michael J.; Hill, Susan G.
In: Persuasive Technology, vol. 10809, pp. 56–69, Springer International Publishing, Cham, Switzerland, 2018, ISBN: 978-3-319-78977-4 978-3-319-78978-1.
@incollection{wang_is_2018,
title = {Is It My Looks? Or Something I Said? The Impact of Explanations, Embodiment, and Expectations on Trust and Performance in Human-Robot Teams},
author = {Ning Wang and David V. Pynadath and Ericka Rovira and Michael J. Barnes and Susan G. Hill},
url = {http://link.springer.com/10.1007/978-3-319-78978-1_5},
doi = {10.1007/978-3-319-78978-1_5},
isbn = {978-3-319-78977-4 978-3-319-78978-1},
year = {2018},
date = {2018-04-01},
booktitle = {Persuasive Technology},
volume = {10809},
pages = {56–69},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Trust is critical to the success of human-robot interaction. Research has shown that people will more accurately trust a robot if they have an accurate understanding of its decision-making process. The Partially Observable Markov Decision Process (POMDP) is one such decision-making process, but its quantitative reasoning is typically opaque to people. This lack of transparency is exacerbated when a robot can learn, making its decision making better, but also less predictable. Recent research has shown promise in calibrating human-robot trust by automatically generating explanations of POMDP-based decisions. In this work, we explore factors that can potentially interact with such explanations in influencing human decision-making in human-robot teams. We focus on explanations with quantitative expressions of uncertainty and experiment with common design factors of a robot: its embodiment and its communication strategy in case of an error. Results help us identify valuable properties and dynamics of the human-robot trust relationship.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Trust is critical to the success of human-robot interaction. Research has shown that people will more accurately trust a robot if they have an accurate understanding of its decision-making process. The Partially Observable Markov Decision Process (POMDP) is one such decision-making process, but its quantitative reasoning is typically opaque to people. This lack of transparency is exacerbated when a robot can learn, making its decision making better, but also less predictable. Recent research has shown promise in calibrating human-robot trust by automatically generating explanations of POMDP-based decisions. In this work, we explore factors that can potentially interact with such explanations in influencing human decision-making in human-robot teams. We focus on explanations with quantitative expressions of uncertainty and experiment with common design factors of a robot: its embodiment and its communication strategy in case of an error. Results help us identify valuable properties and dynamics of the human-robot trust relationship.
48.
Rizzo, Albert; Roy, Michael J.; Hartholt, Arno; Costanzo, Michelle; Highland, Krista Beth; Jovanovic, Tanja; Norrholm, Seth D.; Reist, Chris; Rothbaum, Barbara; Difede, JoAnn
Virtual Reality Applications for the Assessment and Treatment of PTSD Book Section
In: Handbook of Military Psychology, pp. 453–471, Springer International Publishing, Cham, Switzerland, 2017, ISBN: 978-3-319-66190-2 978-3-319-66192-6.
@incollection{rizzo_virtual_2017,
title = {Virtual Reality Applications for the Assessment and Treatment of PTSD},
author = {Albert Rizzo and Michael J. Roy and Arno Hartholt and Michelle Costanzo and Krista Beth Highland and Tanja Jovanovic and Seth D. Norrholm and Chris Reist and Barbara Rothbaum and JoAnn Difede},
url = {http://link.springer.com/10.1007/978-3-319-66192-6_27},
doi = {10.1007/978-3-319-66192-6_27},
isbn = {978-3-319-66190-2 978-3-319-66192-6},
year = {2017},
date = {2017-12-01},
booktitle = {Handbook of Military Psychology},
pages = {453–471},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {War is one of the most challenging situations that a human being can encounter. The physical, emotional, cognitive, and psychological demands of a combat environment place tremendous stress on even the most well-prepared military people. It is no surprise that the stressful experiences, characteristics of operations in Iraq and Afghanistan, have produced significant numbers of service members (SMs) and veterans at risk for posttraumatic stress disorder (PTSD), as well as other psychosocial/behavioral health conditions. For example, as of June 2015, the Defense Medical Surveillance System reported 138,197 active duty SMs had been diagnosed with PTSD (Fischer, 2015). In a meta-analysis of studies published since 2001, 13.2% of infantry service members met the criteria for PTSD, with incidence rising dramatically to 25–30% in units with high levels of direct combat exposure (Kok, Herrell, Thomas, & Hoge, 2012). Moreover, as of early 2013, the prevalence of PTSD among discharged veterans receiving treatment at Veteran Affairs (VA) clinics was reported to be 29% (Fischer, 2013). These findings make a compelling case for a continued focus on developing and enhancing the availability of diverse evidence- based treatment options to address this military behavioral healthcare challenge. One emerging area of research and clinical focus is of the use of Virtual Reality (VR) simulation technology as a tool for delivering evidence-based approaches for the assessment and treatment of PTSD. Although in recent times, the popular media has lavishly reported on VR’s potential impact on all elements of our evolving digital culture, and has created the impression that VR is a novel technology, the reality is that VR is not a new concept, and many of its developmental roots are traceable to the 1980s and 1990s (Schnipper et al., 2015). Moreover, a large scientific literature has emerged over the last 20 years demonstrating the unique and added value that is accrued with the use of VR to address a wide range of clinical health conditions (Rizzo 1994; Rizzo et al., 1997; 2002; 2010; 2014; Rizzo, Cukor et al., 2015). Within that context, the present chapter will summarize the ways that researchers and clinicians have employed VR to create relevant simulations that can be applied to the assessment and treatment of PTSD.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
War is one of the most challenging situations that a human being can encounter. The physical, emotional, cognitive, and psychological demands of a combat environment place tremendous stress on even the most well-prepared military people. It is no surprise that the stressful experiences, characteristics of operations in Iraq and Afghanistan, have produced significant numbers of service members (SMs) and veterans at risk for posttraumatic stress disorder (PTSD), as well as other psychosocial/behavioral health conditions. For example, as of June 2015, the Defense Medical Surveillance System reported 138,197 active duty SMs had been diagnosed with PTSD (Fischer, 2015). In a meta-analysis of studies published since 2001, 13.2% of infantry service members met the criteria for PTSD, with incidence rising dramatically to 25–30% in units with high levels of direct combat exposure (Kok, Herrell, Thomas, & Hoge, 2012). Moreover, as of early 2013, the prevalence of PTSD among discharged veterans receiving treatment at Veteran Affairs (VA) clinics was reported to be 29% (Fischer, 2013). These findings make a compelling case for a continued focus on developing and enhancing the availability of diverse evidence- based treatment options to address this military behavioral healthcare challenge. One emerging area of research and clinical focus is of the use of Virtual Reality (VR) simulation technology as a tool for delivering evidence-based approaches for the assessment and treatment of PTSD. Although in recent times, the popular media has lavishly reported on VR’s potential impact on all elements of our evolving digital culture, and has created the impression that VR is a novel technology, the reality is that VR is not a new concept, and many of its developmental roots are traceable to the 1980s and 1990s (Schnipper et al., 2015). Moreover, a large scientific literature has emerged over the last 20 years demonstrating the unique and added value that is accrued with the use of VR to address a wide range of clinical health conditions (Rizzo 1994; Rizzo et al., 1997; 2002; 2010; 2014; Rizzo, Cukor et al., 2015). Within that context, the present chapter will summarize the ways that researchers and clinicians have employed VR to create relevant simulations that can be applied to the assessment and treatment of PTSD.
49.
Bunt, Harry; Petukhova, Volha; Traum, David; Alexandersson, Jan
Dialogue Act Annotation with the ISO 24617-2 Standard Book Section
In: Multimodal Interaction with W3C Standards, pp. 109–135, Springer International Publishing, Cham, Switzerland, 2017, ISBN: 978-3-319-42814-7 978-3-319-42816-1.
@incollection{bunt_dialogue_2017,
title = {Dialogue Act Annotation with the ISO 24617-2 Standard},
author = {Harry Bunt and Volha Petukhova and David Traum and Jan Alexandersson},
url = {http://link.springer.com/10.1007/978-3-319-42816-1_6},
isbn = {978-3-319-42814-7 978-3-319-42816-1},
year = {2017},
date = {2017-11-01},
booktitle = {Multimodal Interaction with W3C Standards},
pages = {109–135},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {This chapter describes recent and ongoing annotation efforts using the ISO 24617-2 standard for dialogue act annotation. Experimental studies are reported on the annotation by human annotators and by annotation machines of some of the specific features of the ISO annotation scheme, such as its multidimensional annotation of communicative functions, the recognition of each of its nine dimensions, and the recognition of dialogue act qualifiers for certainty, conditionality, and sentiment. The construction of corpora of dialogues, annotated according to ISO 24617-2, is discussed, including the recent DBOX and DialogBank corpora.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This chapter describes recent and ongoing annotation efforts using the ISO 24617-2 standard for dialogue act annotation. Experimental studies are reported on the annotation by human annotators and by annotation machines of some of the specific features of the ISO annotation scheme, such as its multidimensional annotation of communicative functions, the recognition of each of its nine dimensions, and the recognition of dialogue act qualifiers for certainty, conditionality, and sentiment. The construction of corpora of dialogues, annotated according to ISO 24617-2, is discussed, including the recent DBOX and DialogBank corpora.
50.
Wang, Ning; Shapiro, Ari; Schwartz, David; Lewine, Gabrielle; Feng, Andrew Wei-Wen
Virtual Role-Play with Rapid Avatars Book Section
In: Intelligent Virtual Agents, vol. 10498, pp. 463–466, Springer International Publishing, Cham, Switzerland, 2017, ISBN: 978-3-319-67400-1 978-3-319-67401-8.
@incollection{wang_virtual_2017,
title = {Virtual Role-Play with Rapid Avatars},
author = {Ning Wang and Ari Shapiro and David Schwartz and Gabrielle Lewine and Andrew Wei-Wen Feng},
url = {http://link.springer.com/10.1007/978-3-319-67401-8_59},
isbn = {978-3-319-67400-1 978-3-319-67401-8},
year = {2017},
date = {2017-08-01},
booktitle = {Intelligent Virtual Agents},
volume = {10498},
pages = {463–466},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Digital doppelgangers possess great potential to serve as powerful models for behavioral change. An emerging technology, the Rapid Avatar Capture and Simulation (RACAS), enables low-cost and high-speed scanning of a human user and creation of a digital doppelganger that is a fully animatable virtual 3D model of the user. We designed a virtual role-playing game, DELTA, with digital doppelgangers to influence a human user’s attitude to-wards sexism on college campuses. In this demonstration, we will showcase the RACAS system and the DELTA game.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Digital doppelgangers possess great potential to serve as powerful models for behavioral change. An emerging technology, the Rapid Avatar Capture and Simulation (RACAS), enables low-cost and high-speed scanning of a human user and creation of a digital doppelganger that is a fully animatable virtual 3D model of the user. We designed a virtual role-playing game, DELTA, with digital doppelgangers to influence a human user’s attitude to-wards sexism on college campuses. In this demonstration, we will showcase the RACAS system and the DELTA game.
51.
Nye, Benjamin D.; Mitros, Piotr; Schunn, Christian; Foltz, Peter W.; Gasevic, Dragan; Katz, Irvin R.
Why Assess? The Role of Assessment in Learning Science and Society Book Section
In: Design Recommendations for Intelligent Tutoring Systems: Volume 5- Assessment, vol. 5, pp. 189–202, US Army Research Laboratory, Orlando, FL, 2017, ISBN: 978-0-9977257-2-8.
@incollection{benjamin_d_nye_why_2017,
title = {Why Assess? The Role of Assessment in Learning Science and Society},
author = {Benjamin D. Nye and Piotr Mitros and Christian Schunn and Peter W. Foltz and Dragan Gasevic and Irvin R. Katz},
url = {https://books.google.com/books?id=5tsyDwAAQBAJ&pg=PA189&source=gbs_toc_r&cad=4#v=onepage&q&f=false},
isbn = {978-0-9977257-2-8},
year = {2017},
date = {2017-08-01},
booktitle = {Design Recommendations for Intelligent Tutoring Systems: Volume 5- Assessment},
volume = {5},
pages = {189–202},
publisher = {US Army Research Laboratory},
address = {Orlando, FL},
abstract = {Even though assessment often is imperfect, it provides valuable input to the process of teaching, learning, and educational resource design. However, narrow assessment, especially used in high-stakes settings, can lead to worse educational outcomes (e.g., performance in later courses, workplace, or social settings; Hout & Elliott, 2011). Teachers may have a strong incentive to teach to the test, leading to a strong focus on memorization and rote procedural knowledge, while compromising key skills such as empathy, groupwork, mathematical maturity, and analytical reasoning. These are thorny problems – education shapes the skills1 that shape society, so these questions have broad implications. With that said, by constraining the discussion to the kinds of constructs considered when building learning experiences, the goals of assessment become more tractable.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Even though assessment often is imperfect, it provides valuable input to the process of teaching, learning, and educational resource design. However, narrow assessment, especially used in high-stakes settings, can lead to worse educational outcomes (e.g., performance in later courses, workplace, or social settings; Hout & Elliott, 2011). Teachers may have a strong incentive to teach to the test, leading to a strong focus on memorization and rote procedural knowledge, while compromising key skills such as empathy, groupwork, mathematical maturity, and analytical reasoning. These are thorny problems – education shapes the skills1 that shape society, so these questions have broad implications. With that said, by constraining the discussion to the kinds of constructs considered when building learning experiences, the goals of assessment become more tractable.
52.
Artstein, Ron
Inter-annotator Agreement Book Section
In: Handbook of Linguistic Annotation, pp. 297–313, Springer Netherlands, Dordrecht, Netherlands, 2017, ISBN: 978-94-024-0879-9 978-94-024-0881-2.
@incollection{artstein_inter-annotator_2017,
title = {Inter-annotator Agreement},
author = {Ron Artstein},
url = {http://link.springer.com/10.1007/978-94-024-0881-2_11},
doi = {10.1007/978-94-024-0881-2_11},
isbn = {978-94-024-0879-9 978-94-024-0881-2},
year = {2017},
date = {2017-06-01},
booktitle = {Handbook of Linguistic Annotation},
pages = {297–313},
publisher = {Springer Netherlands},
address = {Dordrecht, Netherlands},
abstract = {This chapter touches upon several issues in the calculation and assessment of interannotator agreement. It gives an introduction to the theory behind agreement coefficients and examples of their application to linguistic annotation tasks. Specific examples explore variation in annotator performance due to heterogeneous data, complex labels, item difficulty, and annotator differences, showing how global agreement coefficients may mask these sources of variation, and how detailed agreement studies can give insight into both the annotation process and the nature of the underlying data. The chapter also reviews recent work on using machine learning to exploit the variation among annotators and learn detailed models from which accurate labels can be inferred. I therefore advocate an approach where agreement studies are not used merely as a means to accept or reject a particular annotation scheme, but as a tool for exploring patterns in the data that are being annotated.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This chapter touches upon several issues in the calculation and assessment of interannotator agreement. It gives an introduction to the theory behind agreement coefficients and examples of their application to linguistic annotation tasks. Specific examples explore variation in annotator performance due to heterogeneous data, complex labels, item difficulty, and annotator differences, showing how global agreement coefficients may mask these sources of variation, and how detailed agreement studies can give insight into both the annotation process and the nature of the underlying data. The chapter also reviews recent work on using machine learning to exploit the variation among annotators and learn detailed models from which accurate labels can be inferred. I therefore advocate an approach where agreement studies are not used merely as a means to accept or reject a particular annotation scheme, but as a tool for exploring patterns in the data that are being annotated.
53.
Talbot, Thomas B
Making Lifelike Medical Games in the Age of Virtual Reality An Update on “Playing Games with Biology” from 2013 Book Section
In: Transforming Gaming and Computer Simulation Technologies across Industries, pp. 103–119, IGI Global, Hershey, PA, 2017, ISBN: 978-1-5225-1817-4 978-1-5225-1818-1.
@incollection{talbot_making_2017,
title = {Making Lifelike Medical Games in the Age of Virtual Reality An Update on “Playing Games with Biology” from 2013},
author = {Thomas B Talbot},
url = {http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-5225-1817-4},
doi = {10.4018/978-1-5225-1817-4},
isbn = {978-1-5225-1817-4 978-1-5225-1818-1},
year = {2017},
date = {2017-01-01},
booktitle = {Transforming Gaming and Computer Simulation Technologies across Industries},
pages = {103–119},
publisher = {IGI Global},
address = {Hershey, PA},
abstract = {Medical simulations differ from other training modalities in that life processes must be simulated as part of the experience. Biological fidelity is the degree to which character anatomical appearance and physiology behavior are represented within a game or simulation. Methods to achieve physiological fidelity include physiology engines, complex state machines, simple state machines and kinetic models. Games health scores that can be used in medical sims. Selection of technique depends upon the goals of the simulation, expected user inputs, development budget and level of fidelity required. Trends include greater availability of physiology engines rapid advances in virtual reality (VR). In VR, the expectation for a naturalistic interface is much greater, resulting in technical challenges regarding natural language and gesture-based interaction. Regardless of the technical approach, the user’s perception of biological fidelity, responsiveness to user inputs and the ability to correct mistakes is often more important than the underlying biological fidelity of the model.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Medical simulations differ from other training modalities in that life processes must be simulated as part of the experience. Biological fidelity is the degree to which character anatomical appearance and physiology behavior are represented within a game or simulation. Methods to achieve physiological fidelity include physiology engines, complex state machines, simple state machines and kinetic models. Games health scores that can be used in medical sims. Selection of technique depends upon the goals of the simulation, expected user inputs, development budget and level of fidelity required. Trends include greater availability of physiology engines rapid advances in virtual reality (VR). In VR, the expectation for a naturalistic interface is much greater, resulting in technical challenges regarding natural language and gesture-based interaction. Regardless of the technical approach, the user’s perception of biological fidelity, responsiveness to user inputs and the ability to correct mistakes is often more important than the underlying biological fidelity of the model.
54.
Bernardet, Ulysses; Chollet, Mathieu; DiPaola, Steve; Scherer, Stefan
An Architecture for Biologically Grounded Real-Time Reflexive Behavior Book Section
In: Intelligent Virtual Agents, vol. 10011, pp. 295–305, Springer International Publishing, Cham, Switzerland, 2016, ISBN: 978-3-319-47664-3 978-3-319-47665-0.
@incollection{bernardet_architecture_2016,
title = {An Architecture for Biologically Grounded Real-Time Reflexive Behavior},
author = {Ulysses Bernardet and Mathieu Chollet and Steve DiPaola and Stefan Scherer},
url = {http://download.springer.com/static/pdf/224/chp%253A10.1007%252F978-3-319-47665-0_26.pdf?originUrl=http%3A%2F%2Flink.springer.com%2Fchapter%2F10.1007%2F978-3-319-47665-0_26&token2=exp=1485296780 acl=%2Fstatic%2Fpdf%2F224%2Fchp%25253A10.1007%25252F978-3-319-47665-0_26.pdf%3ForiginUrl%3Dhttp%253A%252F%252Flink.springer.com%252Fchapter%252F10.1007%252F978-3-319-47665-0_26* hmac=1bf37d11eda93937fedd36843994ffdaf645ebda569c86edbcf61ca905942f89},
isbn = {978-3-319-47664-3 978-3-319-47665-0},
year = {2016},
date = {2016-10-01},
booktitle = {Intelligent Virtual Agents},
volume = {10011},
pages = {295–305},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {In this paper, we present a reflexive behavior architecture, that is geared towards the application in the control of the non-verbal behavior of the virtual humans in a public speaking training system. The model is organized along the distinction between behavior triggers that are internal (endogenous) to the agent, and those that origin in the environment (exogenous). The endogenous subsystem controls gaze behavior, triggers self-adaptors, and shifts between different postures, while the exogenous system controls the reaction towards auditory stimuli with different temporal and valence characteristics. We evaluate the different components empirically by letting participants compare the output of the proposed system to valid alternative variations.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
In this paper, we present a reflexive behavior architecture, that is geared towards the application in the control of the non-verbal behavior of the virtual humans in a public speaking training system. The model is organized along the distinction between behavior triggers that are internal (endogenous) to the agent, and those that origin in the environment (exogenous). The endogenous subsystem controls gaze behavior, triggers self-adaptors, and shifts between different postures, while the exogenous system controls the reaction towards auditory stimuli with different temporal and valence characteristics. We evaluate the different components empirically by letting participants compare the output of the proposed system to valid alternative variations.
55.
Nye, Benjamin D.; Hu, Xiangen
Conceptualizing and Representing Domains to Guide Tutoring Book Section
In: Design Recommendations for Intelligent Tutoring Systems: Volume 4-Domain Modeling, vol. 4, pp. 15–18, US Army Research Laboratory, Orlando, FL, 2016.
@incollection{nye_conceptualizing_2016,
title = {Conceptualizing and Representing Domains to Guide Tutoring},
author = {Benjamin D. Nye and Xiangen Hu},
url = {http://books.google.com/books?hl=en&lr=&id=0suvDAAAQBAJ&oi=fnd&pg=PA15&dq=%22data.+This+chapter+presents+an+excellent+overview+of+current+research+on+Q-matrices%22+%22edge+work+on+ensemble+methods+that+achieve+state+of+the+art+performance+by+combining%22+&ots=6MJhm1XHVV&sig=i14eJyin69Cy-jms2lWIFF4K3CU},
year = {2016},
date = {2016-07-01},
booktitle = {Design Recommendations for Intelligent Tutoring Systems: Volume 4-Domain Modeling},
volume = {4},
pages = {15–18},
publisher = {US Army Research Laboratory},
address = {Orlando, FL},
abstract = {Any discussion about how intelligent tutoring system (ITS) domains must begin with considering how ITS conceptualize and represent domains. This process requires building formal, mathematically-specifiable operationalization of the often implicit knowledge about learning domains and their pedagogy. Across different domains and pedagogical approaches, a wide variety of methods have been taken: a scope that would be better-covered by an encyclopedia rather than a single book. Since this section could not possibly cover every possible approach to domain modeling, the chapters within this section were instead chosen to cover a representative range of fundamentally-different approaches to domain modeling.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Any discussion about how intelligent tutoring system (ITS) domains must begin with considering how ITS conceptualize and represent domains. This process requires building formal, mathematically-specifiable operationalization of the often implicit knowledge about learning domains and their pedagogy. Across different domains and pedagogical approaches, a wide variety of methods have been taken: a scope that would be better-covered by an encyclopedia rather than a single book. Since this section could not possibly cover every possible approach to domain modeling, the chapters within this section were instead chosen to cover a representative range of fundamentally-different approaches to domain modeling.
56.
Nye, Benjamin D.; Boyce, Michael W.; Sottilare, Robert
Defining the Ill-Defined: From Abstract Principles to Applied Pedagogy Book Section
In: Design Recommendations for Intelligent Tutoring Systems: Volume 4-Domain Modeling, vol. 4, pp. 19–37, US Army Research Laboratory, Orlando, FL, 2016, ISBN: 978-0-9893923-9-6.
@incollection{nye_defining_2016,
title = {Defining the Ill-Defined: From Abstract Principles to Applied Pedagogy},
author = {Benjamin D. Nye and Michael W. Boyce and Robert Sottilare},
url = {https://gifttutoring.org/attachments/download/1736/Design%20Recommendations%20for%20ITS_Volume%204%20-%20Domain%20Modeling%20Book_web%20version_final.pdf},
isbn = {978-0-9893923-9-6},
year = {2016},
date = {2016-07-01},
booktitle = {Design Recommendations for Intelligent Tutoring Systems: Volume 4-Domain Modeling},
volume = {4},
pages = {19–37},
publisher = {US Army Research Laboratory},
address = {Orlando, FL},
abstract = {Attempts to define ill-defined domains in intelligent tutoring system (ITS) research has been approached a number of times (Fournier-Viger, Nkambou, & Nguifo, 2010; Lynch, Ashley, Pinkwart, & Aleven, 2009; Mitrovic & Weerasinghe, 2009; Jacovina, Snow, Dai, & McNamara, 2015; Woods, Stensrud, Wray, Haley, & Jones, 2015). Related research has tried to determine levels of ill-definedness for a domain (Le, Loll, & Pinkwart, 2013). Despite such attempts, the field has not yet converged on common guidelines to distinguish between well-defined versus ill-defined domains. We argue that such guidelines struggle to converge because a domain is too large to meaningfully categorize: every domain contains a mixture of well-defined and ill-defined tasks. While the co-existence of well-defined and ill-defined tasks in a single domain is nearly universally-agreed upon by researchers; this key point is often quickly buried by an extensive discussion about what makes certain domain tasks ill-defined (e.g., disagreement about ideal solutions, multiple solution paths). In this chapter, we first take a step back to consider what is meant by a domain in the context of learning. Next, based on this definition for a domain, we map out the components that are in a learning domain, since each component may have ill-defined parts. This leads into a discussion about the strategies that have been used to make ill-defined domains tractable for certain types of pedagogy. Examples of ITS research that applies these strategies are noted. Finally, we conclude with practical how-to considerations and open research questions for approaching ill-defined domains. This chapter should be considered a companion piece to our chapter in the prior volume of this series (Nye, Goldberg, & Hu, 2015). This chapter focuses on how to understand and transform ill-defined parts of domains, while the prior chapter discusses commonly-used learning tasks and authoring approaches for both well-defined and ill-defined tasks. As such, this chapter is intended to help the learner understand if and how different parts of the domain are ill-defined (and what to do about them). The companion piece in the authoring tools volume discusses different categories of well and ill-defined tasks, from the standpoint of attempting to author and maintain an ITS.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Attempts to define ill-defined domains in intelligent tutoring system (ITS) research has been approached a number of times (Fournier-Viger, Nkambou, & Nguifo, 2010; Lynch, Ashley, Pinkwart, & Aleven, 2009; Mitrovic & Weerasinghe, 2009; Jacovina, Snow, Dai, & McNamara, 2015; Woods, Stensrud, Wray, Haley, & Jones, 2015). Related research has tried to determine levels of ill-definedness for a domain (Le, Loll, & Pinkwart, 2013). Despite such attempts, the field has not yet converged on common guidelines to distinguish between well-defined versus ill-defined domains. We argue that such guidelines struggle to converge because a domain is too large to meaningfully categorize: every domain contains a mixture of well-defined and ill-defined tasks. While the co-existence of well-defined and ill-defined tasks in a single domain is nearly universally-agreed upon by researchers; this key point is often quickly buried by an extensive discussion about what makes certain domain tasks ill-defined (e.g., disagreement about ideal solutions, multiple solution paths). In this chapter, we first take a step back to consider what is meant by a domain in the context of learning. Next, based on this definition for a domain, we map out the components that are in a learning domain, since each component may have ill-defined parts. This leads into a discussion about the strategies that have been used to make ill-defined domains tractable for certain types of pedagogy. Examples of ITS research that applies these strategies are noted. Finally, we conclude with practical how-to considerations and open research questions for approaching ill-defined domains. This chapter should be considered a companion piece to our chapter in the prior volume of this series (Nye, Goldberg, & Hu, 2015). This chapter focuses on how to understand and transform ill-defined parts of domains, while the prior chapter discusses commonly-used learning tasks and authoring approaches for both well-defined and ill-defined tasks. As such, this chapter is intended to help the learner understand if and how different parts of the domain are ill-defined (and what to do about them). The companion piece in the authoring tools volume discusses different categories of well and ill-defined tasks, from the standpoint of attempting to author and maintain an ITS.
57.
Ustun, Volkan; Rosenbloom, Paul
Towards Truly Autonomous Synthetic Characters with the Sigma Cognitive Architecture Book Section
In: Integrating Cognitive Architectures into Virtual Character Design, pp. 213 – 237, IGI Global, Hershey, PA, 2016, ISBN: 978-1-5225-0454-2.
@incollection{ustun_towards_2016,
title = {Towards Truly Autonomous Synthetic Characters with the Sigma Cognitive Architecture},
author = {Volkan Ustun and Paul Rosenbloom},
url = {http://services.igi-global.com/resolvedoi/resolve.aspx?doi=10.4018/978-1-5225-0454-2},
isbn = {978-1-5225-0454-2},
year = {2016},
date = {2016-06-01},
booktitle = {Integrating Cognitive Architectures into Virtual Character Design},
pages = {213 – 237},
publisher = {IGI Global},
address = {Hershey, PA},
abstract = {Realism is required not only for how synthetic characters look but also for how they behave. Many applications, such as simulations, virtual worlds, and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Sigma (Σ) is being built as a computational model of general intelligence with a long-term goal of understanding and replicating the architecture of the mind; i.e., the fixed structure underlying intelligent behavior. Sigma leverages probabilistic graphical models towards a uniform grand unification of not only traditional cognitive capabilities but also key non-cognitive aspects, creating unique opportunities for the construction of new kinds of non-modular behavioral models. These ambitions strive for the complete control of synthetic characters that behave as humanly as possible. In this paper, Sigma is introduced along with two disparate proof-of-concept virtual humans – one conversational and the other a pair of ambulatory agents – that demonstrate its diverse capabilities.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Realism is required not only for how synthetic characters look but also for how they behave. Many applications, such as simulations, virtual worlds, and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Sigma (Σ) is being built as a computational model of general intelligence with a long-term goal of understanding and replicating the architecture of the mind; i.e., the fixed structure underlying intelligent behavior. Sigma leverages probabilistic graphical models towards a uniform grand unification of not only traditional cognitive capabilities but also key non-cognitive aspects, creating unique opportunities for the construction of new kinds of non-modular behavioral models. These ambitions strive for the complete control of synthetic characters that behave as humanly as possible. In this paper, Sigma is introduced along with two disparate proof-of-concept virtual humans – one conversational and the other a pair of ambulatory agents – that demonstrate its diverse capabilities.
58.
Swartout, William R.
Virtual Humans as Centaurs: Melding Real and Virtual Book Section
In: Virtual, Augmented and Mixed Reality, vol. 9740, pp. 356–359, Springer International Publishing, Cham, Switzerland, 2016, ISBN: 978-3-319-39906-5 978-3-319-39907-2.
@incollection{swartout_virtual_2016,
title = {Virtual Humans as Centaurs: Melding Real and Virtual},
author = {William R. Swartout},
url = {http://link.springer.com/10.1007/978-3-319-39907-2_34},
isbn = {978-3-319-39906-5 978-3-319-39907-2},
year = {2016},
date = {2016-06-01},
booktitle = {Virtual, Augmented and Mixed Reality},
volume = {9740},
pages = {356–359},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {Centaurs are man-machine teams that can work together on problems and can out-perform, either people or computers working alone in domains as varied as chess-playing and protein folding. But the centaur of Greek mythology was not a team, but rather a hybrid of man and horse with some of the characteristics of each. In this paper, we outline our efforts to build virtual humans, which might be considered hybrid centaurs, combining features of both people and machines. We discuss experimental evidence that shows that these virtual human hybrids can outperform both people and inanimate processes in some tasks such as medical interviewing.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Centaurs are man-machine teams that can work together on problems and can out-perform, either people or computers working alone in domains as varied as chess-playing and protein folding. But the centaur of Greek mythology was not a team, but rather a hybrid of man and horse with some of the characteristics of each. In this paper, we outline our efforts to build virtual humans, which might be considered hybrid centaurs, combining features of both people and machines. We discuss experimental evidence that shows that these virtual human hybrids can outperform both people and inanimate processes in some tasks such as medical interviewing.
59.
Kang, Sin-Hwa; Phan, Thai; Bolas, Mark; Krum, David M.
User Perceptions of a Virtual Human Over Mobile Video Chat Interactions Book Section
In: Human-Computer Interaction. Novel User Experiences, vol. 9733, pp. 107–118, Springer International Publishing, Cham, Switzerland, 2016, ISBN: 978-3-319-39512-8 978-3-319-39513-5.
@incollection{kang_user_2016,
title = {User Perceptions of a Virtual Human Over Mobile Video Chat Interactions},
author = {Sin-Hwa Kang and Thai Phan and Mark Bolas and David M. Krum},
url = {http://download.springer.com/static/pdf/913/chp%253A10.1007%252F978-3-319-39513-5_10.pdf?originUrl=http%3A%2F%2Flink.springer.com%2Fchapter%2F10.1007%2F978-3-319-39513-5_10&token2=exp=1474906977 acl=%2Fstatic%2Fpdf%2F913%2Fchp%25253A10.1007%25252F978-3-319-39513-5_10.pdf%3ForiginUrl%3Dhttp%253A%252F%252Flink.springer.com%252Fchapter%252F10.1007%252F978-3-319-39513-5_10* hmac=14d38ee320936bf1edfc65a0d3fcc0855c42e0baba46e0f3a9a81293698b8b68},
isbn = {978-3-319-39512-8 978-3-319-39513-5},
year = {2016},
date = {2016-06-01},
booktitle = {Human-Computer Interaction. Novel User Experiences},
volume = {9733},
pages = {107–118},
publisher = {Springer International Publishing},
address = {Cham, Switzerland},
abstract = {We believe that virtual humans, presented over video chat services, such as Skype, and delivered using smartphones, can be an effective way to deliver innovative applications where social interactions are important, such as counseling and coaching. To explore this subject, we have built a hardware and software apparatus that allows virtual humans to initiate, receive, and interact over video calls using Skype or any similar service. With this platform, we conducted two experiments to investigate the applications and characteristics of virtual humans that interact over mobile video. In Experiment 1, we investigated user reactions to the physical realism of the background scene in which a virtual human was displayed. In Experiment 2, we examined how virtual characters can establish and maintain longer term relationships with users, using ideas from Social Exchange Theory to strengthen bonds between interactants. Experiment 2 involved repeated interactions with a virtual human over a period of time. Both studies used counseling-style interactions with users. The results demonstrated that males were more attracted socially to a virtual human that was presented over a realistic background than a featureless background while females were more socially attracted to a virtual human with a less realistic featureless background. The results further revealed that users felt the virtual human was a compassionate partner when they interacted with the virtual human over multiple calls, rather than just a single call.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
We believe that virtual humans, presented over video chat services, such as Skype, and delivered using smartphones, can be an effective way to deliver innovative applications where social interactions are important, such as counseling and coaching. To explore this subject, we have built a hardware and software apparatus that allows virtual humans to initiate, receive, and interact over video calls using Skype or any similar service. With this platform, we conducted two experiments to investigate the applications and characteristics of virtual humans that interact over mobile video. In Experiment 1, we investigated user reactions to the physical realism of the background scene in which a virtual human was displayed. In Experiment 2, we examined how virtual characters can establish and maintain longer term relationships with users, using ideas from Social Exchange Theory to strengthen bonds between interactants. Experiment 2 involved repeated interactions with a virtual human over a period of time. Both studies used counseling-style interactions with users. The results demonstrated that males were more attracted socially to a virtual human that was presented over a realistic background than a featureless background while females were more socially attracted to a virtual human with a less realistic featureless background. The results further revealed that users felt the virtual human was a compassionate partner when they interacted with the virtual human over multiple calls, rather than just a single call.
60.
Gandhe, Sudeep; Traum, David
A Semi-automated Evaluation Metric for Dialogue Model Coherence Book Section
In: Situated Dialog in Speech-Based Human-Computer Interaction, pp. 217–225, Springer International Publishing, Cham, 2016, ISBN: 978-3-319-21833-5 978-3-319-21834-2.
@incollection{gandhe_semi-automated_2016,
title = {A Semi-automated Evaluation Metric for Dialogue Model Coherence},
author = {Sudeep Gandhe and David Traum},
url = {http://link.springer.com/10.1007/978-3-319-21834-2_19},
isbn = {978-3-319-21833-5 978-3-319-21834-2},
year = {2016},
date = {2016-04-01},
booktitle = {Situated Dialog in Speech-Based Human-Computer Interaction},
pages = {217–225},
publisher = {Springer International Publishing},
address = {Cham},
abstract = {We propose a new metric, Voted Appropriateness, which can be used to automatically evaluate dialogue policy decisions, once some wizard data has been collected. We show that this metric outperforms a previously proposed metric Weak agreement.We also present a taxonomy for dialogue model evaluation schemas, and orient our new metric within this taxonomy.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
We propose a new metric, Voted Appropriateness, which can be used to automatically evaluate dialogue policy decisions, once some wizard data has been collected. We show that this metric outperforms a previously proposed metric Weak agreement.We also present a taxonomy for dialogue model evaluation schemas, and orient our new metric within this taxonomy.
Filter
2011
Debevec, Paul; Tchou, Chris; Gardner, Andrew; Hawkins, Tim; Poullis, Charis; Stumpfel, Jessi; Jones, Andrew; Yun, Nathaniel; Einarsson, Per; Lundgren, Therese; Fajardo, Marcos
Digitizing the Parthenon: Estimating Surface Reflectance under Measured Natural Illumination Book Section
In: Gallo, Giovanni (Ed.): Digital Imaging for Cultural Heritage Preservation: Analysis, Restoration, and Reconstruction of Ancient Artworks, pp. 159–182, CRC Press, 2011, ISBN: 978-1-4398-2173-2.
Abstract | Links | BibTeX | Tags: Graphics
@incollection{debevec_digitizing_2011,
title = {Digitizing the Parthenon: Estimating Surface Reflectance under Measured Natural Illumination},
author = {Paul Debevec and Chris Tchou and Andrew Gardner and Tim Hawkins and Charis Poullis and Jessi Stumpfel and Andrew Jones and Nathaniel Yun and Per Einarsson and Therese Lundgren and Marcos Fajardo},
editor = {Giovanni Gallo},
url = {http://ict.usc.edu/pubs/Digitizing%20the%20Parthenon-%20Estimating%20Surface%20Reflectance%20under%20Measured%20Natural%20Illumination.pdf},
isbn = {978-1-4398-2173-2},
year = {2011},
date = {2011-01-01},
booktitle = {Digital Imaging for Cultural Heritage Preservation: Analysis, Restoration, and Reconstruction of Ancient Artworks},
pages = {159–182},
publisher = {CRC Press},
abstract = {This edition presents the most prominent topics and applications of digital image processing, analysis, and computer graphics in the field of cultural heritage preservation. The text assumes prior knowledge of digital image processing and computer graphics fundamentals. Each chapter contains a table of contents, illustrations, and figures that elucidate the presented concepts in detail, as well as a chapter summary and a bibliography for further reading. Well-known experts cover a wide range of topics and related applications, including spectral imaging, automated restoration, computational reconstruction, digital reproduction, and 3D models.},
keywords = {Graphics},
pubstate = {published},
tppubtype = {incollection}
}
This edition presents the most prominent topics and applications of digital image processing, analysis, and computer graphics in the field of cultural heritage preservation. The text assumes prior knowledge of digital image processing and computer graphics fundamentals. Each chapter contains a table of contents, illustrations, and figures that elucidate the presented concepts in detail, as well as a chapter summary and a bibliography for further reading. Well-known experts cover a wide range of topics and related applications, including spectral imaging, automated restoration, computational reconstruction, digital reproduction, and 3D models.
Gordon, Andrew S.; Hobbs, Jerry R.; Cox, Michael T.
Anthropomorphic self-models for metareasoning agents Book Section
In: Metareasoning: Thinking about thinking, pp. 295–305, MIT Press, Cambridge, MA, 2011.
Abstract | Links | BibTeX | Tags: The Narrative Group
@incollection{gordon_anthropomorphic_2011,
title = {Anthropomorphic self-models for metareasoning agents},
author = {Andrew S. Gordon and Jerry R. Hobbs and Michael T. Cox},
url = {http://ict.usc.edu/pubs/Anthropomorphic%20Self-Models%20for%20Metareasoning%20Agents.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Metareasoning: Thinking about thinking},
pages = {295–305},
publisher = {MIT Press},
address = {Cambridge, MA},
abstract = {Representations of an AI agent's mental states and processes are necessary to enable metareasoning, i.e. thinking about thinking. However, the formulation of suitable representations remains an outstanding AI research challenge, with no clear consensus on how to proceed. This paper outlines an approach involving the formulation of anthropomorphic self-models, where the representations that are used for metareasoning are based on formalizations of commonsense psychology. We describe two research activities that support this approach, the formalization of broad-coverage commonsense psychology theories and use of representations in the monitoring and control of objectlevel reasoning. We focus specifically on metareasoning about memory, but argue that anthropomorphic self-models support the development of integrated, reusable, broadcoverage representations for use in metareasoning systems.},
keywords = {The Narrative Group},
pubstate = {published},
tppubtype = {incollection}
}
Representations of an AI agent's mental states and processes are necessary to enable metareasoning, i.e. thinking about thinking. However, the formulation of suitable representations remains an outstanding AI research challenge, with no clear consensus on how to proceed. This paper outlines an approach involving the formulation of anthropomorphic self-models, where the representations that are used for metareasoning are based on formalizations of commonsense psychology. We describe two research activities that support this approach, the formalization of broad-coverage commonsense psychology theories and use of representations in the monitoring and control of objectlevel reasoning. We focus specifically on metareasoning about memory, but argue that anthropomorphic self-models support the development of integrated, reusable, broadcoverage representations for use in metareasoning systems.
Parsons, Thomas D.
Virtual Simulations and the Second Life Metaverse: Paradigm Shift in Neuropsychological Assessment Book Section
In: Virtual Worlds, Second Life and Metaverse Platforms: New Communication and Identity Paradigms, pp. 234–250, IGI Global, 2011.
Abstract | Links | BibTeX | Tags: MedVR
@incollection{parsons_virtual_2011-1,
title = {Virtual Simulations and the Second Life Metaverse: Paradigm Shift in Neuropsychological Assessment},
author = {Thomas D. Parsons},
url = {http://ict.usc.edu/pubs/Virtual%20Simulations%20and%20the%20Second%20Life%20Metaverse.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Virtual Worlds, Second Life and Metaverse Platforms: New Communication and Identity Paradigms},
pages = {234–250},
publisher = {IGI Global},
abstract = {In neuropsychology's received paradigm, the "normal science" of assessment and treatment planning appears to be approaching a paradigm shift: first, there are the general developments in other neurosci- ences that inform the practice of neuropsychological assessment. Second, there is the shift in the purpose of neuropsychological assessment from differential diagnosis of brain pathology to predictions about activities of everyday functioning and treatment planning. Third, there is growing need that neuropsy- chologists update their outdated technology for ecologically valid assessments. The impending paradigm shift may be well served to include the utility of virtual worlds for ecologically valid neuropsychological assessments. Actualization of the potential of virtual worlds for assessment will require the following: comparisons with well-validated neuropsychological measures, data storage, improved documentation of specific computer hardware and software used in experimental methods, and enhanced methods and result reporting by the researchers publishing studies on virtual worlds.},
keywords = {MedVR},
pubstate = {published},
tppubtype = {incollection}
}
In neuropsychology's received paradigm, the "normal science" of assessment and treatment planning appears to be approaching a paradigm shift: first, there are the general developments in other neurosci- ences that inform the practice of neuropsychological assessment. Second, there is the shift in the purpose of neuropsychological assessment from differential diagnosis of brain pathology to predictions about activities of everyday functioning and treatment planning. Third, there is growing need that neuropsy- chologists update their outdated technology for ecologically valid assessments. The impending paradigm shift may be well served to include the utility of virtual worlds for ecologically valid neuropsychological assessments. Actualization of the potential of virtual worlds for assessment will require the following: comparisons with well-validated neuropsychological measures, data storage, improved documentation of specific computer hardware and software used in experimental methods, and enhanced methods and result reporting by the researchers publishing studies on virtual worlds.
Parsons, Thomas D.
In: Systems in Health Care Using Agents and Virtual Reality, Springer-Verlag, Germany, 2011.
Abstract | Links | BibTeX | Tags: MedVR
@incollection{parsons_neuropsychological_2011,
title = {Neuropsychological Assessment using Virtual Environments: Enhanced Assessment Technology for Improved Ecological Validity},
author = {Thomas D. Parsons},
url = {http://ict.usc.edu/pubs/Neuropsychological%20Assessment%20Using%20Virtual%20Environments-%20Enhanced%20Assessment%20Technology%20for%20Improved%20Ecological%20Validity.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Systems in Health Care Using Agents and Virtual Reality},
publisher = {Springer-Verlag},
address = {Germany},
abstract = {Although today's neuropsychological assessment procedures are widely used, neuropsychologists have been slow to embrace technological advancements. Two essential limitations have resulted from this refusal of technological adaptation: First, current neuropsychological assessment procedures represent a technology that has barely changed since the first scales were developed in the early 1900s. Second, while the historical purpose of clinical neuropsychology was differential diagnosis of brain pathology, technological advances in other clinical neurosciences have changed the neuropsychologist's role to that of making ecologically valid predictions about the impact of a given patient's neurocognitive abilities and disabilities on everyday functioning. After a brief discussion of current applications of computer-based neuropsychological assessment, there is a discussion of an increasingly important topic in recent decades–the design of ecologically valid neuropsychological instruments to address real world outcomes. Finally, there is an exploration of virtual reality environments for ecologically valid neuropsychological assessments that make use of current technological advances. It is concluded that a future possible virtual reality-based neuropsychological assessment battery will combine the control and rigor of technologically advanced computerized laboratory measures, the psychometric rigor (i.e., veridicality) of traditional paper-and-pencil assessments, and verisimilitude approximating real life situations.},
keywords = {MedVR},
pubstate = {published},
tppubtype = {incollection}
}
Although today's neuropsychological assessment procedures are widely used, neuropsychologists have been slow to embrace technological advancements. Two essential limitations have resulted from this refusal of technological adaptation: First, current neuropsychological assessment procedures represent a technology that has barely changed since the first scales were developed in the early 1900s. Second, while the historical purpose of clinical neuropsychology was differential diagnosis of brain pathology, technological advances in other clinical neurosciences have changed the neuropsychologist's role to that of making ecologically valid predictions about the impact of a given patient's neurocognitive abilities and disabilities on everyday functioning. After a brief discussion of current applications of computer-based neuropsychological assessment, there is a discussion of an increasingly important topic in recent decades–the design of ecologically valid neuropsychological instruments to address real world outcomes. Finally, there is an exploration of virtual reality environments for ecologically valid neuropsychological assessments that make use of current technological advances. It is concluded that a future possible virtual reality-based neuropsychological assessment battery will combine the control and rigor of technologically advanced computerized laboratory measures, the psychometric rigor (i.e., veridicality) of traditional paper-and-pencil assessments, and verisimilitude approximating real life situations.
Rizzo, Albert; Requejo, Phil; Winstein, Carolee J.; Lange, Belinda; Ragusa, Gisele; Merians, Alma; Patton, James; Banerjee, Pat; Aisen, Mindy
NIDRR Perspectives on VR Applications for Addressing the Needs of those Aging with and into Disability Book Section
In: Technology and Informatics, pp. 510–516, IOS Press, Amsterdam, The Netherlands, 2011.
Abstract | Links | BibTeX | Tags: MedVR
@incollection{rizzo_nidrr_2011,
title = {NIDRR Perspectives on VR Applications for Addressing the Needs of those Aging with and into Disability},
author = {Albert Rizzo and Phil Requejo and Carolee J. Winstein and Belinda Lange and Gisele Ragusa and Alma Merians and James Patton and Pat Banerjee and Mindy Aisen},
url = {http://ict.usc.edu/pubs/NIDRR%20Perspectives%20on%20VR%20Applications.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Technology and Informatics},
pages = {510–516},
publisher = {IOS Press},
address = {Amsterdam, The Netherlands},
abstract = {As persons with disabilities age, progressive declines in health and medical status can challenge the adaptive resources required to maintain functional independence and quality of life [1]. These challenges are further compounded by economic factors, medication side effects, loss of a spouse or caregiver, and psychosocial disorders [1-2]. The gradual loss of functional independence and increased reliance on others for transportation, access to general medical and rehabilitation care can be jeopardized [2]. The combination of these factors when seen in the context of the average increase in lifespan in industrialized societies has lead to a growing crisis that is truly global in proportion. While research indicates that functional motor capacity can be improved, maintained, or recovered via consistent participation in a motor exercise and rehabilitation regimen [3], independent adherence to such preventative and/or rehabilitative programming outside the clinic setting is notoriously low [1]. This state of affairs has produced a compelling and ethical motivation to address the needs of individuals who are aging with disabilities by promoting home-based access to low-cost, interactive virtual reality (VR) systems designed to engage and motivate individuals to participate with "game"-driven physical activities and rehabilitation programming. The creation of such systems could serve to enhance, maintain and rehabilitate the sensorimotor processes that are needed to maximize independence and quality of life. This is the theme of the workshop to be presented at the MMVR conference.},
keywords = {MedVR},
pubstate = {published},
tppubtype = {incollection}
}
As persons with disabilities age, progressive declines in health and medical status can challenge the adaptive resources required to maintain functional independence and quality of life [1]. These challenges are further compounded by economic factors, medication side effects, loss of a spouse or caregiver, and psychosocial disorders [1-2]. The gradual loss of functional independence and increased reliance on others for transportation, access to general medical and rehabilitation care can be jeopardized [2]. The combination of these factors when seen in the context of the average increase in lifespan in industrialized societies has lead to a growing crisis that is truly global in proportion. While research indicates that functional motor capacity can be improved, maintained, or recovered via consistent participation in a motor exercise and rehabilitation regimen [3], independent adherence to such preventative and/or rehabilitative programming outside the clinic setting is notoriously low [1]. This state of affairs has produced a compelling and ethical motivation to address the needs of individuals who are aging with disabilities by promoting home-based access to low-cost, interactive virtual reality (VR) systems designed to engage and motivate individuals to participate with "game"-driven physical activities and rehabilitation programming. The creation of such systems could serve to enhance, maintain and rehabilitate the sensorimotor processes that are needed to maximize independence and quality of life. This is the theme of the workshop to be presented at the MMVR conference.
Parsons, Thomas D.
Affect-sensitive Virtual Standardized Patient Interface System Book Section
In: Clinical Technologies: Concepts, Methodologies, Tools and Applications, vol. 3, 2011.
Abstract | Links | BibTeX | Tags: MedVR, Virtual Humans
@incollection{parsons_affect-sensitive_2011,
title = {Affect-sensitive Virtual Standardized Patient Interface System},
author = {Thomas D. Parsons},
url = {http://ict.usc.edu/pubs/Affect-Sensitive%20Virtual%20Standardized%20Patient%20Interface%20System.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Clinical Technologies: Concepts, Methodologies, Tools and Applications},
volume = {3},
abstract = {Virtual Standardized Patients (VSPs) are advanced conversational virtual human agents that have been applied to training of clinicians. These interactive agents portray standardized patient scenarios involving VSPs with clinical or physical conditions. VSPs are capable of verbal and nonverbal interaction (receptive and expressive communication) with a clinician in an effort to enhance differential diagnosis of psychiatric disorders and teach interpersonal skills. This chapter describes the design and development of both software to create social interaction modules on a VSP platform and individualized affective models for affect recognition. This author describes clinically relevant scenarios for affect elicitation and protocols for reliable affect recognition. Further, there is an elucidation of a VSP interface system that has the capacity to monitor the trainee's affective response using physiological signals. Research findings will be summarized from studies on (1) the usability and applicability of VSPs with training clinicians on various mental health disorders (e.g., adolescent male with conduct disorder; adolescent female who has recently been physically traumatized); and (2) preliminary use of the affect-sensitive system to systematically assess and manipulate aspects of VSPs to more fully develop cognitive and affective models of virtual humans with pathological characteristics.},
keywords = {MedVR, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Virtual Standardized Patients (VSPs) are advanced conversational virtual human agents that have been applied to training of clinicians. These interactive agents portray standardized patient scenarios involving VSPs with clinical or physical conditions. VSPs are capable of verbal and nonverbal interaction (receptive and expressive communication) with a clinician in an effort to enhance differential diagnosis of psychiatric disorders and teach interpersonal skills. This chapter describes the design and development of both software to create social interaction modules on a VSP platform and individualized affective models for affect recognition. This author describes clinically relevant scenarios for affect elicitation and protocols for reliable affect recognition. Further, there is an elucidation of a VSP interface system that has the capacity to monitor the trainee's affective response using physiological signals. Research findings will be summarized from studies on (1) the usability and applicability of VSPs with training clinicians on various mental health disorders (e.g., adolescent male with conduct disorder; adolescent female who has recently been physically traumatized); and (2) preliminary use of the affect-sensitive system to systematically assess and manipulate aspects of VSPs to more fully develop cognitive and affective models of virtual humans with pathological characteristics.
Hobbs, Jerry R.; Gordon, Andrew S.
The Deep Lexical Semantics of Emotions Book Section
In: Ahmad, Khurshid (Ed.): Affective Computing and Sentiment Analysis: Emotion, Metaphor and Terminology, vol. 45, pp. 27–43, Springer, New York, 2011.
Links | BibTeX | Tags: The Narrative Group
@incollection{hobbs_deep_2011,
title = {The Deep Lexical Semantics of Emotions},
author = {Jerry R. Hobbs and Andrew S. Gordon},
editor = {Khurshid Ahmad},
url = {http://ict.usc.edu//pubs/The%20Deep%20Lexical%20Semantics%20of%20Emotions%20Affective%20Computing%20and%20Sentiment%20Analysis-%20Emotion%20Metaphor%20and%20Terminology.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Affective Computing and Sentiment Analysis: Emotion, Metaphor and Terminology},
volume = {45},
pages = {27–43},
publisher = {Springer},
address = {New York},
series = {Text, Speech and Language Technology},
keywords = {The Narrative Group},
pubstate = {published},
tppubtype = {incollection}
}
Morbini, Fabrizio; Schubert, Lenhart
Metareasoning as an Integral Part of Commonsense and Autocognitive Reasoning Book Section
In: Cox, Michael T.; Raja, Anita (Ed.): Metareasoning: Thinking about thinking, MIT Press, 2011.
Abstract | Links | BibTeX | Tags:
@incollection{morbini_metareasoning_2011,
title = {Metareasoning as an Integral Part of Commonsense and Autocognitive Reasoning},
author = {Fabrizio Morbini and Lenhart Schubert},
editor = {Michael T. Cox and Anita Raja},
url = {http://ict.usc.edu/pubs/Metareasoning%20as%20an%20Integral%20Part%20of%20Commonsense%20and%20Autocognitive%20Reasoning.pdf},
year = {2011},
date = {2011-01-01},
booktitle = {Metareasoning: Thinking about thinking},
publisher = {MIT Press},
abstract = {In this chapter we summarize our progress towards building a self-aware agent based on the notion of explicit self-awareness (Schubert, 2005). An explicitly self-aware agent is characterized by 1) being based on extensive, human-like knowledge about the world and itself, 2) being transparent both in its behavior and in how the knowledge is represented and used, and 3) being able to communicate in natural language and directly display awareness through its dialogues. The second point is not strictly related to self-awareness, but it facilitates implementation of some aspects, such as answer explanation, and simplifies testing and debugging of the agent. In addition, we emphasize the importance of meta-level reasoning in commonsense reasoning and self-awareness, while questioning the common view of agent control structure in terms of separate object-level and meta-level strata. Instead, we suggest a "continual planning" (and execution) control structure wherein the agent's meta-level and object-level reasoning steps mingle seamlessly. We first review the requirements imposed by explicit self-awareness and by this intermingling of object and meta-level reasoning on the knowledge representation and reasoning system and then describe how these have been realized in the new version of the EPILOG system. Then we demonstrate our agent on a few questions, each related to some aspects of self-awareness. Finally we conclude with a discussion of our long-term plans.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
In this chapter we summarize our progress towards building a self-aware agent based on the notion of explicit self-awareness (Schubert, 2005). An explicitly self-aware agent is characterized by 1) being based on extensive, human-like knowledge about the world and itself, 2) being transparent both in its behavior and in how the knowledge is represented and used, and 3) being able to communicate in natural language and directly display awareness through its dialogues. The second point is not strictly related to self-awareness, but it facilitates implementation of some aspects, such as answer explanation, and simplifies testing and debugging of the agent. In addition, we emphasize the importance of meta-level reasoning in commonsense reasoning and self-awareness, while questioning the common view of agent control structure in terms of separate object-level and meta-level strata. Instead, we suggest a "continual planning" (and execution) control structure wherein the agent's meta-level and object-level reasoning steps mingle seamlessly. We first review the requirements imposed by explicit self-awareness and by this intermingling of object and meta-level reasoning on the knowledge representation and reasoning system and then describe how these have been realized in the new version of the EPILOG system. Then we demonstrate our agent on a few questions, each related to some aspects of self-awareness. Finally we conclude with a discussion of our long-term plans.
2010
Gratch, Jonathan
In: Doveling, Katrin; Scheve, Christian; Konijn, Elly A. (Ed.): The Routledge Handbook of Emotions and Mass Media, 2010, ISBN: 978-0-415-48160-1.
Links | BibTeX | Tags: Virtual Humans
@incollection{gratch_emotionally_2010,
title = {Emotionally resonant media: Advances in sensing, understanding and influencing human emotion through interactive media},
author = {Jonathan Gratch},
editor = {Katrin Doveling and Christian Scheve and Elly A. Konijn},
url = {http://ict.usc.edu/pubs/Emotionally%20resonant%20media.pdf},
isbn = {978-0-415-48160-1},
year = {2010},
date = {2010-09-01},
booktitle = {The Routledge Handbook of Emotions and Mass Media},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Morency, Louis-Philippe
The Role of Context in Affective Behavior Understanding Book Section
In: Social Emotions in nature and Artifact: Emotions in Human and Human-Computer Interaction, 2010.
Abstract | Links | BibTeX | Tags: Virtual Humans
@incollection{morency_role_2010,
title = {The Role of Context in Affective Behavior Understanding},
author = {Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Role_of_Context_in_Afective_Behavior.pdf},
year = {2010},
date = {2010-09-01},
booktitle = {Social Emotions in nature and Artifact: Emotions in Human and Human-Computer Interaction},
abstract = {Face-to-face communication is highly interactive. Even when only one person speaks at the time, other participants exchange information continuously amongst themselves and with the speaker through gesture, gaze, posture and facial expressions. Such affective feedback is an essential and predictable aspect of natural conversation and its absence can significantly disrupt participants ability to communicate. During multiparty interactions such as in meetings, information is exchanged between participants using both audio and visual channels. Visual feedback can range from a simple eye glance to a large arm gesture or posture change. One important visual cue is head nod during conversation. Head nods are used for displaying agreement, grounding information or during turn-taking. Recognizing these affective gestures is important for understanding all the information exchanged during a meeting or conversation, and can be particularly crucial for identifying more subtle factors such as the effectiveness of communication, points of confusion, status relationships between participants, or the diagnosis social disorders. This chapter argues that it is possible to significantly improve state-of-the art recognition techniques by exploiting regularities in how people communicate. People do not provide affective feedback at random. Rather they react to the current topic, previous utterances and the speaker's current verbal and nonverbal behavior. For example, listeners are far more likely to nod or shake if the speaker has just asked them a question, and incorporating such dialogue context can improve recognition performance during human-robot interaction. More generally, speakers and listeners co-produce a range of lexical, prosodic, and nonverbal patterns. Our goal is to automatically discover these patterns using only easily observable features of human face-to-face interaction (e.g. prosodic features and eye gaze), and exploit them to improve recognition accuracy.},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Face-to-face communication is highly interactive. Even when only one person speaks at the time, other participants exchange information continuously amongst themselves and with the speaker through gesture, gaze, posture and facial expressions. Such affective feedback is an essential and predictable aspect of natural conversation and its absence can significantly disrupt participants ability to communicate. During multiparty interactions such as in meetings, information is exchanged between participants using both audio and visual channels. Visual feedback can range from a simple eye glance to a large arm gesture or posture change. One important visual cue is head nod during conversation. Head nods are used for displaying agreement, grounding information or during turn-taking. Recognizing these affective gestures is important for understanding all the information exchanged during a meeting or conversation, and can be particularly crucial for identifying more subtle factors such as the effectiveness of communication, points of confusion, status relationships between participants, or the diagnosis social disorders. This chapter argues that it is possible to significantly improve state-of-the art recognition techniques by exploiting regularities in how people communicate. People do not provide affective feedback at random. Rather they react to the current topic, previous utterances and the speaker's current verbal and nonverbal behavior. For example, listeners are far more likely to nod or shake if the speaker has just asked them a question, and incorporating such dialogue context can improve recognition performance during human-robot interaction. More generally, speakers and listeners co-produce a range of lexical, prosodic, and nonverbal patterns. Our goal is to automatically discover these patterns using only easily observable features of human face-to-face interaction (e.g. prosodic features and eye gaze), and exploit them to improve recognition accuracy.
Morie, Jacquelyn; Fink, Mike
Introduction to the Visual Effect Society Visual Effects Handbook Book Section
In: The Visual Effect Society Visual Effects Handbook, 2010.
@incollection{morie_introduction_2010,
title = {Introduction to the Visual Effect Society Visual Effects Handbook},
author = {Jacquelyn Morie and Mike Fink},
year = {2010},
date = {2010-02-01},
booktitle = {The Visual Effect Society Visual Effects Handbook},
abstract = {Over the last decade, visual effects artists, when asked what they do for a living, have gone from hearing the response "What's that?", to hearing "Wow, that's cool!", followed by a pause that implied that there still lurked a... "What's that?" This book is meant to lift the veil on the people behind the curtain, and answer that question. We hope it provides, for professionals, students, or the merely interested, an exhaustive source that clearly describes and explains the techniques we use in the incredibly creative process of visual effects.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
Over the last decade, visual effects artists, when asked what they do for a living, have gone from hearing the response "What's that?", to hearing "Wow, that's cool!", followed by a pause that implied that there still lurked a... "What's that?" This book is meant to lift the veil on the people behind the curtain, and answer that question. We hope it provides, for professionals, students, or the merely interested, an exhaustive source that clearly describes and explains the techniques we use in the incredibly creative process of visual effects.
Marsella, Stacy C.; Gratch, Jonathan; Petta, Paola
Computational Models of Emotion Book Section
In: Scherer, K. R.; Bänziger, T.; Roesch, (Ed.): A blueprint for an affectively competent agent: Cross-fertilization between Emotion Psychology, Affective Neuroscience, and Affective Computing, Oxford University Press, Oxford, 2010.
Abstract | Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{marsella_computational_2010,
title = {Computational Models of Emotion},
author = {Stacy C. Marsella and Jonathan Gratch and Paola Petta},
editor = {K. R. Scherer and T. Bänziger and Roesch},
url = {http://ict.usc.edu/pubs/Computational%20Models%20of%20Emotion.pdf},
year = {2010},
date = {2010-01-01},
booktitle = {A blueprint for an affectively competent agent: Cross-fertilization between Emotion Psychology, Affective Neuroscience, and Affective Computing},
publisher = {Oxford University Press},
address = {Oxford},
abstract = {Recent years have seen a significant expansion in research on computational models of human emotional processes, driven both by their potential for basic research on emotion and cognition as well as their promise for an ever increasing range of applications. This has led to a truly interdisciplinary, mutually beneficial partnership between emotion research in psychology and computational science, of which this volume is an exemplar. To understand this partnership and its potential for transforming existing practices in emotion research across disciplines and for disclosing important novel areas of research, we explore in this chapter the history of work in computational models of emotion including the various uses to which they have been put, the theoretical traditions that have shaped their development, and how these uses and traditions are reflected in their underlying architectures.},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Recent years have seen a significant expansion in research on computational models of human emotional processes, driven both by their potential for basic research on emotion and cognition as well as their promise for an ever increasing range of applications. This has led to a truly interdisciplinary, mutually beneficial partnership between emotion research in psychology and computational science, of which this volume is an exemplar. To understand this partnership and its potential for transforming existing practices in emotion research across disciplines and for disclosing important novel areas of research, we explore in this chapter the history of work in computational models of emotion including the various uses to which they have been put, the theoretical traditions that have shaped their development, and how these uses and traditions are reflected in their underlying architectures.
Ogan, Amy; Lane, H. Chad
Virtual Learning Environments for Culture and Intercultural Competence Book Section
In: Handbook of Research on Culturally-Aware Information Technology: Perspectives and Models, pp. 501–519, IGI Global, Hershey, PA, 2010.
Abstract | Links | BibTeX | Tags:
@incollection{ogan_virtual_2010,
title = {Virtual Learning Environments for Culture and Intercultural Competence},
author = {Amy Ogan and H. Chad Lane},
url = {http://ict.usc.edu/pubs/Virtual%20Learning%20Environments%20for%20Culture%20and%20Intercultural%20Competence.pdf},
year = {2010},
date = {2010-01-01},
booktitle = {Handbook of Research on Culturally-Aware Information Technology: Perspectives and Models},
pages = {501–519},
publisher = {IGI Global},
address = {Hershey, PA},
abstract = {The authors review six virtual learning environments built to support the acquisition of cultural knowl- edge and communication skills: ATL, BiLAT, Croquelandia, Second China, TLCTS, and VECTOR. Each leverages modern 3D video game engine technology which allows high-fidelity simulation of new cultural settings, including representations of buildings, streets, art work, dress, voice, gestures, and more. To bring more realism to simulated cultural interactions, several of the systems are driven by artificial intelligence (AI) models of culture, communication, and emotion. Additionally, several rely on narrative-based techniques to place the target culture in context and enhance motivation of those using the systems. The authors conclude with a discussion of the reviewed environments and identify potential research directions that focus on (1) intercultural competence skills, (2) learner assessment, and (3) cultural model building and validation.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
The authors review six virtual learning environments built to support the acquisition of cultural knowl- edge and communication skills: ATL, BiLAT, Croquelandia, Second China, TLCTS, and VECTOR. Each leverages modern 3D video game engine technology which allows high-fidelity simulation of new cultural settings, including representations of buildings, streets, art work, dress, voice, gestures, and more. To bring more realism to simulated cultural interactions, several of the systems are driven by artificial intelligence (AI) models of culture, communication, and emotion. Additionally, several rely on narrative-based techniques to place the target culture in context and enhance motivation of those using the systems. The authors conclude with a discussion of the reviewed environments and identify potential research directions that focus on (1) intercultural competence skills, (2) learner assessment, and (3) cultural model building and validation.
Heylen, Dirk; Bevacqua, Elisabetta; Pelachaud, Catherine; Poggi, Isabella; Gratch, Jonathan; Schröder, Marc
Generating Listening Behaviour Book Section
In: Handbook of Emotion-Oriented Technologies, 2010.
Abstract | Links | BibTeX | Tags: Virtual Humans
@incollection{heylen_generating_2010,
title = {Generating Listening Behaviour},
author = {Dirk Heylen and Elisabetta Bevacqua and Catherine Pelachaud and Isabella Poggi and Jonathan Gratch and Marc Schröder},
url = {http://ict.usc.edu/pubs/Generating%20Listening%20Behaviour.pdf},
year = {2010},
date = {2010-01-01},
booktitle = {Handbook of Emotion-Oriented Technologies},
abstract = {In face-to-face conversations listeners provide feedback and comments at the same time as speakers are uttering their words and sentence. This 'talk' in the backchannel provides speakers with information about reception and acceptance – or lack thereof – of their speech. Listeners, through short verbalisations and non-verbal signals, show how they are engaged in the dialogue. The lack of incremental, real-time processing has hampered the creation of conversational agents that can respond to the human interlocutor in real time as the speech is being produced. The need for such feedback in conversational agents is, however, undeniable for reasons of naturalism or believability, to increase the efficiency of communication and to show engagement and building of rapport. In this chapter, the joint activity of speakers and listeners that constitutes a conversation is more closely examined and the work that is devoted to the construction of agents that are able to show that they are listening is reviewed. Two issues are dealt with in more detail. The first is the search for appropriate responses for an agent to display. The second is the study of how listening responses may increase rapport between agents and their human partners in conversation.},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
In face-to-face conversations listeners provide feedback and comments at the same time as speakers are uttering their words and sentence. This 'talk' in the backchannel provides speakers with information about reception and acceptance – or lack thereof – of their speech. Listeners, through short verbalisations and non-verbal signals, show how they are engaged in the dialogue. The lack of incremental, real-time processing has hampered the creation of conversational agents that can respond to the human interlocutor in real time as the speech is being produced. The need for such feedback in conversational agents is, however, undeniable for reasons of naturalism or believability, to increase the efficiency of communication and to show engagement and building of rapport. In this chapter, the joint activity of speakers and listeners that constitutes a conversation is more closely examined and the work that is devoted to the construction of agents that are able to show that they are listening is reviewed. Two issues are dealt with in more detail. The first is the search for appropriate responses for an agent to display. The second is the study of how listening responses may increase rapport between agents and their human partners in conversation.
2009
Clark, Richard E.; Yates, Kenneth A.; Early, Sean; Moulton, Kathrine
In: Handbook of Training and Improving Workplace Performance, vol. 1, International Society for Performance Improvement, Washington, DC, 2009.
Abstract | Links | BibTeX | Tags:
@incollection{clark_analysis_2009,
title = {An analysis of the failure of electronic media and discovery-based learning: Evidence for the performance benefits of guided training methods.},
author = {Richard E. Clark and Kenneth A. Yates and Sean Early and Kathrine Moulton},
url = {http://www.ict.usc.edu/pubs/An%20analysis%20of%20the%20failure%20of%20electronic%20media%20and%20discovery-based%20learning.pdf},
year = {2009},
date = {2009-12-01},
booktitle = {Handbook of Training and Improving Workplace Performance},
volume = {1},
publisher = {International Society for Performance Improvement},
address = {Washington, DC},
abstract = {This chapter will present a direct, evidence-based argument that while media provide economic benefits for training organizations, they have not and will not influence learning, motivation or work performance. We begin with a discussion of popular instructional design models based on discovery and problem-based learning and argue that a half-century of research has indicated that they are also ineffective for all but a small minority of learners. We will briefly describe the half-century of research that supports our conclusions and describe the consequences for business and education. Contrary to popular belief regarding the importance of media in training, we will suggest that a handful of specific training methods are the only environmental factors that have been found to have a major influence on learning and performance. We will argue that the methods we describe are successful in many different delivery media because they support the mental process by which people learn complex knowledge. We will then describe an example of the current training models that promote guided learning. The chapter will conclude with a description of a powerful tool for selecting the most cost-beneficial media to deliver guided learning methods for nearly any training or performance goal.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This chapter will present a direct, evidence-based argument that while media provide economic benefits for training organizations, they have not and will not influence learning, motivation or work performance. We begin with a discussion of popular instructional design models based on discovery and problem-based learning and argue that a half-century of research has indicated that they are also ineffective for all but a small minority of learners. We will briefly describe the half-century of research that supports our conclusions and describe the consequences for business and education. Contrary to popular belief regarding the importance of media in training, we will suggest that a handful of specific training methods are the only environmental factors that have been found to have a major influence on learning and performance. We will argue that the methods we describe are successful in many different delivery media because they support the mental process by which people learn complex knowledge. We will then describe an example of the current training models that promote guided learning. The chapter will conclude with a description of a powerful tool for selecting the most cost-beneficial media to deliver guided learning methods for nearly any training or performance goal.
Morie, Jacquelyn; El-Nasr, Magy Seif; Drachen, Anders
A Scientific Look at the Design of Aesthetically and Emotionally Engaging Interactive Entertainment Experiences Book Section
In: Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives, 2009.
Abstract | Links | BibTeX | Tags:
@incollection{morie_scientific_2009,
title = {A Scientific Look at the Design of Aesthetically and Emotionally Engaging Interactive Entertainment Experiences},
author = {Jacquelyn Morie and Magy Seif El-Nasr and Anders Drachen},
url = {http://ict.usc.edu/pubs/Emotional%20Affordances%20within%20Interactive%20Experiences-%20A%20scientific%20approach%20to%20understanding%20artistic%20tools%20for%20stimulating%20emotions%20within%20Interactive%20Entertainment%20Experiences.pdf},
year = {2009},
date = {2009-12-01},
booktitle = {Affective Computing and Interaction: Psychological, Cognitive and Neuroscientific Perspectives},
abstract = {The interactive entertainment industry has become a multi-billion dollar industry with revenues overcom- ing those of the movie industry (ESA, 2009). Beyond the demand for high fidelity graphics or stylized imagery, participants in these environments have come to expect certain aesthetic and artistic qualities that engage them at a very deep emotional level. These qualities pertain to the visual aesthetic, dramatic structure, pacing, and sensory systems embedded within the experience. All these qualities are carefully crafted by the creator of the interactive experience to evoke affect. In this book chapter, the authors will attempt to discuss the design techniques developed by artists to craft such emotionally engaging experi- ences. In addition, they take a scientific approach whereby we discuss case studies of the use of these design techniques and experiments that attempt to validate their use in stimulating emotions.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
The interactive entertainment industry has become a multi-billion dollar industry with revenues overcom- ing those of the movie industry (ESA, 2009). Beyond the demand for high fidelity graphics or stylized imagery, participants in these environments have come to expect certain aesthetic and artistic qualities that engage them at a very deep emotional level. These qualities pertain to the visual aesthetic, dramatic structure, pacing, and sensory systems embedded within the experience. All these qualities are carefully crafted by the creator of the interactive experience to evoke affect. In this book chapter, the authors will attempt to discuss the design techniques developed by artists to craft such emotionally engaging experi- ences. In addition, they take a scientific approach whereby we discuss case studies of the use of these design techniques and experiments that attempt to validate their use in stimulating emotions.
Lane, H. Chad; Johnson, W. Lewis
Intelligent Tutoring and Pedagogical Experience Manipulation in Virtual Learning Environments Book Section
In: The Handbook of Virtual Environments for Training and Education, vol. 2, pp. 393–406, Praeger Security International, Westport, CT, 2009.
Abstract | Links | BibTeX | Tags:
@incollection{lane_intelligent_2009,
title = {Intelligent Tutoring and Pedagogical Experience Manipulation in Virtual Learning Environments},
author = {H. Chad Lane and W. Lewis Johnson},
url = {http://ict.usc.edu/pubs/Intelligent%20Tutoring%20and%20Pedagogical%20Experience%20Manipulation%20in%20Virtual%20Learning%20Environments.pdf},
year = {2009},
date = {2009-01-01},
booktitle = {The Handbook of Virtual Environments for Training and Education},
volume = {2},
pages = {393–406},
publisher = {Praeger Security International},
address = {Westport, CT},
abstract = {This book chapter covers how techniques of artificial intelligence (AI) can be used to promote and enhance learning in virtual environments. The authors summarize prominent historical examples of intelligent tutors in immersive simulations and serious games, discuss evidence supporting the value of implementing such tutors, highlight current areas of active research, and identify crucial open questions that require continued exploration.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This book chapter covers how techniques of artificial intelligence (AI) can be used to promote and enhance learning in virtual environments. The authors summarize prominent historical examples of intelligent tutors in immersive simulations and serious games, discuss evidence supporting the value of implementing such tutors, highlight current areas of active research, and identify crucial open questions that require continued exploration.
Gordon, Andrew S.
Story-Based Learning Environments Book Section
In: The PSI Handbook of Virtual Environments for Training and Education: Developments for the Military and Beyond, Volume 2: Components and Training Technologies, vol. 2, Praeger Security International, Westport, CT, 2009.
Links | BibTeX | Tags: The Narrative Group
@incollection{gordon_story-based_2009,
title = {Story-Based Learning Environments},
author = {Andrew S. Gordon},
url = {http://www.ict.usc.edu/pubs/Story%20based%20Learning%20Environments.pdf},
year = {2009},
date = {2009-01-01},
booktitle = {The PSI Handbook of Virtual Environments for Training and Education: Developments for the Military and Beyond, Volume 2: Components and Training Technologies},
volume = {2},
publisher = {Praeger Security International},
address = {Westport, CT},
keywords = {The Narrative Group},
pubstate = {published},
tppubtype = {incollection}
}
2008
Traum, David; Swartout, William; Gratch, Jonathan; Marsella, Stacy C.
A Virtual Human Dialogue Model for Non-team Interaction Book Section
In: Recent Trends in Discourse and Dialogue, vol. 39, pp. 45–67, Springer, Dordecht, The Netherlands, 2008.
Abstract | Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{traum_virtual_2008,
title = {A Virtual Human Dialogue Model for Non-team Interaction},
author = {David Traum and William Swartout and Jonathan Gratch and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/A%20Virtual%20Human%20Dialogue%20Model%20for%20Non-team%20Interaction.pdf},
year = {2008},
date = {2008-01-01},
booktitle = {Recent Trends in Discourse and Dialogue},
volume = {39},
pages = {45–67},
publisher = {Springer},
address = {Dordecht, The Netherlands},
series = {Text, Speech and Language Technology},
abstract = {We describe the dialogue model for the virtual humans developed at the Institute for Creative Technologies at the University of Southern California. The dialogue model contains a rich set of information state and dialogue moves to allow a wide range of behaviour in multimodal, multiparty interaction. We extend this model to enable non-team negotiation, using ideas from social science literature on negotiation and implemented strategies and dialogue moves for this area. We present a virtual human doctor who uses this model to engage in multimodal negotiation dialogue with people from other organisations. The doctor is part of the SASO-ST system, used for training for non-team interactions.},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
We describe the dialogue model for the virtual humans developed at the Institute for Creative Technologies at the University of Southern California. The dialogue model contains a rich set of information state and dialogue moves to allow a wide range of behaviour in multimodal, multiparty interaction. We extend this model to enable non-team negotiation, using ideas from social science literature on negotiation and implemented strategies and dialogue moves for this area. We present a virtual human doctor who uses this model to engage in multimodal negotiation dialogue with people from other organisations. The doctor is part of the SASO-ST system, used for training for non-team interactions.
Fullerton, Tracy; Fron, Janine; Pearce, Celia; Morie, Jacquelyn
Getting girls into the game: Towards a "Virtuous Cycle" Book Section
In: Beyond Barbie and Mortal Kombat: New Perspectives on Gender and Computer Games, MIT Press, 2008.
Abstract | Links | BibTeX | Tags:
@incollection{fullerton_getting_2008,
title = {Getting girls into the game: Towards a "Virtuous Cycle"},
author = {Tracy Fullerton and Janine Fron and Celia Pearce and Jacquelyn Morie},
url = {http://ict.usc.edu/pubs/Getting%20girls%20into%20the%20game-%20Towards%20a%20Virtuous%20Cycle.pdf},
year = {2008},
date = {2008-01-01},
booktitle = {Beyond Barbie and Mortal Kombat: New Perspectives on Gender and Computer Games},
publisher = {MIT Press},
abstract = {This chapter was authored by Ludica, a game design and art collective devoted to developing innovative design concepts that explore the potential of games to express women's narratives, aesthetics, culture, and play. Ludica's members combine experience in virtual reality, art and technology collaboration, game design, computer science, academic research, the game and theme park industries, fine art, photography, and graphic design. A critical component of Ludica's mission is to identify and develop methodologies and organizational contexts that provide more inclusive and productive environments in which women can actively contribute to the game design process. In this chapter, we bring to bear our own experience as designers, authors, researchers, and teachers, as well as conversations with women in industry, academia, and current and former students, to outline a vision for a future in which the culture of game design is more conducive to female participation.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This chapter was authored by Ludica, a game design and art collective devoted to developing innovative design concepts that explore the potential of games to express women's narratives, aesthetics, culture, and play. Ludica's members combine experience in virtual reality, art and technology collaboration, game design, computer science, academic research, the game and theme park industries, fine art, photography, and graphic design. A critical component of Ludica's mission is to identify and develop methodologies and organizational contexts that provide more inclusive and productive environments in which women can actively contribute to the game design process. In this chapter, we bring to bear our own experience as designers, authors, researchers, and teachers, as well as conversations with women in industry, academia, and current and former students, to outline a vision for a future in which the culture of game design is more conducive to female participation.
2007
Morie, Jacquelyn; Tortell, Rebecca; Williams, Josh
Would You Like to Play a Game? Experience and Expectation in Game-Based Learning Environments Book Section
In: Computer Games and Team and Individual Learning, Amsterdam, The Netherlands, 2007.
@incollection{morie_would_2007,
title = {Would You Like to Play a Game? Experience and Expectation in Game-Based Learning Environments},
author = {Jacquelyn Morie and Rebecca Tortell and Josh Williams},
year = {2007},
date = {2007-11-01},
booktitle = {Computer Games and Team and Individual Learning},
address = {Amsterdam, The Netherlands},
abstract = {We present results from a series of experiments that looked at how previous experience and immediate priming affect a user's arousal state, performance and memory in a virtual environment used for training. We found that people's game play experience had effects on these measures, and that if participants expected the environment to be a game, they approached it with expectations that were not always conducive to optimal training. We suggest that the type of game being used for training will have the best outcome if users are familiar with that mode and have the appropriate schema to approach the training.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
We present results from a series of experiments that looked at how previous experience and immediate priming affect a user's arousal state, performance and memory in a virtual environment used for training. We found that people's game play experience had effects on these measures, and that if participants expected the environment to be a game, they approached it with expectations that were not always conducive to optimal training. We suggest that the type of game being used for training will have the best outcome if users are familiar with that mode and have the appropriate schema to approach the training.
Gratch, Jonathan; Marsella, Stacy C.
The Architectural Role of Emotion in Cognitive Systems Book Section
In: Integrated Models of Cognitive Systems, Oxford University Press, New York, 2007.
Abstract | Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{gratch_architectural_2007,
title = {The Architectural Role of Emotion in Cognitive Systems},
author = {Jonathan Gratch and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/The%20Architectural%20Role%20of%20Emotion%20in%20Cognitive%20Systems.pdf},
year = {2007},
date = {2007-03-01},
booktitle = {Integrated Models of Cognitive Systems},
publisher = {Oxford University Press},
address = {New York},
abstract = {In this chapter, we will revive an old argument that theories of human emotion can give insight into the design and control of complex cognitive systems. In particular, we claim that appraisal theories of emotion provide essential insight into the influences of emotion over cognition and can help translate such findings into concrete guidance for the design of cognitive systems. Ap- praisal theory claims that emotion plays a central and functional role in sensing external events, characterizing them as opportunity or threats and recruiting the cognitive, physical and social resources needed to adaptively respond. Further, because it argues for a close association be- tween emotion and cognition, the theoretical claims of appraisal theory can be recast as a re- quirement specification for how to build a cognitive system. This specification asserts a set of judgments that must be supported in order to correctly interpret and respond to stimuli and pro- vides a unifying framework for integrating these judgments into a coherent physical or social re- sponse. This chapter elaborates argument in some detail based on our joint experience in build- ing complex cognitive systems and computational models of emotion.},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
In this chapter, we will revive an old argument that theories of human emotion can give insight into the design and control of complex cognitive systems. In particular, we claim that appraisal theories of emotion provide essential insight into the influences of emotion over cognition and can help translate such findings into concrete guidance for the design of cognitive systems. Ap- praisal theory claims that emotion plays a central and functional role in sensing external events, characterizing them as opportunity or threats and recruiting the cognitive, physical and social resources needed to adaptively respond. Further, because it argues for a close association be- tween emotion and cognition, the theoretical claims of appraisal theory can be recast as a re- quirement specification for how to build a cognitive system. This specification asserts a set of judgments that must be supported in order to correctly interpret and respond to stimuli and pro- vides a unifying framework for integrating these judgments into a coherent physical or social re- sponse. This chapter elaborates argument in some detail based on our joint experience in build- ing complex cognitive systems and computational models of emotion.
2006
Gordon, Andrew S.
Language evidence for changes in a Theory of Mind Book Section
In: Arbib, Michael A. (Ed.): Action to Language via the Mirror Neuron System, University of Cambridge Press, Cambridge, UK, 2006, ISBN: 978-0-521-84755-1.
Abstract | Links | BibTeX | Tags: The Narrative Group
@incollection{gordon_language_2006,
title = {Language evidence for changes in a Theory of Mind},
author = {Andrew S. Gordon},
editor = {Michael A. Arbib},
url = {http://www.cambridge.org/gb/knowledge/isbn/item1172518/?site_locale=en_GB},
isbn = {978-0-521-84755-1},
year = {2006},
date = {2006-09-01},
booktitle = {Action to Language via the Mirror Neuron System},
publisher = {University of Cambridge Press},
address = {Cambridge, UK},
abstract = {Mirror neurons may hold the brain's key to social interaction - each coding not only a particular action or emotion but also the recognition of that action or emotion in others. The Mirror System Hypothesis adds an evolutionary arrow to the story - from the mirror system for hand actions, shared with monkeys and chimpanzees, to the uniquely human mirror system for language. In this accessible 2006 volume, experts from child development, computer science, linguistics, neuroscience, primatology and robotics present and analyse the mirror system and show how studies of action and language can illuminate each other. Topics discussed in the fifteen chapters include: what do chimpanzees and humans have in common? Does the human capability for language rest on brain mechanisms shared with other animals? How do human infants acquire language? What can be learned from imaging the human brain? How are sign- and spoken-language related? Will robots learn to act and speak like humans?},
keywords = {The Narrative Group},
pubstate = {published},
tppubtype = {incollection}
}
Mirror neurons may hold the brain's key to social interaction - each coding not only a particular action or emotion but also the recognition of that action or emotion in others. The Mirror System Hypothesis adds an evolutionary arrow to the story - from the mirror system for hand actions, shared with monkeys and chimpanzees, to the uniquely human mirror system for language. In this accessible 2006 volume, experts from child development, computer science, linguistics, neuroscience, primatology and robotics present and analyse the mirror system and show how studies of action and language can illuminate each other. Topics discussed in the fifteen chapters include: what do chimpanzees and humans have in common? Does the human capability for language rest on brain mechanisms shared with other animals? How do human infants acquire language? What can be learned from imaging the human brain? How are sign- and spoken-language related? Will robots learn to act and speak like humans?
Traum, David
Talking to Virtual Humans: Dialogue Models and Methodologies for Embodied Conversational Agents Book Section
In: Modeling Communication with Robots and Virtual Humans, pp. 296–309, 2006.
Abstract | Links | BibTeX | Tags: Virtual Humans
@incollection{traum_talking_2006,
title = {Talking to Virtual Humans: Dialogue Models and Methodologies for Embodied Conversational Agents},
author = {David Traum},
url = {http://ict.usc.edu/pubs/Talking%20to%20Virtual%20Humans.pdf},
year = {2006},
date = {2006-04-01},
booktitle = {Modeling Communication with Robots and Virtual Humans},
pages = {296–309},
abstract = {Virtual Humans are artificial characters who look and act like humans, but inhabit a simulated environment. One important aspect of many virtual humans is their communicative dialogue ability. In this paper we outline a methodology for study of dialogue behavior and construction of virtual humans. We also consider three architectures for different types of virtual humans that have been built at the Institute for Creative Technologies.},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Virtual Humans are artificial characters who look and act like humans, but inhabit a simulated environment. One important aspect of many virtual humans is their communicative dialogue ability. In this paper we outline a methodology for study of dialogue behavior and construction of virtual humans. We also consider three architectures for different types of virtual humans that have been built at the Institute for Creative Technologies.
Swartout, William; Gratch, Jonathan; Hill, Randall W.; Hovy, Eduard; Lindheim, Richard; Marsella, Stacy C.; Rickel, Jeff; Traum, David
Simulation Meets Hollywood: Integrating Graphics, Sound, Story and Character for Immersive Simulation Book Section
In: Multimodal Intelligent Information Presentation, vol. 27, pp. 305–321, Springer, Netherlands, 2006.
Abstract | Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{swartout_simulation_2006,
title = {Simulation Meets Hollywood: Integrating Graphics, Sound, Story and Character for Immersive Simulation},
author = {William Swartout and Jonathan Gratch and Randall W. Hill and Eduard Hovy and Richard Lindheim and Stacy C. Marsella and Jeff Rickel and David Traum},
url = {http://ict.usc.edu/pubs/SIMULATION%20MEETS%20HOLLYWOOD-%20Integrating%20Graphics,%20Sound,%20Story%20and%20Character%20for%20Immersive%20Simulation.pdf},
year = {2006},
date = {2006-01-01},
booktitle = {Multimodal Intelligent Information Presentation},
volume = {27},
pages = {305–321},
publisher = {Springer},
address = {Netherlands},
abstract = {The Institute for Creative Technologies was created at the University of Southern California with the goal of bringing together researchers in simulation technology to collaborate with people from the entertainment industry. The idea was that much more compelling simulations could be developed if researchers who understood state-of-the-art simulation technology worked together with writers and directors who knew how to create compelling stories and characters. This paper presents our first major effort to realize that vision, the Mission Rehearsal Exercise Project, which confronts a soldier trainee with the kinds of dilemmas he might reasonably encounter in a peacekeeping operation. The trainee is immersed in a synthetic world and interacts with virtual humans: artificially intelligent and graphically embodied conversational agents that understand and generate natural language, reason about world events and respond appropriately to the trainee's actions or commands. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have also joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. In this paper we describe the Mission Rehearsal Exercise system and the insights gained through this large-scale integration.},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
The Institute for Creative Technologies was created at the University of Southern California with the goal of bringing together researchers in simulation technology to collaborate with people from the entertainment industry. The idea was that much more compelling simulations could be developed if researchers who understood state-of-the-art simulation technology worked together with writers and directors who knew how to create compelling stories and characters. This paper presents our first major effort to realize that vision, the Mission Rehearsal Exercise Project, which confronts a soldier trainee with the kinds of dilemmas he might reasonably encounter in a peacekeeping operation. The trainee is immersed in a synthetic world and interacts with virtual humans: artificially intelligent and graphically embodied conversational agents that understand and generate natural language, reason about world events and respond appropriately to the trainee's actions or commands. This project is an ambitious exercise in integration, both in the sense of integrating technology with entertainment industry content, but also in that we have also joined a number of component technologies that have not been integrated before. This integration has not only raised new research issues, but it has also suggested some new approaches to difficult problems. In this paper we describe the Mission Rehearsal Exercise system and the insights gained through this large-scale integration.
Gratch, Jonathan; Mao, Wenji; Marsella, Stacy C.
Modeling Social Emotions and Social Attributions Book Section
In: Sun, R. (Ed.): Cognition and Multi-Agent Interaction: Extending Cognitive Modeling to Social Simulation, Cambridge University Press, 2006.
Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{gratch_modeling_2006,
title = {Modeling Social Emotions and Social Attributions},
author = {Jonathan Gratch and Wenji Mao and Stacy C. Marsella},
editor = {R. Sun},
url = {http://ict.usc.edu/pubs/Modeling%20Social%20Emotions%20and%20Social%20Attributions.pdf},
year = {2006},
date = {2006-01-01},
booktitle = {Cognition and Multi-Agent Interaction: Extending Cognitive Modeling to Social Simulation},
publisher = {Cambridge University Press},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
2005
Rizzo, Albert; Pair, Jarrell; McNerney, Peter J.; Eastlund, Ernie; Manson, Brian; Gratch, Jonathan; Hill, Randall W.; Swartout, William
Development of a VR Therapy Application for Iraq War Military Personnel with PTSD Book Section
In: Studies in Health Technology and Informatics, vol. 111, no. 13, pp. 407+413, 13th Annual Medicine Meets Virtual Reality Conference, Long Beach, CA, 2005.
Abstract | Links | BibTeX | Tags: MedVR, Virtual Humans
@incollection{rizzo_development_2005-1,
title = {Development of a VR Therapy Application for Iraq War Military Personnel with PTSD},
author = {Albert Rizzo and Jarrell Pair and Peter J. McNerney and Ernie Eastlund and Brian Manson and Jonathan Gratch and Randall W. Hill and William Swartout},
url = {http://ict.usc.edu/pubs/Development%20of%20a%20VR%20Therapy%20Application%20for%20Iraq%20War%20Veterans%20with%20PTSD.pdf},
year = {2005},
date = {2005-01-01},
booktitle = {Studies in Health Technology and Informatics},
volume = {111},
number = {13},
pages = {407+413},
address = {13th Annual Medicine Meets Virtual Reality Conference, Long Beach, CA},
series = {Medicine Meets Virtual Reality},
abstract = {Post Traumatic Stress Disorder (PTSD) is reported to be caused by traumatic events that are outside the range of usual human experiences including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that 1 out of 6 returning Iraq War military personnel are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) exposure therapy has been used in previous treatments of PTSD patients with reports of positive outcomes. The aim of the current paper is to specify the rationale, design and development of an Iraq War PTSD VR application that is being created from the virtual assets that were initially developed for theX-Box game entitled Full Spectrum Warrior which was inspired by a combat tactical training simulation, Full Spectrum Command.},
keywords = {MedVR, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Post Traumatic Stress Disorder (PTSD) is reported to be caused by traumatic events that are outside the range of usual human experiences including (but not limited to) military combat, violent personal assault, being kidnapped or taken hostage and terrorist attacks. Initial data suggests that 1 out of 6 returning Iraq War military personnel are exhibiting symptoms of depression, anxiety and PTSD. Virtual Reality (VR) exposure therapy has been used in previous treatments of PTSD patients with reports of positive outcomes. The aim of the current paper is to specify the rationale, design and development of an Iraq War PTSD VR application that is being created from the virtual assets that were initially developed for theX-Box game entitled Full Spectrum Warrior which was inspired by a combat tactical training simulation, Full Spectrum Command.
2004
Marsella, Stacy C.; Gratch, Jonathan; Rickel, Jeff
Expressive Behaviors for Virtual Worlds Book Section
In: Life-Like Characters: Tools, Affective Functions, and Applications, 2004.
Abstract | Links | BibTeX | Tags: Social Simulation, Virtual Humans
@incollection{marsella_expressive_2004,
title = {Expressive Behaviors for Virtual Worlds},
author = {Stacy C. Marsella and Jonathan Gratch and Jeff Rickel},
url = {http://ict.usc.edu/pubs/Expressive%20Behaviors%20for%20Virtual%20Worlds.pdf},
year = {2004},
date = {2004-06-01},
booktitle = {Life-Like Characters: Tools, Affective Functions, and Applications},
abstract = {A person's behavior provides signi⬚cant information about their emotional state, attitudes, and attention. Our goal is to create virtual humans that convey such information to people while interacting with them in virtual worlds. The virtual humans must respond dynamically to the events surrounding them, which are fundamentally influenced by users' actions, while providing an illusion of human-like behavior. A user must be able to interpret the dynamic cognitive and emotional state of the virtual humans using the same nonverbal cues that people use to understand one another. Towards these goals, we are integrating and extending components from three prior systems: a virtual human architecture with a wide range of cognitive and motor capabilities, a model of task-oriented emotional appraisal and socially situated planning, and a model of how emotions and coping impact physical behavior. We describe the key research issues and approach in each of these prior systems, as well as our integration and its initial implementation in a leadership training system.},
keywords = {Social Simulation, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
A person's behavior provides signi⬚cant information about their emotional state, attitudes, and attention. Our goal is to create virtual humans that convey such information to people while interacting with them in virtual worlds. The virtual humans must respond dynamically to the events surrounding them, which are fundamentally influenced by users' actions, while providing an illusion of human-like behavior. A user must be able to interpret the dynamic cognitive and emotional state of the virtual humans using the same nonverbal cues that people use to understand one another. Towards these goals, we are integrating and extending components from three prior systems: a virtual human architecture with a wide range of cognitive and motor capabilities, a model of task-oriented emotional appraisal and socially situated planning, and a model of how emotions and coping impact physical behavior. We describe the key research issues and approach in each of these prior systems, as well as our integration and its initial implementation in a leadership training system.
2003
Traum, David; Larsson, Staffan
The Information State Approach to Dialogue Management Book Section
In: Current and New Directions in Discourse and Dialogue, pp. 325–353, 2003.
Links | BibTeX | Tags: Virtual Humans
@incollection{traum_information_2003,
title = {The Information State Approach to Dialogue Management},
author = {David Traum and Staffan Larsson},
url = {http://ict.usc.edu/pubs/The%20Information%20State%20Approach%20to%20Dialogue%20Management.pdf},
year = {2003},
date = {2003-01-01},
booktitle = {Current and New Directions in Discourse and Dialogue},
pages = {325–353},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
2002
Rickel, Jeff; Johnson, W. Lewis
Extending Virtual Human to Support Team Training in Virtual Reality Book Section
In: Lakemeyer, G.; Nebel, B. (Ed.): Exploring Artificial Intelligence in the New Millennium, Morgan Kaufmann Publishers, San Francisco, CA, 2002.
Abstract | Links | BibTeX | Tags:
@incollection{rickel_extending_2002,
title = {Extending Virtual Human to Support Team Training in Virtual Reality},
author = {Jeff Rickel and W. Lewis Johnson},
editor = {G. Lakemeyer and B. Nebel},
url = {http://ict.usc.edu/pubs/Extending%20Virtual%20Humans%20to%20Support%20Team%20Training%20in%20Virtual%20Reality.pdf},
year = {2002},
date = {2002-01-01},
booktitle = {Exploring Artificial Intelligence in the New Millennium},
publisher = {Morgan Kaufmann Publishers},
address = {San Francisco, CA},
abstract = {This paper describes the use of virtual humans and distributed virtual reality to support team training, where students must learn their individual role in the team as well as how to coordinate their actions with their teammates. Students, instructors, and virtual humans cohabit a three-dimensional, interactive, simulated mock-up of their work environment, where they can practice together in realistic situations. The virtual humans can serve as instructors for individual students, and they can substitute for missing team members, allowing students to practive team tasks when some or all human instructors and teammates are unavailable. The paper describes our learning environment, the issues that arise in developing virtual humans for team training, and our design for the virtual humans, which is an extension of our Steve agent previously used for one-on-one tutoring.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
This paper describes the use of virtual humans and distributed virtual reality to support team training, where students must learn their individual role in the team as well as how to coordinate their actions with their teammates. Students, instructors, and virtual humans cohabit a three-dimensional, interactive, simulated mock-up of their work environment, where they can practice together in realistic situations. The virtual humans can serve as instructors for individual students, and they can substitute for missing team members, allowing students to practive team tasks when some or all human instructors and teammates are unavailable. The paper describes our learning environment, the issues that arise in developing virtual humans for team training, and our design for the virtual humans, which is an extension of our Steve agent previously used for one-on-one tutoring.
2000
Gratch, Jonathan
Socially Situated Planning Book Section
In: Socially Intelligent Agents, Multiagent Systems, Artificial Societies, and Simulated Organizations, vol. 3, pp. 181–188, AAAI Fall Symposium on Socially Intelligent Agents - The Human in the Loop, North Falmouth, MA, 2000.
Abstract | Links | BibTeX | Tags: Virtual Humans
@incollection{gratch_socially_2000,
title = {Socially Situated Planning},
author = {Jonathan Gratch},
url = {http://ict.usc.edu/pubs/Socially%20Situated%20Planning.pdf},
year = {2000},
date = {2000-01-01},
booktitle = {Socially Intelligent Agents, Multiagent Systems, Artificial Societies, and Simulated Organizations},
volume = {3},
pages = {181–188},
address = {AAAI Fall Symposium on Socially Intelligent Agents - The Human in the Loop, North Falmouth, MA},
abstract = {Introduction: Virtual environments such as training simulators and video games do an impressive job at modeling the physical dynamics of synthetic worlds but fall short when modeling the social dynamics of anything but the most impoverished human encounters. Yet the social dimension is at least as important as good graphics for creating an engaging game or effective training tool. Commercial flight simulators accurately model the technical aspects of flight but many aviation disasters arise from social breakdowns: poor management skills in the cockpit, or the effects of stress and emotion. Perhaps the biggest consumer of simulation technology, the U.S. military, identifies unrealistic human and organizational behavior as a major limitation of existing simulation technology (NRC, 1998). And of course the entertainment industry has long recognized the importance of good character, emotional attachment and rich social interactions to "put butts in seats." This article describes a research effort to endow virtual training environments with richer models of social behavior. We have been developing autonomous and semi-autonomous software agents that plan and act while situated in a social network of other entities, human and synthetic (Hill et. al, 1997; Tambe, 1997; Gratch and Hill, 1999). My work has focused on making agents act in an organization and obey social constraints, coordinate their behavior, negotiate conflicts, but also obey their own self-interest and show a range of individual differences in their behavior and willingness to violate social norms, albeit within the relatively narrow context of a specific training exercise.},
keywords = {Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Introduction: Virtual environments such as training simulators and video games do an impressive job at modeling the physical dynamics of synthetic worlds but fall short when modeling the social dynamics of anything but the most impoverished human encounters. Yet the social dimension is at least as important as good graphics for creating an engaging game or effective training tool. Commercial flight simulators accurately model the technical aspects of flight but many aviation disasters arise from social breakdowns: poor management skills in the cockpit, or the effects of stress and emotion. Perhaps the biggest consumer of simulation technology, the U.S. military, identifies unrealistic human and organizational behavior as a major limitation of existing simulation technology (NRC, 1998). And of course the entertainment industry has long recognized the importance of good character, emotional attachment and rich social interactions to "put butts in seats." This article describes a research effort to endow virtual training environments with richer models of social behavior. We have been developing autonomous and semi-autonomous software agents that plan and act while situated in a social network of other entities, human and synthetic (Hill et. al, 1997; Tambe, 1997; Gratch and Hill, 1999). My work has focused on making agents act in an organization and obey social constraints, coordinate their behavior, negotiate conflicts, but also obey their own self-interest and show a range of individual differences in their behavior and willingness to violate social norms, albeit within the relatively narrow context of a specific training exercise.