Publications
Search
241.
Saito, Shunsuke; Hu, Liwen; Ma, Chongyang; Ibayashi, Hikaru; Luo, Linjie; Li, Hao
3D Hair Synthesis Using Volumetric Variational Autoencoders Proceedings Article
In: SIGGRAPH Asia 2018 Technical Papers on - SIGGRAPH Asia '18, pp. 1–12, ACM Press, Tokyo, Japan, 2018, ISBN: 978-1-4503-6008-1.
@inproceedings{saito_3d_2018,
title = {3D Hair Synthesis Using Volumetric Variational Autoencoders},
author = {Shunsuke Saito and Liwen Hu and Chongyang Ma and Hikaru Ibayashi and Linjie Luo and Hao Li},
url = {http://dl.acm.org/citation.cfm?doid=3272127.3275019},
doi = {10.1145/3272127.3275019},
isbn = {978-1-4503-6008-1},
year = {2018},
date = {2018-12-01},
booktitle = {SIGGRAPH Asia 2018 Technical Papers on - SIGGRAPH Asia '18},
pages = {1–12},
publisher = {ACM Press},
address = {Tokyo, Japan},
abstract = {Recent advances in single-view 3D hair digitization have made the creation of high-quality CG characters scalable and accessible to end-users, enabling new forms of personalized VR and gaming experiences. To handle the complexity and variety of hair structures, most cutting-edge techniques rely on the successful retrieval of a particular hair model from a comprehensive hair database. Not only are the aforementioned data-driven methods storage intensive, but they are also prone to failure for highly unconstrained input images, complicated hairstyles, and failed face detection. Instead of using a large collection of 3D hair models directly, we propose to represent the manifold of 3D hairstyles implicitly through a compact latent space of a volumetric variational autoencoder (VAE). This deep neural network is trained with volumetric orientation field representations of 3D hair models and can synthesize new hairstyles from a compressed code. To enable end-to-end 3D hair inference, we train an additional embedding network to predict the code in the VAE latent space from any input image. Strand-level hairstyles can then be generated from the predicted volumetric representation. Our fully automatic framework does not require any ad-hoc face fitting, intermediate classification and segmentation, or hairstyle database retrieval. Our hair synthesis approach is significantly more robust and can handle a much wider variation of hairstyles than state-of-the-art data-driven hair modeling techniques with challenging inputs, including photos that are low-resolution, overexposured, or contain extreme head poses. The storage requirements are minimal and a 3D hair model can be produced from an image in a second. Our evaluations also show that successful reconstructions are possible from highly stylized cartoon images, non-human subjects, and pictures taken from behind a person. Our approach is particularly well suited for continuous and plausible hair interpolation between very different hairstyles.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent advances in single-view 3D hair digitization have made the creation of high-quality CG characters scalable and accessible to end-users, enabling new forms of personalized VR and gaming experiences. To handle the complexity and variety of hair structures, most cutting-edge techniques rely on the successful retrieval of a particular hair model from a comprehensive hair database. Not only are the aforementioned data-driven methods storage intensive, but they are also prone to failure for highly unconstrained input images, complicated hairstyles, and failed face detection. Instead of using a large collection of 3D hair models directly, we propose to represent the manifold of 3D hairstyles implicitly through a compact latent space of a volumetric variational autoencoder (VAE). This deep neural network is trained with volumetric orientation field representations of 3D hair models and can synthesize new hairstyles from a compressed code. To enable end-to-end 3D hair inference, we train an additional embedding network to predict the code in the VAE latent space from any input image. Strand-level hairstyles can then be generated from the predicted volumetric representation. Our fully automatic framework does not require any ad-hoc face fitting, intermediate classification and segmentation, or hairstyle database retrieval. Our hair synthesis approach is significantly more robust and can handle a much wider variation of hairstyles than state-of-the-art data-driven hair modeling techniques with challenging inputs, including photos that are low-resolution, overexposured, or contain extreme head poses. The storage requirements are minimal and a 3D hair model can be produced from an image in a second. Our evaluations also show that successful reconstructions are possible from highly stylized cartoon images, non-human subjects, and pictures taken from behind a person. Our approach is particularly well suited for continuous and plausible hair interpolation between very different hairstyles.
242.
Wang, Ning; Schwartz, David; Lewine, Gabrielle; Shapiro, Ari; Feng, Andrew; Zhuang, Cindy
Addressing Sexist Attitudes on a College Campus through Virtual Role-Play with Digital Doppelgangers Proceedings Article
In: Proceedings of the 18th International Conference on Intelligent Virtual Agents - IVA '18, pp. 219–226, ACM Press, Sydney, NSW, Australia, 2018, ISBN: 978-1-4503-6013-5.
@inproceedings{wang_addressing_2018,
title = {Addressing Sexist Attitudes on a College Campus through Virtual Role-Play with Digital Doppelgangers},
author = {Ning Wang and David Schwartz and Gabrielle Lewine and Ari Shapiro and Andrew Feng and Cindy Zhuang},
url = {http://dl.acm.org/citation.cfm?doid=3267851.3267913},
doi = {10.1145/3267851.3267913},
isbn = {978-1-4503-6013-5},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of the 18th International Conference on Intelligent Virtual Agents - IVA '18},
pages = {219–226},
publisher = {ACM Press},
address = {Sydney, NSW, Australia},
abstract = {Digital doppelgangers are virtual humans that highly resemble the real self but behave independently. Digital doppelgangers possess great potential to serve as powerful models for behavioral change. An emerging technology, the Rapid Avatar Capture and Simulation (RACAS) system, enables low-cost and high-speed scanning of a human user and creation of a digital doppelganger that is a fully animatable virtual 3D model of the user. We designed a virtual role-playing game, DELTA, that implements a powerful cognitive dissonance-based paradigm for attitudinal and behavioral change, and integrated it with digital doppelgangers to influence a human user’s attitude towards sexism on college campuses. In this paper, we discuss the design and evaluation the RACAS system and the DELTA game-based environment. Results indicate the potential impact of the DELTA game-based environment in creating an immersive virtual experience for attitudinal change.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Digital doppelgangers are virtual humans that highly resemble the real self but behave independently. Digital doppelgangers possess great potential to serve as powerful models for behavioral change. An emerging technology, the Rapid Avatar Capture and Simulation (RACAS) system, enables low-cost and high-speed scanning of a human user and creation of a digital doppelganger that is a fully animatable virtual 3D model of the user. We designed a virtual role-playing game, DELTA, that implements a powerful cognitive dissonance-based paradigm for attitudinal and behavioral change, and integrated it with digital doppelgangers to influence a human user’s attitude towards sexism on college campuses. In this paper, we discuss the design and evaluation the RACAS system and the DELTA game-based environment. Results indicate the potential impact of the DELTA game-based environment in creating an immersive virtual experience for attitudinal change.
243.
Gordon, Carla; Georgila, Kallirroi; Choi, Hyungtak; Boberg, Jill; Traum, David
Evaluating Subjective Feedback for Internet of Things Dialogues Proceedings Article
In: Proceedings of the 22nd Workshop on the Semantics and Pragmatics of Dialogue, pp. 64–72, Aix-en-Provence, France, 2018.
@inproceedings{gordon_evaluating_2018,
title = {Evaluating Subjective Feedback for Internet of Things Dialogues},
author = {Carla Gordon and Kallirroi Georgila and Hyungtak Choi and Jill Boberg and David Traum},
url = {https://amubox.univ-amu.fr/s/6YcAg3TpLpfzGEn#pdfviewer},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of the 22nd Workshop on the Semantics and Pragmatics of Dialogue},
pages = {64–72},
address = {Aix-en-Provence, France},
abstract = {This paper discusses the process of determining which subjective features are seen as ideal in a dialogue system, and linking these features to objectively quantifiable behaviors. A corpus of simulated system-user dialogues in the Internet of Things domain was manually annotated with a set of system communicative and action responses, and crowd-sourced ratings and qualitative feedback of these dialogues were collected. This corpus of subjective feedback was analyzed, revealing that raters described top ranked dialogues as Intelligent, Natural, Pleasant, and as having Personality. Additionally, certain communicative and action responses were statistically more likely to be present in dialogues described as having these features. There was also found to be a lack of agreement among raters as to whether a direct communication style, or a conversational one was preferred, suggesting that future research and development should consider creating models for different communication styles.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper discusses the process of determining which subjective features are seen as ideal in a dialogue system, and linking these features to objectively quantifiable behaviors. A corpus of simulated system-user dialogues in the Internet of Things domain was manually annotated with a set of system communicative and action responses, and crowd-sourced ratings and qualitative feedback of these dialogues were collected. This corpus of subjective feedback was analyzed, revealing that raters described top ranked dialogues as Intelligent, Natural, Pleasant, and as having Personality. Additionally, certain communicative and action responses were statistically more likely to be present in dialogues described as having these features. There was also found to be a lack of agreement among raters as to whether a direct communication style, or a conversational one was preferred, suggesting that future research and development should consider creating models for different communication styles.
244.
Ustun, Volkan; Rosenbloom, Paul S; Sajjadi, Seyed; Nuttall, Jeremy
Controlling Synthetic Characters in Simulations: A Case for Cognitive Architectures and Sigma Proceedings Article
In: Proceedings of I/ITSEC 2018, National Training and Simulation Association, Orlando, FL, 2018.
@inproceedings{ustun_controlling_2018,
title = {Controlling Synthetic Characters in Simulations: A Case for Cognitive Architectures and Sigma},
author = {Volkan Ustun and Paul S Rosenbloom and Seyed Sajjadi and Jeremy Nuttall},
url = {http://bcf.usc.edu/ rosenblo/Pubs/Ustun_IITSEC2018_D.pdf},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of I/ITSEC 2018},
publisher = {National Training and Simulation Association},
address = {Orlando, FL},
abstract = {Simulations, along with other similar applications like virtual worlds and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Cognitive architectures, which are models of the fixed structure underlying intelligent behavior in both natural and artificial systems, provide a conceptually valid common basis, as evidenced by the current efforts towards a standard model of the mind, to generate human-like intelligent behavior for these synthetic characters. Developments in the field of artificial intelligence, mainly in probabilistic graphical models and neural networks, open up new opportunities for cognitive architectures to make the synthetic characters more autonomous and to enrich their behavior. Sigma (Σ) is a cognitive architecture and system that strives to combine what has been learned from four decades of independent work on symbolic cognitive architectures, probabilistic graphical models, and more recently neural models, under its graphical architecture hypothesis. Sigma leverages an extended form of factor graphs towards a uniform grand unification of not only traditional cognitive capabilities but also key non-cognitive aspects, creating unique opportunities for the construction of new kinds of cognitive models that possess a Theory-of-Mind and that are perceptual, autonomous, interactive, affective, and adaptive. In this paper, we will introduce Sigma along with its diverse capabilities and then use three distinct proof-of-concept Sigma models to highlight combinations of these capabilities: (1) Distributional reinforcement learning models in a simple OpenAI Gym problem; (2) A pair of adaptive and interactive agent models that demonstrate rule-based, probabilistic, and social reasoning in a physical security scenario instantiated within the SmartBody character animation platform; and (3) A knowledge-free exploration model in which an agent leverages only architectural appraisal variables, namely attention and curiosity, to locate an item while building up a map in a Unity environment.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Simulations, along with other similar applications like virtual worlds and video games, require computational models of intelligence that generate realistic and credible behavior for the participating synthetic characters. Cognitive architectures, which are models of the fixed structure underlying intelligent behavior in both natural and artificial systems, provide a conceptually valid common basis, as evidenced by the current efforts towards a standard model of the mind, to generate human-like intelligent behavior for these synthetic characters. Developments in the field of artificial intelligence, mainly in probabilistic graphical models and neural networks, open up new opportunities for cognitive architectures to make the synthetic characters more autonomous and to enrich their behavior. Sigma (Σ) is a cognitive architecture and system that strives to combine what has been learned from four decades of independent work on symbolic cognitive architectures, probabilistic graphical models, and more recently neural models, under its graphical architecture hypothesis. Sigma leverages an extended form of factor graphs towards a uniform grand unification of not only traditional cognitive capabilities but also key non-cognitive aspects, creating unique opportunities for the construction of new kinds of cognitive models that possess a Theory-of-Mind and that are perceptual, autonomous, interactive, affective, and adaptive. In this paper, we will introduce Sigma along with its diverse capabilities and then use three distinct proof-of-concept Sigma models to highlight combinations of these capabilities: (1) Distributional reinforcement learning models in a simple OpenAI Gym problem; (2) A pair of adaptive and interactive agent models that demonstrate rule-based, probabilistic, and social reasoning in a physical security scenario instantiated within the SmartBody character animation platform; and (3) A knowledge-free exploration model in which an agent leverages only architectural appraisal variables, namely attention and curiosity, to locate an item while building up a map in a Unity environment.
245.
Hoegen, Rens; Schalk, Job Van Der; Lucas, Gale; Gratch, Jonathan
The impact of agent facial mimicry on social behavior in a prisoner’s dilemma Proceedings Article
In: Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 275–280, ACM, Sydney, Australia, 2018, ISBN: 978-1-4503-6013-5.
@inproceedings{hoegen_impact_2018,
title = {The impact of agent facial mimicry on social behavior in a prisoner’s dilemma},
author = {Rens Hoegen and Job Van Der Schalk and Gale Lucas and Jonathan Gratch},
url = {https://dl.acm.org/citation.cfm?id=3267911},
doi = {10.1145/3267851.3267911},
isbn = {978-1-4503-6013-5},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of the 18th International Conference on Intelligent Virtual Agents},
pages = {275–280},
publisher = {ACM},
address = {Sydney, Australia},
abstract = {A long tradition of research suggests a relationship between emotional mimicry and pro-social behavior, but the nature of this relationship is unclear. Does mimicry cause rapport and cooperation, or merely reflect it? Virtual humans can provide unique insights into these social processes by allowing unprecedented levels of experimental control. In a 2 x 2 factorial design, we examined the impact of facial mimicry and counter-mimicry in the iterated prisoner’s dilemma. Participants played with an agent that copied their smiles and frowns or one that showed the opposite pattern – i.e., that frowned when they smiled. As people tend to smile more than frown, we independently manipulated the contingency of expressions to ensure any effects are due to mimicry alone, and not the overall positivity/negativity of the agent: i.e., participants saw either a reflection of their own expressions or saw the expressions shown to a previous participant. Results show that participants smiled significantly more when playing an agent that mimicked them. Results also show a complex association between smiling, feelings of rapport, and cooperation. We discuss the implications of these findings on virtual human systems and theories of cooperation.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
A long tradition of research suggests a relationship between emotional mimicry and pro-social behavior, but the nature of this relationship is unclear. Does mimicry cause rapport and cooperation, or merely reflect it? Virtual humans can provide unique insights into these social processes by allowing unprecedented levels of experimental control. In a 2 x 2 factorial design, we examined the impact of facial mimicry and counter-mimicry in the iterated prisoner’s dilemma. Participants played with an agent that copied their smiles and frowns or one that showed the opposite pattern – i.e., that frowned when they smiled. As people tend to smile more than frown, we independently manipulated the contingency of expressions to ensure any effects are due to mimicry alone, and not the overall positivity/negativity of the agent: i.e., participants saw either a reflection of their own expressions or saw the expressions shown to a previous participant. Results show that participants smiled significantly more when playing an agent that mimicked them. Results also show a complex association between smiling, feelings of rapport, and cooperation. We discuss the implications of these findings on virtual human systems and theories of cooperation.
246.
Lucas, Gale M; Kramer, Nicole; Peters, Clara; Taesch, Lisa-Sophie; Mell, Johnathan; Gratch, Jonathan
Effects of Perceived Agency and Message Tone in Responding to a Virtual Personal Trainer Proceedings Article
In: Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 247–254, ACM, Sydney, Australia, 2018, ISBN: 978-1-4503-6013-5.
@inproceedings{lucas_effects_2018,
title = {Effects of Perceived Agency and Message Tone in Responding to a Virtual Personal Trainer},
author = {Gale M Lucas and Nicole Kramer and Clara Peters and Lisa-Sophie Taesch and Johnathan Mell and Jonathan Gratch},
url = {https://dl.acm.org/citation.cfm?id=3267855},
doi = {10.1145/3267851.3267855},
isbn = {978-1-4503-6013-5},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of the 18th International Conference on Intelligent Virtual Agents},
pages = {247–254},
publisher = {ACM},
address = {Sydney, Australia},
abstract = {Research has demonstrated promising benefits of applying virtual trainers to promote physical fitness. The current study investigated the value of virtual agents in the context of personal fitness, compared to trainers with greater levels of perceived agency (avatar or live human). We also explored the possibility that the effectiveness of the virtual trainer might depend on the affective tone it uses when trying to motivate users. Accordingly, participants received either positively or negatively valenced motivational messages from a virtual human they believed to be either an agent or an avatar, or they received the messages from a human instructor via skype. Both self-report and physiological data were collected. Like in-person coaches, the live human trainer who used negatively valenced messages were well-regarded; however, when the agent or avatar used negatively valenced messages, participants responded more poorly than when they used positively valenced ones. Perceived agency also affected rapport: compared to the agent, users felt more rapport with the live human trainer or the avatar. Regardless of trainer type, they also felt more rapport - and said they put in more effort - with trainers that used positively valenced messages than those that used negatively valenced ones. However, in reality, they put in more physical effort (as measured by heart rate) when trainers employed the more negatively valenced affective tone. We discuss implications for human–computer interaction.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Research has demonstrated promising benefits of applying virtual trainers to promote physical fitness. The current study investigated the value of virtual agents in the context of personal fitness, compared to trainers with greater levels of perceived agency (avatar or live human). We also explored the possibility that the effectiveness of the virtual trainer might depend on the affective tone it uses when trying to motivate users. Accordingly, participants received either positively or negatively valenced motivational messages from a virtual human they believed to be either an agent or an avatar, or they received the messages from a human instructor via skype. Both self-report and physiological data were collected. Like in-person coaches, the live human trainer who used negatively valenced messages were well-regarded; however, when the agent or avatar used negatively valenced messages, participants responded more poorly than when they used positively valenced ones. Perceived agency also affected rapport: compared to the agent, users felt more rapport with the live human trainer or the avatar. Regardless of trainer type, they also felt more rapport - and said they put in more effort - with trainers that used positively valenced messages than those that used negatively valenced ones. However, in reality, they put in more physical effort (as measured by heart rate) when trainers employed the more negatively valenced affective tone. We discuss implications for human–computer interaction.
247.
Lucas, Gale M; Boberg, Jill; Traum, David; Artstein, Ron; Gratch, Jonathan; Gainer, Alesia; Johnson, Emmanuel; Leuski, Anton; Nakano, Mikio
Culture, Errors, and Rapport-building Dialogue in Social Agents Proceedings Article
In: Proceedings of the 18th International Conference on Intelligent Virtual Agents, pp. 51–58, ACM, Sydney, Australia, 2018, ISBN: 978-1-4503-6013-5.
@inproceedings{lucas_culture_2018,
title = {Culture, Errors, and Rapport-building Dialogue in Social Agents},
author = {Gale M Lucas and Jill Boberg and David Traum and Ron Artstein and Jonathan Gratch and Alesia Gainer and Emmanuel Johnson and Anton Leuski and Mikio Nakano},
url = {https://dl.acm.org/citation.cfm?id=3267887},
doi = {10.1145/3267851.3267887},
isbn = {978-1-4503-6013-5},
year = {2018},
date = {2018-11-01},
booktitle = {Proceedings of the 18th International Conference on Intelligent Virtual Agents},
pages = {51–58},
publisher = {ACM},
address = {Sydney, Australia},
abstract = {This work explores whether culture impacts the extent to which social dialogue can mitigate (or exacerbate) the loss of trust caused when agents make conversational errors. Our study uses an agent designed to persuade users to agree with its rankings on two tasks. Participants from the U.S. and Japan completed our study. We perform two manipulations: (1) The presence of conversational errors – the agent exhibited errors in the second task or not; (2) The presence of social dialogue – between the two tasks, users either engaged in a social dialogue with the agent or completed a control task. Replicating previous research, conversational errors reduce the agent’s influence. However, we found that culture matters: there was a marginally significant three-way interaction with culture, presence of social dialogue, and presence of errors. The pattern of results suggests that, for American participants, social dialogue backfired if it is followed by errors, presumably because it extends the period of good performance, creating a stronger contrast effect with the subsequent errors. However, for Japanese participants, social dialogue if anything mitigates the detrimental effect of errors; the negative effect of errors is only seen in the absence of a social dialogue. Agent design should therefore take the culture of the intended users into consideration when considering use of social dialogue to bolster agents against conversational errors.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
This work explores whether culture impacts the extent to which social dialogue can mitigate (or exacerbate) the loss of trust caused when agents make conversational errors. Our study uses an agent designed to persuade users to agree with its rankings on two tasks. Participants from the U.S. and Japan completed our study. We perform two manipulations: (1) The presence of conversational errors – the agent exhibited errors in the second task or not; (2) The presence of social dialogue – between the two tasks, users either engaged in a social dialogue with the agent or completed a control task. Replicating previous research, conversational errors reduce the agent’s influence. However, we found that culture matters: there was a marginally significant three-way interaction with culture, presence of social dialogue, and presence of errors. The pattern of results suggests that, for American participants, social dialogue backfired if it is followed by errors, presumably because it extends the period of good performance, creating a stronger contrast effect with the subsequent errors. However, for Japanese participants, social dialogue if anything mitigates the detrimental effect of errors; the negative effect of errors is only seen in the absence of a social dialogue. Agent design should therefore take the culture of the intended users into consideration when considering use of social dialogue to bolster agents against conversational errors.
248.
Marge, Matthew; Bonial, Claire; Lukin, Stephanie M.; Hayes, Cory J.; Foots, Ashley; Artstein, Ron; Henry, Cassidy; Pollard, Kimberly A.; Gordon, Carla; Gervits, Felix; Leuski, Anton; Hill, Susan G.; Voss, Clare R.; Traum, David
Balancing Efficiency and Coverage in Human-Robot Dialogue Collection Proceedings Article
In: Proceedings of the AAAI Fall Symposium on Interactive Learning in Artificial Intelligence for Human-Robot Interaction, arXiv, Arlington, Virginia, 2018.
@inproceedings{marge_balancing_2018,
title = {Balancing Efficiency and Coverage in Human-Robot Dialogue Collection},
author = {Matthew Marge and Claire Bonial and Stephanie M. Lukin and Cory J. Hayes and Ashley Foots and Ron Artstein and Cassidy Henry and Kimberly A. Pollard and Carla Gordon and Felix Gervits and Anton Leuski and Susan G. Hill and Clare R. Voss and David Traum},
url = {https://arxiv.org/abs/1810.02017},
year = {2018},
date = {2018-10-01},
booktitle = {Proceedings of the AAAI Fall Symposium on Interactive Learning in Artificial Intelligence for Human-Robot Interaction},
publisher = {arXiv},
address = {Arlington, Virginia},
abstract = {We describe a multi-phased Wizard-of-Oz approach to collecting human-robot dialogue in a collaborative search and navigation task. The data is being used to train an initial automated robot dialogue system to support collaborative exploration tasks. In the first phase, a wizard freely typed robot utterances to human participants. For the second phase, this data was used to design a GUI that includes buttons for the most common communications, and templates for communications with varying parameters. Comparison of the data gathered in these phases show that the GUI enabled a faster pace of dialogue while still maintaining high coverage of suitable responses, enabling more efficient targeted data collection, and improvements in natural language understanding using GUI-collected data. As a promising first step towardsinteractivelearning,thisworkshowsthatourapproach enables the collection of useful training data for navigationbased HRI tasks.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We describe a multi-phased Wizard-of-Oz approach to collecting human-robot dialogue in a collaborative search and navigation task. The data is being used to train an initial automated robot dialogue system to support collaborative exploration tasks. In the first phase, a wizard freely typed robot utterances to human participants. For the second phase, this data was used to design a GUI that includes buttons for the most common communications, and templates for communications with varying parameters. Comparison of the data gathered in these phases show that the GUI enabled a faster pace of dialogue while still maintaining high coverage of suitable responses, enabling more efficient targeted data collection, and improvements in natural language understanding using GUI-collected data. As a promising first step towardsinteractivelearning,thisworkshowsthatourapproach enables the collection of useful training data for navigationbased HRI tasks.
249.
Huang, Zeng; Li, Tianye; Chen, Weikai; Zhao, Yajie; Xing, Jun; LeGendre, Chloe; Luo, Linjie; Ma, Chongyang; Li, Hao
Deep Volumetric Video From Very Sparse Multi-View Performance Capture Proceedings Article
In: Proceedings of the 15th European Conference on Computer Vision, Computer Vision Foundation, Munich, Germany, 2018.
@inproceedings{huang_deep_2018,
title = {Deep Volumetric Video From Very Sparse Multi-View Performance Capture},
author = {Zeng Huang and Tianye Li and Weikai Chen and Yajie Zhao and Jun Xing and Chloe LeGendre and Linjie Luo and Chongyang Ma and Hao Li},
url = {http://openaccess.thecvf.com/content_ECCV_2018/papers/Zeng_Huang_Deep_Volumetric_Video_ECCV_2018_paper.pdf},
year = {2018},
date = {2018-09-01},
booktitle = {Proceedings of the 15th European Conference on Computer Vision},
publisher = {Computer Vision Foundation},
address = {Munich, Germany},
abstract = {We present a deep learning based volumetric approach for performance capture using a passive and highly sparse multi-view capture system. State-of-the-art performance capture systems require either prescanned actors, large number of cameras or active sensors. In this work, we focus on the task of template-free, per-frame 3D surface reconstruction from as few as three RGB sensors, for which conventional visual hull or multi-view stereo methods fail to generate plausible results. We introduce a novel multi-view Convolutional Neural Network (CNN) that maps 2D images to a 3D volumetric field and we use this field to encode the probabilistic distribution of surface points of the captured subject. By querying the resulting field, we can instantiate the clothed human body at arbitrary resolutions. Our approach scales to different numbers of input images, which yield increased reconstruction quality when more views are used. Although only trained on synthetic data, our network can generalize to handle real footage from body performance capture. Our method is suitable for high-quality low-cost full body volumetric capture solutions, which are gaining popularity for VR and AR content creation. Experimental results demonstrate that our method is significantly more robust and accurate than existing techniques when only very sparse views are available.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a deep learning based volumetric approach for performance capture using a passive and highly sparse multi-view capture system. State-of-the-art performance capture systems require either prescanned actors, large number of cameras or active sensors. In this work, we focus on the task of template-free, per-frame 3D surface reconstruction from as few as three RGB sensors, for which conventional visual hull or multi-view stereo methods fail to generate plausible results. We introduce a novel multi-view Convolutional Neural Network (CNN) that maps 2D images to a 3D volumetric field and we use this field to encode the probabilistic distribution of surface points of the captured subject. By querying the resulting field, we can instantiate the clothed human body at arbitrary resolutions. Our approach scales to different numbers of input images, which yield increased reconstruction quality when more views are used. Although only trained on synthetic data, our network can generalize to handle real footage from body performance capture. Our method is suitable for high-quality low-cost full body volumetric capture solutions, which are gaining popularity for VR and AR content creation. Experimental results demonstrate that our method is significantly more robust and accurate than existing techniques when only very sparse views are available.
250.
Zhou, Yi; Hu, Liwen; Xing, Jun; Chen, Weikai; Kung, Han-Wei; Tong, Xin; Li, Hao
HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks Proceedings Article
In: Proceedings of the 15th European Conference on Computer Vision, Computer Vision Foundation, Munich, Germany, 2018.
@inproceedings{zhou_hairnet_2018,
title = {HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks},
author = {Yi Zhou and Liwen Hu and Jun Xing and Weikai Chen and Han-Wei Kung and Xin Tong and Hao Li},
url = {http://openaccess.thecvf.com/content_ECCV_2018/papers/Yi_Zhou_Single-view_Hair_Reconstruction_ECCV_2018_paper.pdf},
year = {2018},
date = {2018-09-01},
booktitle = {Proceedings of the 15th European Conference on Computer Vision},
publisher = {Computer Vision Foundation},
address = {Munich, Germany},
abstract = {We introduce a deep learning-based method to generate full 3D hair geometry from an unconstrained image. Our method can recover local strand details and has real-time performance. State-of-the-art hair modeling techniques rely on large hairstyle collections for nearest neighbor retrieval and then perform ad-hoc refinement. Our deep learning approach, in contrast, is highly efficient in storage and can run 1000 times faster while generating hair with 30K strands. The convolutional neural network takes the 2D orientation field of a hair image as input and generates strand features that are evenly distributed on the parameterized 2D scalp. We introduce a collision loss to synthesize more plausible hairstyles, and the visibility of each strand is also used as a weight term to improve the reconstruction accuracy. The encoder-decoder architecture of our network naturally provides a compact and continuous representation for hairstyles, which allows us to interpolate naturally between hairstyles. We use a large set of rendered synthetic hair models to train our network. Our method scales to real images because an intermediate 2D orientation field, automatically calculated from the real image, factors out the difference between synthetic and real hairs. We demonstrate the effectiveness and robustness of our method on a wide range of challenging real Internet pictures, and show reconstructed hair sequences from videos.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We introduce a deep learning-based method to generate full 3D hair geometry from an unconstrained image. Our method can recover local strand details and has real-time performance. State-of-the-art hair modeling techniques rely on large hairstyle collections for nearest neighbor retrieval and then perform ad-hoc refinement. Our deep learning approach, in contrast, is highly efficient in storage and can run 1000 times faster while generating hair with 30K strands. The convolutional neural network takes the 2D orientation field of a hair image as input and generates strand features that are evenly distributed on the parameterized 2D scalp. We introduce a collision loss to synthesize more plausible hairstyles, and the visibility of each strand is also used as a weight term to improve the reconstruction accuracy. The encoder-decoder architecture of our network naturally provides a compact and continuous representation for hairstyles, which allows us to interpolate naturally between hairstyles. We use a large set of rendered synthetic hair models to train our network. Our method scales to real images because an intermediate 2D orientation field, automatically calculated from the real image, factors out the difference between synthetic and real hairs. We demonstrate the effectiveness and robustness of our method on a wide range of challenging real Internet pictures, and show reconstructed hair sequences from videos.
251.
Song, Yuhang; Yang, Chao; Lin, Zhe; Liu, Xiaofeng; Li, Hao; Huang, Qin
Contextual Based Image Inpainting: Infer, Match and Translate Proceedings Article
In: Proceedings of the 15th European Conference on Computer Vision, Computer Vision Foundation, Munich, Germany, 2018.
@inproceedings{song_contextual_2018,
title = {Contextual Based Image Inpainting: Infer, Match and Translate},
author = {Yuhang Song and Chao Yang and Zhe Lin and Xiaofeng Liu and Hao Li and Qin Huang},
url = {http://openaccess.thecvf.com/content_ECCV_2018/papers/Yuhang_Song_Contextual_Based_Image_ECCV_2018_paper.pdf},
year = {2018},
date = {2018-09-01},
booktitle = {Proceedings of the 15th European Conference on Computer Vision},
publisher = {Computer Vision Foundation},
address = {Munich, Germany},
abstract = {We study the task of image inpainting, which is to fill in the missing region of an incomplete image with plausible contents. To this end, we propose a learning-based approach to generate visually coherent completion given a high-resolution image with missing components. In order to overcome the difficulty to directly learn the distribution of highdimensional image data, we divide the task into inference and translation as two separate steps and model each step with a deep neural network. We also use simple heuristics to guide the propagation of local textures from the boundary to the hole. We show that, by using such techniques, inpainting reduces to the problem of learning two image-feature translation functions in much smaller space and hence easier to train. We evaluate our method on several public datasets and show that we generate results of better visual quality than previous state-of-the-art methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We study the task of image inpainting, which is to fill in the missing region of an incomplete image with plausible contents. To this end, we propose a learning-based approach to generate visually coherent completion given a high-resolution image with missing components. In order to overcome the difficulty to directly learn the distribution of highdimensional image data, we divide the task into inference and translation as two separate steps and model each step with a deep neural network. We also use simple heuristics to guide the propagation of local textures from the boundary to the hole. We show that, by using such techniques, inpainting reduces to the problem of learning two image-feature translation functions in much smaller space and hence easier to train. We evaluate our method on several public datasets and show that we generate results of better visual quality than previous state-of-the-art methods.
252.
Zheng, Zerong; Yu, Tao; Li, Hao; Guo, Kaiwen; Dai, Qionghai; Fang, Lu; Liu, Yebin
HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs Proceedings Article
In: Proceedings of the 15th European Conference on Computer Vision, Computer Vision Foundation, Munich, Germany, 2018.
@inproceedings{zheng_hybridfusion_2018,
title = {HybridFusion: Real-Time Performance Capture Using a Single Depth Sensor and Sparse IMUs},
author = {Zerong Zheng and Tao Yu and Hao Li and Kaiwen Guo and Qionghai Dai and Lu Fang and Yebin Liu},
url = {http://openaccess.thecvf.com/content_ECCV_2018/papers/Zerong_Zheng_HybridFusion_Real-Time_Performance_ECCV_2018_paper.pdf},
year = {2018},
date = {2018-09-01},
booktitle = {Proceedings of the 15th European Conference on Computer Vision},
publisher = {Computer Vision Foundation},
address = {Munich, Germany},
abstract = {We propose a light-weight yet highly robust method for realtime human performance capture based on a single depth camera and sparse inertial measurement units (IMUs). Our method combines nonrigid surface tracking and volumetric fusion to simultaneously reconstruct challenging motions, detailed geometries and the inner human body of a clothed subject. The proposed hybrid motion tracking algorithm and efficient per-frame sensor calibration technique enable nonrigid surface reconstruction for fast motions and challenging poses with severe occlusions. Significant fusion artifacts are reduced using a new confidence measurement for our adaptive TSDF-based fusion. The above contributions are mutually beneficial in our reconstruction system, which enable practical human performance capture that is real-time, robust, low-cost and easy to deploy. Experiments show that extremely challenging performances and loop closure problems can be handled successfully.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We propose a light-weight yet highly robust method for realtime human performance capture based on a single depth camera and sparse inertial measurement units (IMUs). Our method combines nonrigid surface tracking and volumetric fusion to simultaneously reconstruct challenging motions, detailed geometries and the inner human body of a clothed subject. The proposed hybrid motion tracking algorithm and efficient per-frame sensor calibration technique enable nonrigid surface reconstruction for fast motions and challenging poses with severe occlusions. Significant fusion artifacts are reduced using a new confidence measurement for our adaptive TSDF-based fusion. The above contributions are mutually beneficial in our reconstruction system, which enable practical human performance capture that is real-time, robust, low-cost and easy to deploy. Experiments show that extremely challenging performances and loop closure problems can be handled successfully.
253.
Wei, Lingyu; Hu, Liwen; Kim, Vladimir; Yumer, Ersin; Li, Hao
Real-Time Hair Rendering using Sequential Adversarial Networks Proceedings Article
In: Proceedings of the 15th European Conference on Computer Vision, Computer Vision Foundation, Munich, Germany, 2018.
@inproceedings{wei_real-time_2018,
title = {Real-Time Hair Rendering using Sequential Adversarial Networks},
author = {Lingyu Wei and Liwen Hu and Vladimir Kim and Ersin Yumer and Hao Li},
url = {http://openaccess.thecvf.com/content_ECCV_2018/papers/Lingyu_Wei_Real-Time_Hair_Rendering_ECCV_2018_paper.pdf},
year = {2018},
date = {2018-09-01},
booktitle = {Proceedings of the 15th European Conference on Computer Vision},
publisher = {Computer Vision Foundation},
address = {Munich, Germany},
abstract = {We present an adversarial network for rendering photorealistic hair as an alternative to conventional computer graphics pipelines. Our deep learning approach does not require low-level parameter tuning nor ad-hoc asset design. Our method simply takes a strand-based 3D hair model as input and provides intuitive user-control for color and lighting through reference images. To handle the diversity of hairstyles and its appearance complexity, we disentangle hair structure, color, and illumination properties using a sequential GAN architecture and a semisupervised training approach. We also introduce an intermediate edge activation map to orientation field conversion step to ensure a successful CG-to-photoreal transition, while preserving the hair structures of the original input data. As we only require a feed-forward pass through the network, our rendering performs in real-time. We demonstrate the synthesis of photorealistic hair images on a wide range of intricate hairstyles and compare our technique with state-of-the-art hair rendering methods.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present an adversarial network for rendering photorealistic hair as an alternative to conventional computer graphics pipelines. Our deep learning approach does not require low-level parameter tuning nor ad-hoc asset design. Our method simply takes a strand-based 3D hair model as input and provides intuitive user-control for color and lighting through reference images. To handle the diversity of hairstyles and its appearance complexity, we disentangle hair structure, color, and illumination properties using a sequential GAN architecture and a semisupervised training approach. We also introduce an intermediate edge activation map to orientation field conversion step to ensure a successful CG-to-photoreal transition, while preserving the hair structures of the original input data. As we only require a feed-forward pass through the network, our rendering performs in real-time. We demonstrate the synthesis of photorealistic hair images on a wide range of intricate hairstyles and compare our technique with state-of-the-art hair rendering methods.
254.
Chen, Meida; Koc, Eyuphan; Shi, Zhuoya; Soibelman, Lucio
Proactive 2D model-based scan planning for existing buildings Journal Article
In: Automation in Construction, vol. 93, pp. 165–177, 2018, ISSN: 09265805.
@article{chen_proactive_2018,
title = {Proactive 2D model-based scan planning for existing buildings},
author = {Meida Chen and Eyuphan Koc and Zhuoya Shi and Lucio Soibelman},
url = {https://linkinghub.elsevier.com/retrieve/pii/S0926580517310385},
doi = {10.1016/j.autcon.2018.05.010},
issn = {09265805},
year = {2018},
date = {2018-09-01},
journal = {Automation in Construction},
volume = {93},
pages = {165–177},
abstract = {Creating a building information model (BIM) is known to be valuable during the life-cycle of a building. In most cases, a BIM of an existing building either does not exist or is out of date. For existing buildings, an as-is BIM is needed to leverage the technology towards building life-cycle objectives. To create an as-is BIM, field surveying is a necessary task in collecting current building related information. Terrestrial laser scanners have been widely accepted as field surveying instruments due to their high level of accuracy. However, laser scanning is a timeconsuming and labor-intensive process. Site revisiting and reworking of the scanning process is generally unavoidable because ofinappropriate datacollection processes. In thiscontext, creatinga scanplan beforegoing to a job-site can improve the data collection process. In this study, the authors have proposed a 2D proactive scanplanning frameworkthatincludesthreemodules: aninformation-gathering module,apreparation module,anda searching module. In addition, three search algorithms — a greedy best-first search algorithm, a greedy search algorithm with a backtracking process, and a simulated annealing algorithm — were compared based on 64 actual building site drawings to identify strength and limitations. The experimental results demonstrate that the greedy search algorithm with a backtracking process could be used to compute an initial scan plan and the simulated annealing algorithm couldbe used tofurther refinethe initial scanplan. This paperwill alsointroduce the results of a case study that deployed the proposed scan-planning framework. In the case study, the resulting 3D-point cloud that was generated based on the proposed framework was compared with the 3D point cloud created with data collected through a planned scanning process performed by a scan technician.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Creating a building information model (BIM) is known to be valuable during the life-cycle of a building. In most cases, a BIM of an existing building either does not exist or is out of date. For existing buildings, an as-is BIM is needed to leverage the technology towards building life-cycle objectives. To create an as-is BIM, field surveying is a necessary task in collecting current building related information. Terrestrial laser scanners have been widely accepted as field surveying instruments due to their high level of accuracy. However, laser scanning is a timeconsuming and labor-intensive process. Site revisiting and reworking of the scanning process is generally unavoidable because ofinappropriate datacollection processes. In thiscontext, creatinga scanplan beforegoing to a job-site can improve the data collection process. In this study, the authors have proposed a 2D proactive scanplanning frameworkthatincludesthreemodules: aninformation-gathering module,apreparation module,anda searching module. In addition, three search algorithms — a greedy best-first search algorithm, a greedy search algorithm with a backtracking process, and a simulated annealing algorithm — were compared based on 64 actual building site drawings to identify strength and limitations. The experimental results demonstrate that the greedy search algorithm with a backtracking process could be used to compute an initial scan plan and the simulated annealing algorithm couldbe used tofurther refinethe initial scanplan. This paperwill alsointroduce the results of a case study that deployed the proposed scan-planning framework. In the case study, the resulting 3D-point cloud that was generated based on the proposed framework was compared with the 3D point cloud created with data collected through a planned scanning process performed by a scan technician.
255.
Larue, Othalia; West, Robert; Rosenbloom, Paul S.; Dancy, Christopher L.; Samsonovich, Alexei V.; Petters, Dean; Juvina, Ion
Emotion in the Common Model of Cognition Journal Article
In: Procedia Computer Science, vol. 145, pp. 740–746, 2018, ISSN: 18770509.
@article{larue_emotion_2018,
title = {Emotion in the Common Model of Cognition},
author = {Othalia Larue and Robert West and Paul S. Rosenbloom and Christopher L. Dancy and Alexei V. Samsonovich and Dean Petters and Ion Juvina},
url = {https://linkinghub.elsevier.com/retrieve/pii/S1877050918323317},
doi = {10.1016/j.procs.2018.11.045},
issn = {18770509},
year = {2018},
date = {2018-08-01},
journal = {Procedia Computer Science},
volume = {145},
pages = {740–746},
abstract = {Emotions play an important role in human cognition and therefore need to be present in the Common Model of Cognition. In this paper, the emotion working group focuses on functional aspects of emotions and describes what we believe are the points of interactions with the Common Model of Cognition. The present paper should not be viewed as a consensus of the group but rather as a first attempt to extract common and divergent aspects of different models of emotions and how they relate to the Common Model of Cognition.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Emotions play an important role in human cognition and therefore need to be present in the Common Model of Cognition. In this paper, the emotion working group focuses on functional aspects of emotions and describes what we believe are the points of interactions with the Common Model of Cognition. The present paper should not be viewed as a consensus of the group but rather as a first attempt to extract common and divergent aspects of different models of emotions and how they relate to the Common Model of Cognition.
256.
Kralik, Jerald D.; Lee, Jee Hang; Rosenbloom, Paul S.; Jackson, Philip C.; Epstein, Susan L.; Romero, Oscar J.; Sanz, Ricardo; Larue, Othalia; Schmidtke, Hedda R.; Lee, Sang Wan; McGreggor, Keith
Metacognition for a Common Model of Cognition Journal Article
In: Procedia Computer Science, vol. 145, pp. 730–739, 2018, ISSN: 18770509.
@article{kralik_metacognition_2018,
title = {Metacognition for a Common Model of Cognition},
author = {Jerald D. Kralik and Jee Hang Lee and Paul S. Rosenbloom and Philip C. Jackson and Susan L. Epstein and Oscar J. Romero and Ricardo Sanz and Othalia Larue and Hedda R. Schmidtke and Sang Wan Lee and Keith McGreggor},
url = {https://linkinghub.elsevier.com/retrieve/pii/S1877050918323329},
doi = {10.1016/j.procs.2018.11.046},
issn = {18770509},
year = {2018},
date = {2018-08-01},
journal = {Procedia Computer Science},
volume = {145},
pages = {730–739},
abstract = {This paper provides a starting point for the development of metacognition in a common model of cognition. It identifies significant theoretical work on metacognition from multiple disciplines that the authors believe worthy of consideration. After first defining cognition and metacognition, we outline three general categories of metacognition, provide an initial list of its main components, consider the more difficult problem of consciousness, and present examples of prominent artificial systems that have implemented metacognitive components. Finally, we identify pressing design issues for the future},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
This paper provides a starting point for the development of metacognition in a common model of cognition. It identifies significant theoretical work on metacognition from multiple disciplines that the authors believe worthy of consideration. After first defining cognition and metacognition, we outline three general categories of metacognition, provide an initial list of its main components, consider the more difficult problem of consciousness, and present examples of prominent artificial systems that have implemented metacognitive components. Finally, we identify pressing design issues for the future
257.
Goldberg, Benjamin; Nye, Benjamin; Lane, H Chad; Guadagnoli, Mark
Team Assessment and Pedagogy as Informed by Sports Coaching and Assessment Book Section
In: Design Recommendations for Intelligent Tutoring Systems: Volume 6-Team Modeling, pp. 105–119, US Army Research Laboratory (ARL), Orlando, Florida, 2018, ISBN: 978-0-9977257-4-2.
@incollection{goldberg_team_2018,
title = {Team Assessment and Pedagogy as Informed by Sports Coaching and Assessment},
author = {Benjamin Goldberg and Benjamin Nye and H Chad Lane and Mark Guadagnoli},
url = {https://gifttutoring.org/attachments/download/3029/Design%20Recommendations%20for%20ITS_Volume%206%20-%20Team%20Tutoring_final.pdf},
isbn = {978-0-9977257-4-2},
year = {2018},
date = {2018-08-01},
booktitle = {Design Recommendations for Intelligent Tutoring Systems: Volume 6-Team Modeling},
pages = {105–119},
publisher = {US Army Research Laboratory (ARL)},
address = {Orlando, Florida},
abstract = {In this chapter, we consider pedagogical insights offered by three different sources of information from sports coaching and assessment: published reports of sports training, first-hand accounts of team training, and a review of assessment approaches for measuring team performance. These issues are considered in the context of an integrated taxonomy of feedback that considers when feedback was given, who it was given to (e.g., individual vs. team), the type of feedback (e.g., positive vs. negative), and the specificity of feedback (e.g., detailed issues vs. brief note). The goal of this work is to consider how these patterns might generalize to a wider range of learning tasks, to improve both learning and assessment of team performance.},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
In this chapter, we consider pedagogical insights offered by three different sources of information from sports coaching and assessment: published reports of sports training, first-hand accounts of team training, and a review of assessment approaches for measuring team performance. These issues are considered in the context of an integrated taxonomy of feedback that considers when feedback was given, who it was given to (e.g., individual vs. team), the type of feedback (e.g., positive vs. negative), and the specificity of feedback (e.g., detailed issues vs. brief note). The goal of this work is to consider how these patterns might generalize to a wider range of learning tasks, to improve both learning and assessment of team performance.
258.
Yamaguchi, Shuco; Saito, Shunsuke; Nagano, Koki; Zhao, Yajie; Chen, Weikai; Olszewski, Kyle; Morishima, Shigeo; Li, Hao
High-fidelity facial reflectance and geometry inference from an unconstrained image Journal Article
In: ACM Transactions on Graphics, vol. 37, no. 4, pp. 1–14, 2018, ISSN: 07300301.
@article{yamaguchi_high-fidelity_2018,
title = {High-fidelity facial reflectance and geometry inference from an unconstrained image},
author = {Shuco Yamaguchi and Shunsuke Saito and Koki Nagano and Yajie Zhao and Weikai Chen and Kyle Olszewski and Shigeo Morishima and Hao Li},
url = {http://dl.acm.org/citation.cfm?doid=3197517.3201364},
doi = {10.1145/3197517.3201364},
issn = {07300301},
year = {2018},
date = {2018-08-01},
journal = {ACM Transactions on Graphics},
volume = {37},
number = {4},
pages = {1–14},
abstract = {We present a deep learning-based technique to infer high-quality facial reflectance and geometry given a single unconstrained image of the subject, which may contain partial occlusions and arbitrary illumination conditions. The reconstructed high-resolution textures, which are generated in only a few seconds, include high-resolution skin surface reflectance maps, representing both the diffuse and specular albedo, and medium- and high-frequency displacement maps, thereby allowing us to render compelling digital avatars under novel lighting conditions. To extract this data, we train our deep neural networks with a high-quality skin reflectance and geometry database created with a state-of-the-art multi-view photometric stereo system using polarized gradient illumination. Given the raw facial texture map extracted from the input image, our neural networks synthesize complete reflectance and displacement maps, as well as complete missing regions caused by occlusions. The completed textures exhibit consistent quality throughout the face due to our network architecture, which propagates texture features from the visible region, resulting in high-fidelity details that are consistent with those seen in visible regions. We describe how this highly underconstrained problem is made tractable by dividing the full inference into smaller tasks, which are addressed by dedicated neural networks. We demonstrate the effectiveness of our network design with robust texture completion from images of faces that are largely occluded. With the inferred reflectance and geometry data, we demonstrate the rendering of high-fidelity 3D avatars from a variety of subjects captured under different lighting conditions. In addition, we perform evaluations demonstrating that our method can infer plausible facial reflectance and geometric details comparable to those obtained from high-end capture devices, and outperform alternative approaches that require only a single unconstrained input image.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a deep learning-based technique to infer high-quality facial reflectance and geometry given a single unconstrained image of the subject, which may contain partial occlusions and arbitrary illumination conditions. The reconstructed high-resolution textures, which are generated in only a few seconds, include high-resolution skin surface reflectance maps, representing both the diffuse and specular albedo, and medium- and high-frequency displacement maps, thereby allowing us to render compelling digital avatars under novel lighting conditions. To extract this data, we train our deep neural networks with a high-quality skin reflectance and geometry database created with a state-of-the-art multi-view photometric stereo system using polarized gradient illumination. Given the raw facial texture map extracted from the input image, our neural networks synthesize complete reflectance and displacement maps, as well as complete missing regions caused by occlusions. The completed textures exhibit consistent quality throughout the face due to our network architecture, which propagates texture features from the visible region, resulting in high-fidelity details that are consistent with those seen in visible regions. We describe how this highly underconstrained problem is made tractable by dividing the full inference into smaller tasks, which are addressed by dedicated neural networks. We demonstrate the effectiveness of our network design with robust texture completion from images of faces that are largely occluded. With the inferred reflectance and geometry data, we demonstrate the rendering of high-fidelity 3D avatars from a variety of subjects captured under different lighting conditions. In addition, we perform evaluations demonstrating that our method can infer plausible facial reflectance and geometric details comparable to those obtained from high-end capture devices, and outperform alternative approaches that require only a single unconstrained input image.
259.
Manuvinakurike, Ramesh; Brixey, Jacqueline; Bui, Trung; Chang, Walter; Artstein, Ron; Georgila, Kallirroi
DialEdit: Annotations for Spoken Conversational Image Editing Proceedings Article
In: Proceedings of the 14th Joint ACL - ISO Workshop on Interoperable Semantic Annotation, Association for Computational Linguistics, Santa Fe, New Mexico, 2018.
@inproceedings{manuvinakurike_dialedit_2018,
title = {DialEdit: Annotations for Spoken Conversational Image Editing},
author = {Ramesh Manuvinakurike and Jacqueline Brixey and Trung Bui and Walter Chang and Ron Artstein and Kallirroi Georgila},
url = {https://aclanthology.info/papers/W18-4701/w18-4701},
year = {2018},
date = {2018-08-01},
booktitle = {Proceedings of the 14th Joint ACL - ISO Workshop on Interoperable Semantic Annotation},
publisher = {Association for Computational Linguistics},
address = {Santa Fe, New Mexico},
abstract = {We present a spoken dialogue corpus and annotation scheme for conversational image editing, where people edit an image interactively through spoken language instructions. Our corpus contains spoken conversations between two human participants: users requesting changes to images and experts performing these modifications in real time. Our annotation scheme consists of 26 dialogue act labels covering instructions, requests, and feedback, together with actions and entities for the content of the edit requests. The corpus supports research and development in areas such as incremental intent recognition, visual reference resolution, image-grounded dialogue modeling, dialogue state tracking, and user modeling.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a spoken dialogue corpus and annotation scheme for conversational image editing, where people edit an image interactively through spoken language instructions. Our corpus contains spoken conversations between two human participants: users requesting changes to images and experts performing these modifications in real time. Our annotation scheme consists of 26 dialogue act labels covering instructions, requests, and feedback, together with actions and entities for the content of the edit requests. The corpus supports research and development in areas such as incremental intent recognition, visual reference resolution, image-grounded dialogue modeling, dialogue state tracking, and user modeling.
260.
Mell, Johnathan; Lucas, Gale M.; Gratch, Jonathan
Welcome to the Real World: How Agent Strategy Increases Human Willingness to Deceive Proceedings Article
In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pp. 1250–1257, International Foundation for Autonomous Agents and Multiagent Systems, Stockholm, Sweden, 2018.
@inproceedings{mell_welcome_2018,
title = {Welcome to the Real World: How Agent Strategy Increases Human Willingness to Deceive},
author = {Johnathan Mell and Gale M. Lucas and Jonathan Gratch},
url = {https://dl.acm.org/citation.cfm?id=3237884},
year = {2018},
date = {2018-07-01},
booktitle = {Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems},
pages = {1250–1257},
publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
address = {Stockholm, Sweden},
abstract = {Humans that negotiate through representatives often instruct those representatives to act in certain ways that align with both the client's goals and his or her social norms. However, which tactics and ethical norms humans endorse vary widely from person to person, and these endorsements may be easy to manipulate. This work presents the results of a study that demonstrates that humans that interact with an artificial agent may change what kinds of tactics and norms they endorse-often dramatically. Previous work has indicated that people that negotiate through artificial agent representatives may be more inclined to fairness than those people that negotiate directly. Our work qualifies that initial picture, demonstrating that subsequent experience may change this tendency toward fairness. By exposing human negotiators to tough, automated agents, we are able to shift the participant's willingness to deceive others and utilize "hard-ball" negotiation techniques. In short, what techniques people decide to endorse is dependent upon their context and experience. We examine the effects of interacting with four different types of automated agents, each with a unique strategy, and how this subsequently changes which strategies a human negotiator might later endorse. In the study, which was conducted on an online negotiation platform, four different types of automated agents negotiate with humans over the course of a 10-minute interaction. The agents differ in a 2x2 design according to agent strategy (tough vs. fair) and agent attitude (nice vs. nasty). These results show that in this multi-issue bargaining task, humans that interacted with a tough agent were more willing to endorse deceptive techniques when instructing their own representative. These kinds of techniques were endorsed even if the agent the human encountered did not use deception as part of its strategy. In contrast to some previous work, there was not a significant effect of agent attitude. These results indicate the power of allowing people to program agents that follow their instructions, but also indicate that these social norms and tactic endorsements may be mutable in the presence of real negotiation experience.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Humans that negotiate through representatives often instruct those representatives to act in certain ways that align with both the client's goals and his or her social norms. However, which tactics and ethical norms humans endorse vary widely from person to person, and these endorsements may be easy to manipulate. This work presents the results of a study that demonstrates that humans that interact with an artificial agent may change what kinds of tactics and norms they endorse-often dramatically. Previous work has indicated that people that negotiate through artificial agent representatives may be more inclined to fairness than those people that negotiate directly. Our work qualifies that initial picture, demonstrating that subsequent experience may change this tendency toward fairness. By exposing human negotiators to tough, automated agents, we are able to shift the participant's willingness to deceive others and utilize "hard-ball" negotiation techniques. In short, what techniques people decide to endorse is dependent upon their context and experience. We examine the effects of interacting with four different types of automated agents, each with a unique strategy, and how this subsequently changes which strategies a human negotiator might later endorse. In the study, which was conducted on an online negotiation platform, four different types of automated agents negotiate with humans over the course of a 10-minute interaction. The agents differ in a 2x2 design according to agent strategy (tough vs. fair) and agent attitude (nice vs. nasty). These results show that in this multi-issue bargaining task, humans that interacted with a tough agent were more willing to endorse deceptive techniques when instructing their own representative. These kinds of techniques were endorsed even if the agent the human encountered did not use deception as part of its strategy. In contrast to some previous work, there was not a significant effect of agent attitude. These results indicate the power of allowing people to program agents that follow their instructions, but also indicate that these social norms and tactic endorsements may be mutable in the presence of real negotiation experience.
Filter
2013
Wang, Ning; Pynadath, David V.; Marsella, Stacy C.
Subjective Perceptions in Wartime Negotiation Proceedings Article
In: International Conference on Affective Computing and Intelligent Interaction, pp. 540 –545, Geneva, Switzerland, 2013.
Abstract | Links | BibTeX | Tags: Social Simulation, UARC
@inproceedings{wang_subjective_2013,
title = {Subjective Perceptions in Wartime Negotiation},
author = {Ning Wang and David V. Pynadath and Stacy C. Marsella},
url = {http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6681486&tag=1},
doi = {10.1109/ACII.2013.95},
year = {2013},
date = {2013-09-01},
booktitle = {International Conference on Affective Computing and Intelligent Interaction},
pages = {540 –545},
address = {Geneva, Switzerland},
abstract = {The prevalence of negotiation in social interaction has motivated researchers to develop virtual agents that can understand, facilitate, teach and even carry out negotiations. While much of this research has analyzed how to maximize the objective outcome, there is a growing body of work demonstrating that subjective perceptions of the outcome also play a critical role in human negotiation behavior. People derive subjective value from not only the outcome, but also from the process by which they achieve that outcome, from their relationship with their negotiation partner, etc. The affective responses evoked by these subjective valuations can be very different from what would be evoked by the objective outcome alone. We investigate such subjective valuations within human-agent negotiation in four variations of a wartime negotiation game. We observe that the objective outcomes of these negotiations are not strongly correlated with the human negotiators’ subjective perceptions, as measured by the Subjective Value Index. We examine the game dynamics and agent behaviors to identify features that induce different subjective values in the participants. We thus are able to identify characteristics of the negotiation process and the agents’ behavior that most impact people’s subjective valuations in our wartime negotiation games.⬚},
keywords = {Social Simulation, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
The prevalence of negotiation in social interaction has motivated researchers to develop virtual agents that can understand, facilitate, teach and even carry out negotiations. While much of this research has analyzed how to maximize the objective outcome, there is a growing body of work demonstrating that subjective perceptions of the outcome also play a critical role in human negotiation behavior. People derive subjective value from not only the outcome, but also from the process by which they achieve that outcome, from their relationship with their negotiation partner, etc. The affective responses evoked by these subjective valuations can be very different from what would be evoked by the objective outcome alone. We investigate such subjective valuations within human-agent negotiation in four variations of a wartime negotiation game. We observe that the objective outcomes of these negotiations are not strongly correlated with the human negotiators’ subjective perceptions, as measured by the Subjective Value Index. We examine the game dynamics and agent behaviors to identify features that induce different subjective values in the participants. We thus are able to identify characteristics of the negotiation process and the agents’ behavior that most impact people’s subjective valuations in our wartime negotiation games.⬚
Bousmalis, Konstantinos; Zafeiriou, Stefanos; Morency, Louis–Philippe; Pantic, Maja; Ghahramani, Zoubin
Variational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures Proceedings Article
In: Hutchison, David; Kanade, Takeo; Kittler, Josef; Kleinberg, Jon M.; Mattern, Friedemann; Mitchell, John C.; Naor, Moni; Nierstrasz, Oscar; Rangan, C. Pandu; Steffen, Bernhard; Sudan, Madhu; Terzopoulos, Demetri; Tygar, Doug; Vardi, Moshe Y.; Weikum, Gerhard; Blockeel, Hendrik; Kersting, Kristian; Nijssen, Siegfried; Železný, Filip (Ed.): Machine Learning and Knowledge Discovery in Databases, pp. 531–547, Springer Berlin Heidelberg, Prague, Czech Republic, 2013, ISBN: 978-3-642-40990-5 978-3-642-40991-2.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{bousmalis_variational_2013,
title = {Variational Hidden Conditional Random Fields with Coupled Dirichlet Process Mixtures},
author = {Konstantinos Bousmalis and Stefanos Zafeiriou and Louis–Philippe Morency and Maja Pantic and Zoubin Ghahramani},
editor = {David Hutchison and Takeo Kanade and Josef Kittler and Jon M. Kleinberg and Friedemann Mattern and John C. Mitchell and Moni Naor and Oscar Nierstrasz and C. Pandu Rangan and Bernhard Steffen and Madhu Sudan and Demetri Terzopoulos and Doug Tygar and Moshe Y. Vardi and Gerhard Weikum and Hendrik Blockeel and Kristian Kersting and Siegfried Nijssen and Filip Železný},
url = {http://link.springer.com/10.1007/978-3-642-40991-2_34},
doi = {10.1007/978-3-642-40991-2_34},
isbn = {978-3-642-40990-5 978-3-642-40991-2},
year = {2013},
date = {2013-09-01},
booktitle = {Machine Learning and Knowledge Discovery in Databases},
volume = {8189},
pages = {531–547},
publisher = {Springer Berlin Heidelberg},
address = {Prague, Czech Republic},
abstract = {Hidden Conditional Random Fields (HCRFs) are discriminative latent variable models which have been shown to successfully learn the hidden structure of a given classification problem. An infinite HCRF is an HCRF with a countably infinite number of hidden states, which rids us not only of the necessity to specify a priori a fixed number of hidden states available but also of the problem of overfitting. Markov chain Monte Carlo (MCMC) sampling algorithms are often employed for inference in such models. However, convergence of such algorithm is rather difficult to verify, and as the complexity of the task at han increases, the computational cost of such algorithms often becomes prohibitive. These limitations can be overcome by variational techniques. In this paper, we present a generalized framework for infinite HCRF models, and a novel variational inference approach on a model based on coupled Dirichlet Process Mixtures, the HCRF–DPM. We show that the variational HCRF–DPM is able to converge to a correct number of represented hidden states, and performs as well as the best parametric HCRFs—chosen via cross-validation— for the difficult tasks of recognizing instances of agreement, disagreement, and pain in audiovisual sequences.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Hidden Conditional Random Fields (HCRFs) are discriminative latent variable models which have been shown to successfully learn the hidden structure of a given classification problem. An infinite HCRF is an HCRF with a countably infinite number of hidden states, which rids us not only of the necessity to specify a priori a fixed number of hidden states available but also of the problem of overfitting. Markov chain Monte Carlo (MCMC) sampling algorithms are often employed for inference in such models. However, convergence of such algorithm is rather difficult to verify, and as the complexity of the task at han increases, the computational cost of such algorithms often becomes prohibitive. These limitations can be overcome by variational techniques. In this paper, we present a generalized framework for infinite HCRF models, and a novel variational inference approach on a model based on coupled Dirichlet Process Mixtures, the HCRF–DPM. We show that the variational HCRF–DPM is able to converge to a correct number of represented hidden states, and performs as well as the best parametric HCRFs—chosen via cross-validation— for the difficult tasks of recognizing instances of agreement, disagreement, and pain in audiovisual sequences.
Pynadath, David V.; Wang, Ning; Marsella, Stacy C.
Computational Models of Human Behavior in Wartime Negotiations Proceedings Article
In: Annual Meeting of the Cognitive Science Society, Berlin, Germany, 2013.
Abstract | Links | BibTeX | Tags: Social Simulation, UARC
@inproceedings{pynadath_computational_2013,
title = {Computational Models of Human Behavior in Wartime Negotiations},
author = {David V. Pynadath and Ning Wang and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Computational%20Models%20of%20Human%20Behavior%20in%20Wartime%20Negotiations.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Annual Meeting of the Cognitive Science Society},
address = {Berlin, Germany},
abstract = {Political scientists are increasingly turning to game-theoretic models to understand and predict the behavior of national leaders in wartime scenarios, where two sides have the options of seeking resolution at either the bargaining table or on the battlefield. While the theoretical analyses of these models is suggestive of their ability to capture these scenarios, it is not clear to what degree human behavior conforms to such equilibrium-based expectations. We present the results of a study that placed people within two of these game models, playing against an intelligent agent. We consider several testable hypotheses drawn from the theoretical analyses and evaluate the degree to which the observed human decisionmaking conforms to those hypotheses.⬚⬚⬚⬚},
keywords = {Social Simulation, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Political scientists are increasingly turning to game-theoretic models to understand and predict the behavior of national leaders in wartime scenarios, where two sides have the options of seeking resolution at either the bargaining table or on the battlefield. While the theoretical analyses of these models is suggestive of their ability to capture these scenarios, it is not clear to what degree human behavior conforms to such equilibrium-based expectations. We present the results of a study that placed people within two of these game models, playing against an intelligent agent. We consider several testable hypotheses drawn from the theoretical analyses and evaluate the degree to which the observed human decisionmaking conforms to those hypotheses.⬚⬚⬚⬚
Scherer, Stefan; Stratou, Giota; Gratch, Jonathan; Morency, Louis-Philippe
Investigating Voice Quality as a Speaker-Independent Indicator of Depression and PTSD Proceedings Article
In: Annual Conference of the International Speech Communication Association (INTERSPEECH), Lyon, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{scherer_investigating_2013-1,
title = {Investigating Voice Quality as a Speaker-Independent Indicator of Depression and PTSD},
author = {Stefan Scherer and Giota Stratou and Jonathan Gratch and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Investigating%20Voice%20Quality%20as%20a%20Speaker-Independent%20Indicator%20of%20Depression%20and%20PTSD.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Annual Conference of the International Speech Communication Association (INTERSPEECH)},
address = {Lyon, France},
abstract = {We seek to investigate voice quality characteristics, in particular on a breathy to tense dimension, as an indicator for psychological distress, i.e. depression and post-traumatic stress disorder (PTSD), within semi-structured virtual human interviews. Our evaluation identifies significant differences between the voice quality of psychologically distressed participants and not-distressed participants within this limited corpus. We investigate the capability of automatic algorithms to classify psychologically distressed speech in speaker-independent experiments. Additionally, we examine the impact of the posed questions’ affective polarity, as motivated by findings in the literature on positive stimulus attenuation and negative stimulus potentiation in emotional reactivity of psychologically distressed participants. The experiments yield promising results using standard machine learning algorithms and solely four distinct features capturing the tenseness of the speaker’s voice⬚⬚⬚⬚⬚⬚⬚.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We seek to investigate voice quality characteristics, in particular on a breathy to tense dimension, as an indicator for psychological distress, i.e. depression and post-traumatic stress disorder (PTSD), within semi-structured virtual human interviews. Our evaluation identifies significant differences between the voice quality of psychologically distressed participants and not-distressed participants within this limited corpus. We investigate the capability of automatic algorithms to classify psychologically distressed speech in speaker-independent experiments. Additionally, we examine the impact of the posed questions’ affective polarity, as motivated by findings in the literature on positive stimulus attenuation and negative stimulus potentiation in emotional reactivity of psychologically distressed participants. The experiments yield promising results using standard machine learning algorithms and solely four distinct features capturing the tenseness of the speaker’s voice⬚⬚⬚⬚⬚⬚⬚.
Gratch, Jonathan; Hartholt, Arno; Dehghani, Morteza; Marsella, Stacy C.
Virtual Humans: A New Toolkit for Cognitive Science Research Proceedings Article
In: Cognitive Science, Berlin, Germany, 2013.
Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{gratch_virtual_2013,
title = {Virtual Humans: A New Toolkit for Cognitive Science Research},
author = {Jonathan Gratch and Arno Hartholt and Morteza Dehghani and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Virtual%20Humans-%20A%20New%20Toolkit%20for%20Cognitive%20Science%20Research.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Cognitive Science},
address = {Berlin, Germany},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Ito, Jonathan Y.; Marsella, Stacy C.
Context Dependent Utility: Modeling Decision Behavior Across Contexts Proceedings Article
In: Annual Meeting of the Cognitive Science Society, Berlin, Germany, 2013.
Abstract | Links | BibTeX | Tags: Social Simulation, UARC
@inproceedings{ito_context_2013,
title = {Context Dependent Utility: Modeling Decision Behavior Across Contexts},
author = {Jonathan Y. Ito and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Context%20Dependent%20Utility-%20Modeling%20Decision%20Behavior%20Across%20Contexts.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Annual Meeting of the Cognitive Science Society},
address = {Berlin, Germany},
abstract = {One significant challenge in creating accurate models of human decision behavior is accounting for the effect of context. Research shows that seemingly minor changes in the presentation of a decision can lead to drastic shifts in behavior; phenomena collectively referred to as framing effects. Previous work has developed Context Dependent Utility (CDU), a framework integrating Appraisal Theory with decision-theoretic principles. This work extends existing research by presenting a study exploring the behavioral predictions offered by CDU regarding the multidimensional effect of context on decision behavior. The present study finds support for the predictions of CDU regarding the impact of context on decisions: 1) as perceptions of pleasantness increase, decision behavior tends towards riskaversion; 2) as perceptions of goal-congruence increase, decision behavior tends towards risk-aversion; 3) as perceptions of controllability increase, i.e., perceptions that outcomes would have been primarily caused by the decision maker, behavior tends towards risk-seeking.⬚⬚⬚⬚⬚⬚⬚⬚⬚⬚},
keywords = {Social Simulation, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
One significant challenge in creating accurate models of human decision behavior is accounting for the effect of context. Research shows that seemingly minor changes in the presentation of a decision can lead to drastic shifts in behavior; phenomena collectively referred to as framing effects. Previous work has developed Context Dependent Utility (CDU), a framework integrating Appraisal Theory with decision-theoretic principles. This work extends existing research by presenting a study exploring the behavioral predictions offered by CDU regarding the multidimensional effect of context on decision behavior. The present study finds support for the predictions of CDU regarding the impact of context on decisions: 1) as perceptions of pleasantness increase, decision behavior tends towards riskaversion; 2) as perceptions of goal-congruence increase, decision behavior tends towards risk-aversion; 3) as perceptions of controllability increase, i.e., perceptions that outcomes would have been primarily caused by the decision maker, behavior tends towards risk-seeking.⬚⬚⬚⬚⬚⬚⬚⬚⬚⬚
Gandhe, Sudeep; Traum, David
Surface Text based Dialogue Models for Virtual Humans Proceedings Article
In: SIGDIAL, Metz, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{gandhe_surface_2013,
title = {Surface Text based Dialogue Models for Virtual Humans},
author = {Sudeep Gandhe and David Traum},
url = {http://ict.usc.edu/pubs/Surface%20Text%20based%20Dialogue%20Models%20for%20Virtual%20Humans.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {SIGDIAL},
address = {Metz, France},
abstract = {We present virtual human dialogue models which primarily operate on the surface text level and can be extended to incorporate additional information state annotations such as topics or results from simpler models. We compare these models with previously proposed models as well as two human-level upper baselines. The models are evaluated by collecting appropriateness judgments from human judges for responses generated for a set of fixed dialogue contexts. Our results show that the best performing models achieve close to human-level performance and require only surface text dialogue transcripts to train.⬚⬚⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We present virtual human dialogue models which primarily operate on the surface text level and can be extended to incorporate additional information state annotations such as topics or results from simpler models. We compare these models with previously proposed models as well as two human-level upper baselines. The models are evaluated by collecting appropriateness judgments from human judges for responses generated for a set of fixed dialogue contexts. Our results show that the best performing models achieve close to human-level performance and require only surface text dialogue transcripts to train.⬚⬚⬚⬚⬚⬚
Ozkan, Derya; Morency, Louis-Philippe
Prediction of Visual Backchannels in the Absence of Visual Context Using Mutual Influence Proceedings Article
In: International Conference on Intelligent Virtual Humans, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{ozkan_prediction_2013,
title = {Prediction of Visual Backchannels in the Absence of Visual Context Using Mutual Influence},
author = {Derya Ozkan and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Prediction%20of%20Visual%20Backchannels%20in%20the%20Absence%20of%20Visual%20Context%20Using%20Mutual%20In%EF%AC%82uence.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {International Conference on Intelligent Virtual Humans},
address = {Edinburgh, UK},
abstract = {Based on the phenomena of mutual influence between participants of a face-to-face conversation, we propose a context-based prediction approach for modeling visual backchannels. Our goal is to create intelligent virtual listeners with the ability of providing backchannel feedbacks, enabling natural and fluid interactions. In our proposed approach, we first anticipate the speaker behaviors, and then use this anticipated visual context to obtain more accurate listener backchannel moments. We model the mutual influence between speaker and listener gestures using a latent variable sequential model. We compared our approach with state-of-the-art prediction models on a publicly available dataset and showed importance of modeling the mutual influence between the speaker and the listener⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Based on the phenomena of mutual influence between participants of a face-to-face conversation, we propose a context-based prediction approach for modeling visual backchannels. Our goal is to create intelligent virtual listeners with the ability of providing backchannel feedbacks, enabling natural and fluid interactions. In our proposed approach, we first anticipate the speaker behaviors, and then use this anticipated visual context to obtain more accurate listener backchannel moments. We model the mutual influence between speaker and listener gestures using a latent variable sequential model. We compared our approach with state-of-the-art prediction models on a publicly available dataset and showed importance of modeling the mutual influence between the speaker and the listener⬚⬚⬚
Batrinca, Ligia; Stratou, Giota; Shapiro, Ari; Morency, Louis-Philippe; Scherer, Stefan
Cicero - Towards a Multimodal Virtual Audience Platform for Public Speaking Training Proceedings Article
In: International Conference on Intelligent Virtual Humans, pp. 116–128, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{batrinca_cicero_2013,
title = {Cicero - Towards a Multimodal Virtual Audience Platform for Public Speaking Training},
author = {Ligia Batrinca and Giota Stratou and Ari Shapiro and Louis-Philippe Morency and Stefan Scherer},
url = {http://ict.usc.edu/pubs/Cicero%20-%20Towards%20a%20Multimodal%20Virtual%20Audience%20Platform%20for%20Public%20Speaking%20Training.pdf},
doi = {10.1007/978-3-642-40415-3_10},
year = {2013},
date = {2013-08-01},
booktitle = {International Conference on Intelligent Virtual Humans},
pages = {116–128},
address = {Edinburgh, UK},
series = {Lecture Notes on Computer Science},
abstract = {Public speaking performances are not only characterized by the presentation of the content, but also by the presenters’ nonverbal behavior, such as gestures, tone of voice, vocal variety, and facial expressions. Within this work, we seek to identify automatic nonverbal behavior descriptors that correlate with expert-assessments of behaviors characteristic of good and bad public speaking performances. We present a novel multimodal corpus recorded with a virtual audience public speaking training platform. Lastly, we utilize the behavior descriptors to automatically approximate the overall assessment of the performance using support vector regression in a speaker-independent experiment and yield promising results approaching human performance⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Public speaking performances are not only characterized by the presentation of the content, but also by the presenters’ nonverbal behavior, such as gestures, tone of voice, vocal variety, and facial expressions. Within this work, we seek to identify automatic nonverbal behavior descriptors that correlate with expert-assessments of behaviors characteristic of good and bad public speaking performances. We present a novel multimodal corpus recorded with a virtual audience public speaking training platform. Lastly, we utilize the behavior descriptors to automatically approximate the overall assessment of the performance using support vector regression in a speaker-independent experiment and yield promising results approaching human performance⬚⬚⬚
Morbini, Fabrizio; Audhkhasi, Kartik; Sagae, Kenji; Artstein, Ron; Can, Dogan; Georgiou, Panayiotis G.; Narayanan, Shri; Leuski, Anton; Traum, David
Which ASR should I choose for my dialogue system? Proceedings Article
In: SIGDIAL, Metz, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{morbini_which_2013,
title = {Which ASR should I choose for my dialogue system?},
author = {Fabrizio Morbini and Kartik Audhkhasi and Kenji Sagae and Ron Artstein and Dogan Can and Panayiotis G. Georgiou and Shri Narayanan and Anton Leuski and David Traum},
url = {http://ict.usc.edu/pubs/Which%20ASR%20should%20I%20choose%20for%20my%20dialogue%20system.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {SIGDIAL},
address = {Metz, France},
abstract = {We present an analysis of several publicly available automatic speech recognizers (ASRs) in terms of their suitability for use in different types of dialogue systems. We focus in particular on cloud based ASRs that recently have become available to the community. We include features of ASR systems and desiderata and requirements for different dialogue systems, taking into account the genre of the system, type of user, and other features. We then present speech recognition results for six different dialogue systems. The most interesting result is that different ASR systems perform best on these different data sets. We also show that there is an improvement over a previous generation of recognizers on some of these data sets. We also investigate NLU performance on the ASR output, and explore the relationship between ASR and NLU performance.⬚⬚⬚⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We present an analysis of several publicly available automatic speech recognizers (ASRs) in terms of their suitability for use in different types of dialogue systems. We focus in particular on cloud based ASRs that recently have become available to the community. We include features of ASR systems and desiderata and requirements for different dialogue systems, taking into account the genre of the system, type of user, and other features. We then present speech recognition results for six different dialogue systems. The most interesting result is that different ASR systems perform best on these different data sets. We also show that there is an improvement over a previous generation of recognizers on some of these data sets. We also investigate NLU performance on the ASR output, and explore the relationship between ASR and NLU performance.⬚⬚⬚⬚⬚⬚⬚
Nouri, Elnaz; Park, Sunghyun; Scherer, Stefan; Gratch, Jonathan; Carnevale, Peter; Morency, Louis-Philippe; Traum, David
Prediction of Strategy and Outcome as Negotiation Unfolds by Using Basic Verbal and Behavioral Features Proceedings Article
In: Annual Conference of the International Speech Communication Association (INTERSPEECH), Lyon, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{nouri_prediction_2013-1,
title = {Prediction of Strategy and Outcome as Negotiation Unfolds by Using Basic Verbal and Behavioral Features},
author = {Elnaz Nouri and Sunghyun Park and Stefan Scherer and Jonathan Gratch and Peter Carnevale and Louis-Philippe Morency and David Traum},
url = {http://ict.usc.edu/pubs/Prediction%20of%20Strategy%20and%20Outcome%20as%20Negotiation%20Unfolds%20by%20Using%20Basic%20Verbal%20and%20Behavioral%20Features.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Annual Conference of the International Speech Communication Association (INTERSPEECH)},
address = {Lyon, France},
abstract = {Negotiations can be characterized by the strategy participants adopt to achieve their ends (e.g., individualistic strategies are based on self-interest, cooperative strategies are used when participants try to maximize the joint gain, while competitive strategies focus on maximizing each participant's score against the other) and the outcomes that each participant achieves in negotiation. This paper investigates the process and the result of predicting the outcome and strategy of participants throughout the progress of the negotiation by using basic, easy to extract, linguistic and acoustic features. We evaluate our apporach on a face-to-face negotiation dataset consisting of 41 dyadic intereactions and show that it's possible to significantly improve over a majority-class baseline in tasks of predicting the strategy and outcome of th einteraction by analyzing only basic low level features of the negotiation.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Negotiations can be characterized by the strategy participants adopt to achieve their ends (e.g., individualistic strategies are based on self-interest, cooperative strategies are used when participants try to maximize the joint gain, while competitive strategies focus on maximizing each participant's score against the other) and the outcomes that each participant achieves in negotiation. This paper investigates the process and the result of predicting the outcome and strategy of participants throughout the progress of the negotiation by using basic, easy to extract, linguistic and acoustic features. We evaluate our apporach on a face-to-face negotiation dataset consisting of 41 dyadic intereactions and show that it's possible to significantly improve over a majority-class baseline in tasks of predicting the strategy and outcome of th einteraction by analyzing only basic low level features of the negotiation.
Pynadath, David V.; Wang, Ning; Marsella, Stacy C.
Are you thinking what I'm thinking? An Evaluation of Simplified Theory of Mind Proceedings Article
In: International Conference on Intelligent Virtual Humans, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: Social Simulation, UARC
@inproceedings{pynadath_are_2013,
title = {Are you thinking what I'm thinking? An Evaluation of Simplified Theory of Mind},
author = {David V. Pynadath and Ning Wang and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Are%20you%20thinking%20what%20Im%20thinking%20An%20Evaluation%20of%20Simplified%20Theory%20of%20Mind.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {International Conference on Intelligent Virtual Humans},
address = {Edinburgh, UK},
abstract = {We examine the effectiveness of an agent's approximate theory of mind when interacting with human players in a wartime negotitation game. We first measure how accurately the agent's theory of mind captured the players' actual behavior. We observe significant overlap between the players' behavior and the agents' idealized expectations, but we also observe significant deviations. Forming an incorrect expectation about a person is not inherently damaging, so we then analyzed how different deviations affected the game outcomes. We observe that many classes of inaccuracy in the agent's theory of mind did not hurt the agent's performance and, in fact, some of them played to the agent's benefit. The results suggest potential advantages to giving an agent a computational model of theory of mind that is overly simplified, especially as a first step when investigating a domain with as much uncertainty as a wartime negotation.},
keywords = {Social Simulation, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
We examine the effectiveness of an agent's approximate theory of mind when interacting with human players in a wartime negotitation game. We first measure how accurately the agent's theory of mind captured the players' actual behavior. We observe significant overlap between the players' behavior and the agents' idealized expectations, but we also observe significant deviations. Forming an incorrect expectation about a person is not inherently damaging, so we then analyzed how different deviations affected the game outcomes. We observe that many classes of inaccuracy in the agent's theory of mind did not hurt the agent's performance and, in fact, some of them played to the agent's benefit. The results suggest potential advantages to giving an agent a computational model of theory of mind that is overly simplified, especially as a first step when investigating a domain with as much uncertainty as a wartime negotation.
Lhommet, Margaux; Marsella, Stacy C.
Gesture with Meaning Proceedings Article
In: International Conference on Intelligent Virtual Humans, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{lhommet_gesture_2013,
title = {Gesture with Meaning},
author = {Margaux Lhommet and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Gesture%20with%20Meaning.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {International Conference on Intelligent Virtual Humans},
address = {Edinburgh, UK},
abstract = {Embodied conversational agents (ECA) should exhibit nonverbal behaviors that are meaningfully related to their speech and mental state. This paper describes Cerebella, a system that automatically derives communicative functions from the text and audio of an utterance by combining lexical, acoustic, syntactic, semantic and rhetorical analyses. Communicative functions are then mapped to a multimodal behavior performance. Two studies demonstrate that the generated performances are meaningful and consistent with the speech.⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Embodied conversational agents (ECA) should exhibit nonverbal behaviors that are meaningfully related to their speech and mental state. This paper describes Cerebella, a system that automatically derives communicative functions from the text and audio of an utterance by combining lexical, acoustic, syntactic, semantic and rhetorical analyses. Communicative functions are then mapped to a multimodal behavior performance. Two studies demonstrate that the generated performances are meaningful and consistent with the speech.⬚⬚⬚
Perez-Rosas, Veronica; Mihalcea, Rada; Morency, Louis-Philippe
Utterance-Level Multimodal Sentiment Analysis Proceedings Article
In: Association for Computational Linguistics (ACL), Sofia, Bulgaria, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{perez-rosas_utterance-level_2013,
title = {Utterance-Level Multimodal Sentiment Analysis},
author = {Veronica Perez-Rosas and Rada Mihalcea and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Utterance-Level%20Multimodal%20Sentiment%20Analysis.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Association for Computational Linguistics (ACL)},
address = {Sofia, Bulgaria},
abstract = {During real-life interactions, people are naturally gesturing and modulating their voice to emphasize specific points or to express their emotions. With the recent growth of social websites such as YouTube, Facebook, and Amazon, video reviews are emerging as a new source of multimodal and natural opinions that has been left almost untapped by automatic opinion analysis techniques. This paper presents a method for multimodal sentiment classification, which can identify the sentiment expressed in utterance-level visual datastreams. Using a new multimodal dataset consisting of sentiment annotated utterances extracted from video reviews, we show that multimodal sentiment analysis can be effectively performed, and that the joint use of visual, acoustic, and linguistic modalities can lead to error rate reductions of up to 10.5% as compared to the best performing individual modality.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
During real-life interactions, people are naturally gesturing and modulating their voice to emphasize specific points or to express their emotions. With the recent growth of social websites such as YouTube, Facebook, and Amazon, video reviews are emerging as a new source of multimodal and natural opinions that has been left almost untapped by automatic opinion analysis techniques. This paper presents a method for multimodal sentiment classification, which can identify the sentiment expressed in utterance-level visual datastreams. Using a new multimodal dataset consisting of sentiment annotated utterances extracted from video reviews, we show that multimodal sentiment analysis can be effectively performed, and that the joint use of visual, acoustic, and linguistic modalities can lead to error rate reductions of up to 10.5% as compared to the best performing individual modality.
Hart, John; Gratch, Jonathan; Marsella, Stacy C.
How virtual reality training can win friends and influence people Book Section
In: Best, Christopher; Galanis, George; Kerry, James; Sottilare, Robert (Ed.): Fundamental Issues in Defense Training and Simulation, Ashgate, 2013, ISBN: 978-1-4094-4721-4.
Links | BibTeX | Tags: ARL, DoD, UARC, Virtual Humans
@incollection{hart_how_2013,
title = {How virtual reality training can win friends and influence people},
author = {John Hart and Jonathan Gratch and Stacy C. Marsella},
editor = {Christopher Best and George Galanis and James Kerry and Robert Sottilare},
url = {http://www.amazon.com/Fundamental-Defense-Training-Simulation-Factors-ebook/dp/B00EUE2F2I},
isbn = {978-1-4094-4721-4},
year = {2013},
date = {2013-08-01},
booktitle = {Fundamental Issues in Defense Training and Simulation},
publisher = {Ashgate},
series = {Human Factors in Defense},
keywords = {ARL, DoD, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {incollection}
}
Forbell, Eric; Kalisch, Nicolai; Morbini, Fabrizio; Christofferson, Kelly; Sagae, Kenji; Traum, David; Rizzo, Albert
Roundtable: An Online Framework for Building Web-based Conversational Agents Proceedings Article
In: Annual SIGdial Meeting on Discourse and Dialogue, Metz, France, 2013.
Abstract | Links | BibTeX | Tags: MedVR, UARC, Virtual Humans
@inproceedings{forbell_roundtable_2013,
title = {Roundtable: An Online Framework for Building Web-based Conversational Agents},
author = {Eric Forbell and Nicolai Kalisch and Fabrizio Morbini and Kelly Christofferson and Kenji Sagae and David Traum and Albert Rizzo},
url = {http://ict.usc.edu/pubs/Roundtable-%20An%20Online%20Framework%20for%20Building%20Web-based%20Conversational%20Agents.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Annual SIGdial Meeting on Discourse and Dialogue},
address = {Metz, France},
abstract = {We present an online system that provides a complete web-based sandbox for creating, testing and publishing embodied conversational agents. The tool, called Roundtable, empowers many different types of authors and varying team sizes to create flexible interactions by automating many editing workflows while limiting complexity and hiding architectural concerns. Finished characters can be published directly to web servers, enabling highly interactive applications.},
keywords = {MedVR, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We present an online system that provides a complete web-based sandbox for creating, testing and publishing embodied conversational agents. The tool, called Roundtable, empowers many different types of authors and varying team sizes to create flexible interactions by automating many editing workflows while limiting complexity and hiding architectural concerns. Finished characters can be published directly to web servers, enabling highly interactive applications.
Khooshabeh, Peter; Melo, Celso M.; Volkman, Brooks; Gratch, Jonathan; Blascovich, Jim; Carnevale, Peter
Negotiation Strategies with Incongruent Facial Expressions of Emotion Cause Cardiovascular Threat Proceedings Article
In: Cognitive Science, Berlin, Germany, 2013.
Abstract | Links | BibTeX | Tags: ARL, DoD, ICB, UARC, Virtual Humans
@inproceedings{khooshabeh_negotiation_2013,
title = {Negotiation Strategies with Incongruent Facial Expressions of Emotion Cause Cardiovascular Threat},
author = {Peter Khooshabeh and Celso M. Melo and Brooks Volkman and Jonathan Gratch and Jim Blascovich and Peter Carnevale},
url = {http://ict.usc.edu/pubs/Negotiation%20Strategies%20with%20Incongruent%20Facial%20Expressions%20of%20Emotion%20Cause%20Cardiovascular%20Threat.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {Cognitive Science},
address = {Berlin, Germany},
abstract = {Affect is important in motivated performance situations such as negotiation. Longstanding theories of emotion suggest that facial expressions provide enough information to perceive another person’s internal affective state. Alternatively, the contextual emotion hypothesis posits that situational factors bias the perception of emotion in others’ facial displays. This hypothesis predicts that individuals will have different perceptions of the same facial expression depending upon the context in which the expression is displayed. In this study, cardiovascular indexes of motivational states (i.e., challenge vs. threat) were recorded while players engaged in a multi-issue negotiation where the opposing negotiator (confederate) displayed emotional facial expressions (angry vs. happy); the confederate’s negotiation strategy (cooperative vs. competitive) was factorially crossed with his facial expression. During the game, participants’ eye fixations and cardiovascular responses, indexing task engagement and challenge/threat motivation, were recorded. Results indicated that participants playing confederates with incongruent facial expressions (e.g., cooperative strategy, angry face) exhibited a greater threat response, which arises due to increased uncertainty. Eye fixations also suggest that participants look at the face more in order to acquire information to reconcile their uncertainty in the incongruent condition. Taken together, these results suggest that context matters in the perception of emotion.},
keywords = {ARL, DoD, ICB, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Affect is important in motivated performance situations such as negotiation. Longstanding theories of emotion suggest that facial expressions provide enough information to perceive another person’s internal affective state. Alternatively, the contextual emotion hypothesis posits that situational factors bias the perception of emotion in others’ facial displays. This hypothesis predicts that individuals will have different perceptions of the same facial expression depending upon the context in which the expression is displayed. In this study, cardiovascular indexes of motivational states (i.e., challenge vs. threat) were recorded while players engaged in a multi-issue negotiation where the opposing negotiator (confederate) displayed emotional facial expressions (angry vs. happy); the confederate’s negotiation strategy (cooperative vs. competitive) was factorially crossed with his facial expression. During the game, participants’ eye fixations and cardiovascular responses, indexing task engagement and challenge/threat motivation, were recorded. Results indicated that participants playing confederates with incongruent facial expressions (e.g., cooperative strategy, angry face) exhibited a greater threat response, which arises due to increased uncertainty. Eye fixations also suggest that participants look at the face more in order to acquire information to reconcile their uncertainty in the incongruent condition. Taken together, these results suggest that context matters in the perception of emotion.
DeVault, David; Georgila, Kallirroi; Artstein, Ron; Morbini, Fabrizio; Traum, David; Scherer, Stefan; Rizzo, Albert; Morency, Louis-Philippe
Verbal indicators of psychological distress in interactive dialogue with a virtual human Proceedings Article
In: SIGDIAL, Metz, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{devault_verbal_2013,
title = {Verbal indicators of psychological distress in interactive dialogue with a virtual human},
author = {David DeVault and Kallirroi Georgila and Ron Artstein and Fabrizio Morbini and David Traum and Stefan Scherer and Albert Rizzo and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Verbal%20indicators%20of%20psychological%20distress%20in%20interactive%20dialogue%20with%20a%20virtual%20human.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {SIGDIAL},
address = {Metz, France},
abstract = {We explore the presence of indicators of psychological distress in the linguistic behavior of subjects in a corpus of semistructured virtual human interviews. At the level of aggregate dialogue-level features, we identify several significant differences between subjects with depression and PTSD when compared to nondistressed subjects. At a more fine-grained level, we show that significant differences can also be found among features that represent subject behavior during specific moments in the dialogues. Finally, we present statistical classification results that suggest the potential for automatic assessment of psychological distress in individual interactions with a virtual human dialogue system.⬚⬚⬚⬚⬚⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We explore the presence of indicators of psychological distress in the linguistic behavior of subjects in a corpus of semistructured virtual human interviews. At the level of aggregate dialogue-level features, we identify several significant differences between subjects with depression and PTSD when compared to nondistressed subjects. At a more fine-grained level, we show that significant differences can also be found among features that represent subject behavior during specific moments in the dialogues. Finally, we present statistical classification results that suggest the potential for automatic assessment of psychological distress in individual interactions with a virtual human dialogue system.⬚⬚⬚⬚⬚⬚⬚⬚⬚
Huang, Lixing; Gratch, Jonathan
Explaining the Variability of Human Nonverbal Behaviors in Face-to-Face Interaction Proceedings Article
In: 13th International Conference on Intelligent Virtual Humans, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{huang_explaining_2013,
title = {Explaining the Variability of Human Nonverbal Behaviors in Face-to-Face Interaction},
author = {Lixing Huang and Jonathan Gratch},
url = {http://ict.usc.edu/pubs/Explaining%20the%20Variability%20of%20Human%20Nonverbal%20Behaviors%20in%20Face-to-Face%20Interaction.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {13th International Conference on Intelligent Virtual Humans},
address = {Edinburgh, UK},
abstract = {Modeling human nonverbal behaviors is a key factor in creating a successful virtual human system. This is a very challenging problem because human nonverbal behaviors inherently contain a lot of variability. The variability comes from many possible sources, such as the participant’s interactional goal, conversational roles, personality and emotions and so on, making the analysis of the variability hard. Such analysis is even harder in face-to-face interactions since these factors can interact both within and across the participants (i.e. speaker and listener). In this paper, we introduce our initial efforts in analying the variability of human nonverbal behaviors in face-to-face interactions. Specifically, by exploring the Parasocial Consensus Sampling (PCS) framework [13], we show personality has significant influences on listener backchannel feedback and clearly demonstrate how it affects backchannel feedback. Moreover, we suggest that PCS framework provides a general and effective approach to analyze the variability of human nonverbal behaviors, which would be difficult to perform by using the traditional face-to-face interaction data.⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Modeling human nonverbal behaviors is a key factor in creating a successful virtual human system. This is a very challenging problem because human nonverbal behaviors inherently contain a lot of variability. The variability comes from many possible sources, such as the participant’s interactional goal, conversational roles, personality and emotions and so on, making the analysis of the variability hard. Such analysis is even harder in face-to-face interactions since these factors can interact both within and across the participants (i.e. speaker and listener). In this paper, we introduce our initial efforts in analying the variability of human nonverbal behaviors in face-to-face interactions. Specifically, by exploring the Parasocial Consensus Sampling (PCS) framework [13], we show personality has significant influences on listener backchannel feedback and clearly demonstrate how it affects backchannel feedback. Moreover, we suggest that PCS framework provides a general and effective approach to analyze the variability of human nonverbal behaviors, which would be difficult to perform by using the traditional face-to-face interaction data.⬚⬚⬚⬚
Wang, Yuqiong; Khooshabeh, Peter; Gratch, Jonathan
Looking Real and Making Mistakes Proceedings Article
In: 13th International Conference on Intelligent Virtual Humans, Edinburgh, Scotland, 2013.
Abstract | Links | BibTeX | Tags: ARL, DoD, UARC, Virtual Humans
@inproceedings{wang_looking_2013,
title = {Looking Real and Making Mistakes},
author = {Yuqiong Wang and Peter Khooshabeh and Jonathan Gratch},
url = {http://ict.usc.edu/pubs/Looking%20Real%20and%20Making%20Mistakes.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {13th International Conference on Intelligent Virtual Humans},
address = {Edinburgh, Scotland},
abstract = {What happens when a Virtual Human makes mistakes? In this study we investigate the impact of VHs' conversational mistakes in the context of persuasion. The experiment also manipulated the level of photorealism of the VH. Users interacted with a VH that told persuasive information, and they were given the option to use the information to complete a problem-solving task. The VH occasionally made mistakes such as not responding, repeating the same answer, or giving irrelevant feedback. Results indicated that a VH is less persuasive when he or she makes textbackslashtextbackslashtextbackslashtextbackslashemphconversational mistakes. Individual differences also shed light on the cognitive processes of users who interacted with VH who made conversational errors. Participants with a low Need For Cognition are more effected by the conversational errors. VH photorealism or gender did not have significant effects on the persuasion measure. We discuss the implications of these results with regard to Human-Virtual Human interaction.},
keywords = {ARL, DoD, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
What happens when a Virtual Human makes mistakes? In this study we investigate the impact of VHs' conversational mistakes in the context of persuasion. The experiment also manipulated the level of photorealism of the VH. Users interacted with a VH that told persuasive information, and they were given the option to use the information to complete a problem-solving task. The VH occasionally made mistakes such as not responding, repeating the same answer, or giving irrelevant feedback. Results indicated that a VH is less persuasive when he or she makes textbackslashtextbackslashtextbackslashtextbackslashemphconversational mistakes. Individual differences also shed light on the cognitive processes of users who interacted with VH who made conversational errors. Participants with a low Need For Cognition are more effected by the conversational errors. VH photorealism or gender did not have significant effects on the persuasion measure. We discuss the implications of these results with regard to Human-Virtual Human interaction.
Hartholt, Arno; Traum, David; Marsella, Stacy C.; Shapiro, Ari; Stratou, Giota; Leuski, Anton; Morency, Louis-Philippe; Gratch, Jonathan
All Together Now: Introducing the Virtual Human Toolkit Proceedings Article
In: 13th International Conference on Intelligent Virtual Agents, Edinburgh, UK, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{hartholt_all_2013,
title = {All Together Now: Introducing the Virtual Human Toolkit},
author = {Arno Hartholt and David Traum and Stacy C. Marsella and Ari Shapiro and Giota Stratou and Anton Leuski and Louis-Philippe Morency and Jonathan Gratch},
url = {http://ict.usc.edu/pubs/All%20Together%20Now.pdf},
year = {2013},
date = {2013-08-01},
booktitle = {13th International Conference on Intelligent Virtual Agents},
address = {Edinburgh, UK},
abstract = {While virtual humans are proven tools for training, education and re- search, they are far from realizing their full potential. Advances are needed in indi- vidual capabilities, such as character animation and speech synthesis, but perhaps more importantly, fundamental questions remain as to how best to integrate these capabilities into a single framework that allows us to efficiently create characters that can engage users in meaningful and realistic social interactions. This integration re- quires in-depth, inter-disciplinary understanding few individuals, or even teams of individuals, possess. We help address this challenge by introducing the ICT Virtual Human Toolkit1, which offers a flexible framework for exploring a variety of differ- ent types of virtual human systems, from virtual listeners and question-answering characters to virtual role-players. We show that due to its modularity, the Toolkit allows researchers to mix and match provided capabilities with their own, lowering the barrier of entry to this multi-disciplinary research challenge.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
While virtual humans are proven tools for training, education and re- search, they are far from realizing their full potential. Advances are needed in indi- vidual capabilities, such as character animation and speech synthesis, but perhaps more importantly, fundamental questions remain as to how best to integrate these capabilities into a single framework that allows us to efficiently create characters that can engage users in meaningful and realistic social interactions. This integration re- quires in-depth, inter-disciplinary understanding few individuals, or even teams of individuals, possess. We help address this challenge by introducing the ICT Virtual Human Toolkit1, which offers a flexible framework for exploring a variety of differ- ent types of virtual human systems, from virtual listeners and question-answering characters to virtual role-players. We show that due to its modularity, the Toolkit allows researchers to mix and match provided capabilities with their own, lowering the barrier of entry to this multi-disciplinary research challenge.
Hays, Matthew Jensen; Lane, H. Chad; Auerbach, Daniel
Must Feedback Disrupt Presence in Serious Games? Proceedings Article
In: Workshop on Formative Feedback in Interactive Learning Environments at the International Conference on Artificial Intelligence in Education, Memphis, TN, 2013.
Abstract | Links | BibTeX | Tags: Learning Sciences, UARC
@inproceedings{hays_must_2013,
title = {Must Feedback Disrupt Presence in Serious Games?},
author = {Matthew Jensen Hays and H. Chad Lane and Daniel Auerbach},
url = {http://ict.usc.edu/pubs/Must%20Feedback%20Disrupt%20Presence%20in%20Serious%20Games.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {Workshop on Formative Feedback in Interactive Learning Environments at the International Conference on Artificial Intelligence in Education},
address = {Memphis, TN},
abstract = {Serious games are generally designed with two goals in mind: promoting learning and creating compelling and engaging experiences (sometimes termed a sense of presence). Presence itself is believed to promote learning, but serious games often attempt to further increase pedagogical value. One way to do so is to use an intelligent tutoring system (ITS) to provide feedback during gameplay. Some researchers have expressed concern that, because feedback from an ITS is often extrinsic (i.e., it operates outside of the primary game mechanic), attending to it disrupts players’ sense of presence. As a result, learning may be unintentionally hindered by an ITS. However, the most beneficial conditions of instruction are often counterintuitive; in this paper, we challenge the assumption that feedback during learning hinders sense of presence. Across three experiments, we examined how an ITS that provided extrinsic feedback during a serious game affected presence. Across different modalities and conditions, we found that feedback and other ITS features do not always affect presence. Our results suggest that it is possible to provide extrinsic feedback in a serious game without detracting from the immersive power of the game itself.},
keywords = {Learning Sciences, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Serious games are generally designed with two goals in mind: promoting learning and creating compelling and engaging experiences (sometimes termed a sense of presence). Presence itself is believed to promote learning, but serious games often attempt to further increase pedagogical value. One way to do so is to use an intelligent tutoring system (ITS) to provide feedback during gameplay. Some researchers have expressed concern that, because feedback from an ITS is often extrinsic (i.e., it operates outside of the primary game mechanic), attending to it disrupts players’ sense of presence. As a result, learning may be unintentionally hindered by an ITS. However, the most beneficial conditions of instruction are often counterintuitive; in this paper, we challenge the assumption that feedback during learning hinders sense of presence. Across three experiments, we examined how an ITS that provided extrinsic feedback during a serious game affected presence. Across different modalities and conditions, we found that feedback and other ITS features do not always affect presence. Our results suggest that it is possible to provide extrinsic feedback in a serious game without detracting from the immersive power of the game itself.
Marsella, Stacy C.; Xu, Yuyu; Lhommet, Margaux; Feng, Andrew W.; Scherer, Stefan; Shapiro, Ari
Virtual Character Performance From Speech Proceedings Article
In: Symposium on Computer Animation, Anaheim, CA, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{marsella_virtual_2013,
title = {Virtual Character Performance From Speech},
author = {Stacy C. Marsella and Yuyu Xu and Margaux Lhommet and Andrew W. Feng and Stefan Scherer and Ari Shapiro},
url = {http://ict.usc.edu/pubs/Virtual%20Character%20Performance%20From%20Speech.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {Symposium on Computer Animation},
address = {Anaheim, CA},
abstract = {We demonstrate a method for generating a 3D virtual character performance from the audio signal by inferring the acoustic and semantic properties of the utterance. Through a prosodic analysis of the acoustic signal, we perform an analysis for stress and pitch, relate it to the spoken words and identify the agitation state. Our rule-based system performs a shallow analysis of the utterance text to determine its semantic, pragmatic and rhetorical content. Based on these analyses, the system generates facial expressions and behaviors including head movements, eye saccades, gestures, blinks and gazes. Our technique is able to synthesize the performance and generate novel gesture animations based on coarticulation with other closely scheduled animations. Because our method utilizes semantics in addition to prosody, we are able to generate virtual character performances that are more appropriate than methods that use only prosody. We perform a study that shows that our technique outperforms methods that use prosody alone.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We demonstrate a method for generating a 3D virtual character performance from the audio signal by inferring the acoustic and semantic properties of the utterance. Through a prosodic analysis of the acoustic signal, we perform an analysis for stress and pitch, relate it to the spoken words and identify the agitation state. Our rule-based system performs a shallow analysis of the utterance text to determine its semantic, pragmatic and rhetorical content. Based on these analyses, the system generates facial expressions and behaviors including head movements, eye saccades, gestures, blinks and gazes. Our technique is able to synthesize the performance and generate novel gesture animations based on coarticulation with other closely scheduled animations. Because our method utilizes semantics in addition to prosody, we are able to generate virtual character performances that are more appropriate than methods that use only prosody. We perform a study that shows that our technique outperforms methods that use prosody alone.
Nouri, Elnaz; Traum, David
A cross-cultural study of playing simple economic games online with humans and virtual humans Proceedings Article
In: International Conference on Human-Computer Interaction, Las Vegas, NV, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{nouri_cross-cultural_2013,
title = {A cross-cultural study of playing simple economic games online with humans and virtual humans},
author = {Elnaz Nouri and David Traum},
url = {http://ict.usc.edu/pubs/A%20cross-cultural%20study%20of%20playing%20simple%20economic%20games%20online%20with%20humans%20and%20virtual%20humans.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {International Conference on Human-Computer Interaction},
address = {Las Vegas, NV},
abstract = {We compare the simple online economic interactions between a human and a multimodal communication agent (virtual human) to the findings of similar simple interactions with other humans and those that were run in the laboratory. We developed protocols and dialogue capabilities to support the multi modal agent in playing two well-studied economic games (Ultimatum Game, Dictator Game). We analyze the interactions based on the outcome and selfreported values of possible factors involved in the decision making. We compare these parameters across two games, and the two cultures of US and India. Our results show that humans’ interaction with a virtual human is similar to when they are playing with another human and the majority of the people choose to allocate about half of the stakes to the virtual human, just as they would with another human. There are, however, some significant differences between offer distributions and value reports for different conditions (game, opponent, and culture of participant).⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We compare the simple online economic interactions between a human and a multimodal communication agent (virtual human) to the findings of similar simple interactions with other humans and those that were run in the laboratory. We developed protocols and dialogue capabilities to support the multi modal agent in playing two well-studied economic games (Ultimatum Game, Dictator Game). We analyze the interactions based on the outcome and selfreported values of possible factors involved in the decision making. We compare these parameters across two games, and the two cultures of US and India. Our results show that humans’ interaction with a virtual human is similar to when they are playing with another human and the majority of the people choose to allocate about half of the stakes to the virtual human, just as they would with another human. There are, however, some significant differences between offer distributions and value reports for different conditions (game, opponent, and culture of participant).⬚⬚⬚⬚
Morie, Jacquelyn; Kang, Sin-Hwa; Chance, Eric
The Association of In-World Avatar Investment with Expectations of Behavioral Change Proceedings Article
In: Human-Computer Interaction International Conference, Las Vegas, NV, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans, Virtual Worlds
@inproceedings{morie_association_2013,
title = {The Association of In-World Avatar Investment with Expectations of Behavioral Change},
author = {Jacquelyn Morie and Sin-Hwa Kang and Eric Chance},
url = {http://ict.usc.edu/pubs/The%20Association%20of%20In-World%20Avatar%20Investment%20with%20Expectations%20of%20Behavioral%20Change.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {Human-Computer Interaction International Conference},
address = {Las Vegas, NV},
abstract = {We explore whether watching the behavior of an avatar created by a user can affect that users' behavior in the actual world. This research aims to determine if we can achieve results similar to thos obtained from an experimental design detailed in Study 3 of "Virtual Self-Modeling: The Effects of Vicarious Reinforcement and Identification on Exercise Behaviors" (Fox and Bailenson, 2009), but using avatars created by observers rather than experimenter-provided ones enhanced with photographic likeness. Rox and Bailenson theorized that the behavioral change elicited stems from modeling the behavior of physically similar people as supported by social cognitive theory. In this study, we focused more on investigating whether people's owen avatars' behavior would elicit behavioral change based on social-perception theory. Therefore, users observed their own avatars that were doing exercise or not regardless of any physical similarity between the avatars and their owners. The preliminary results showed there was a strong trend for users to engage in physical activities more when they watched their own avatars exercise, compared to observing their own avatars that did not exercise. The results also demonstrated that users with higher body mass index (BMI) engaged in physical activities more when they watched their own avatars with exercise behavior, compared to users with lower BMI. This study seeks to clarify whether or not the notions of psychological reflexivity and avatar ownership/investment are possible factors influencing avatar owners' behavioral outcomes.},
keywords = {UARC, Virtual Humans, Virtual Worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
We explore whether watching the behavior of an avatar created by a user can affect that users' behavior in the actual world. This research aims to determine if we can achieve results similar to thos obtained from an experimental design detailed in Study 3 of "Virtual Self-Modeling: The Effects of Vicarious Reinforcement and Identification on Exercise Behaviors" (Fox and Bailenson, 2009), but using avatars created by observers rather than experimenter-provided ones enhanced with photographic likeness. Rox and Bailenson theorized that the behavioral change elicited stems from modeling the behavior of physically similar people as supported by social cognitive theory. In this study, we focused more on investigating whether people's owen avatars' behavior would elicit behavioral change based on social-perception theory. Therefore, users observed their own avatars that were doing exercise or not regardless of any physical similarity between the avatars and their owners. The preliminary results showed there was a strong trend for users to engage in physical activities more when they watched their own avatars exercise, compared to observing their own avatars that did not exercise. The results also demonstrated that users with higher body mass index (BMI) engaged in physical activities more when they watched their own avatars with exercise behavior, compared to users with lower BMI. This study seeks to clarify whether or not the notions of psychological reflexivity and avatar ownership/investment are possible factors influencing avatar owners' behavioral outcomes.
Fyffe, Graham; Jones, Andrew; Alexander, Oleg; Ichikari, Ryosuke; Graham, Paul; Nagano, Koki; Busch, Jay; Debevec, Paul
Driving High-Resolution Facial Blendshapes with Video Performance Capture Proceedings Article
In: SIGGRAPH, Anaheim, CA, 2013.
Links | BibTeX | Tags: Graphics, UARC
@inproceedings{fyffe_driving_2013,
title = {Driving High-Resolution Facial Blendshapes with Video Performance Capture},
author = {Graham Fyffe and Andrew Jones and Oleg Alexander and Ryosuke Ichikari and Paul Graham and Koki Nagano and Jay Busch and Paul Debevec},
url = {http://ict.usc.edu/pubs/Driving%20High-Resolution%20Facial%20Blendshapes%20with%20Video%20Performance.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {SIGGRAPH},
address = {Anaheim, CA},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Pynadath, David V.; Rosenbloom, Paul; Marsella, Stacy C.; Li, Lingshan
Modeling Two-Player Games in the Sigma Graphical Cognitive Architecture Proceedings Article
In: Proceedings of the 6th Conference on Artificial General Intelligence, pp. 98–108, Beijing, China, 2013.
Abstract | Links | BibTeX | Tags: CogArch, Cognitive Architecture, UARC, Virtual Humans
@inproceedings{pynadath_modeling_2013,
title = {Modeling Two-Player Games in the Sigma Graphical Cognitive Architecture},
author = {David V. Pynadath and Paul Rosenbloom and Stacy C. Marsella and Lingshan Li},
url = {http://ict.usc.edu/pubs/Modeling%20Two-Player%20Games%20in%20the%20Sigma%20Graphical%20Cognitive%20Architecture.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {Proceedings of the 6th Conference on Artificial General Intelligence},
pages = {98–108},
address = {Beijing, China},
abstract = {Effective social interaction and, in particular, a Theory of Mind are critical components of human intelligence, allowing us to form beliefs about other people, generate expectations about their behavior, and use those expectations to inform our own decision-making. This article presents an investigation into methods for realizing Theory of Mind within Sigma, a graphical cognitive architecture. By extending the architecture to capture independent decisions and problem-solving for multiple agents, we implemented Sigma models of several canonical examples from game theory. We show that the resulting Sigma agents can capture the same behaviors prescribed by equilibrium solutions.},
keywords = {CogArch, Cognitive Architecture, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Effective social interaction and, in particular, a Theory of Mind are critical components of human intelligence, allowing us to form beliefs about other people, generate expectations about their behavior, and use those expectations to inform our own decision-making. This article presents an investigation into methods for realizing Theory of Mind within Sigma, a graphical cognitive architecture. By extending the architecture to capture independent decisions and problem-solving for multiple agents, we implemented Sigma models of several canonical examples from game theory. We show that the resulting Sigma agents can capture the same behaviors prescribed by equilibrium solutions.
Marsella, Stacy C.; Shapiro, Ari; Feng, Andrew W.; Xu, Yuyu; Lhommet, Margaux; Scherer, Stefan
Towards Higher Quality Character Performance in Previz Proceedings Article
In: Digital Production Symposium, Anaheim, CA, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{marsella_towards_2013,
title = {Towards Higher Quality Character Performance in Previz},
author = {Stacy C. Marsella and Ari Shapiro and Andrew W. Feng and Yuyu Xu and Margaux Lhommet and Stefan Scherer},
url = {http://ict.usc.edu/pubs/Towards%20Higher%20Quality%20Character%20Performance%20in%20Previz.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {Digital Production Symposium},
address = {Anaheim, CA},
abstract = {Previsualization tools are used to obtain a preliminary but rough version of a film, television or other production. Used for both live-action and animated films, they allow a director to set up camera angles, arrange scenes, dialogue, and other scene elements without the expense of paying live actors, constructing physical sets, or related production costs. By seeing an early approximation of the final production, decisions about scenes, elements, story and the factors affecting it can be made early in the process, potentially reducing costs and improving overall quality. Current previsualization technologies have made inroads into generating these "videomatics", where controls over cameras and static elements, such as buildings, roads and scenery, can be quickly incorporated from a low cost libraries of 3D assets. Even the generation of effects such as explosions, running water, and smoke can be quickly generated in previz scenes from commodity software.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Previsualization tools are used to obtain a preliminary but rough version of a film, television or other production. Used for both live-action and animated films, they allow a director to set up camera angles, arrange scenes, dialogue, and other scene elements without the expense of paying live actors, constructing physical sets, or related production costs. By seeing an early approximation of the final production, decisions about scenes, elements, story and the factors affecting it can be made early in the process, potentially reducing costs and improving overall quality. Current previsualization technologies have made inroads into generating these "videomatics", where controls over cameras and static elements, such as buildings, roads and scenery, can be quickly incorporated from a low cost libraries of 3D assets. Even the generation of effects such as explosions, running water, and smoke can be quickly generated in previz scenes from commodity software.
Parsons, Thomas D.; Courtney, Chris; Dawson, Michael E.; Rizzo, Albert; Arizmendi, Brian
Visuospatial Processing and Learning Effects in Virtual Reality Based Mental Rotation and Navigational Tasks Proceedings Article
In: International Conference on Human-Computer Interaction, Las Vegas, NV, 2013.
Abstract | Links | BibTeX | Tags: DoD, MedVR, UARC
@inproceedings{parsons_visuospatial_2013,
title = {Visuospatial Processing and Learning Effects in Virtual Reality Based Mental Rotation and Navigational Tasks},
author = {Thomas D. Parsons and Chris Courtney and Michael E. Dawson and Albert Rizzo and Brian Arizmendi},
url = {http://ict.usc.edu/pubs/Visuospatial%20Processing%20and%20Learning%20Effects%20in%20Virtual%20Reality%20Based%20Mental%20Rotation%20and%20Navigational%20Tasks.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {International Conference on Human-Computer Interaction},
address = {Las Vegas, NV},
abstract = {Visuospatial function and performance in interactions between humans and computers involve the human identification and manipulation of computer generated stimuli and their location. The impact of learning on mental rotation has been demonstrated in studies relating everyday spatial activities and spatial abilities. An aspect of visuospatial learning in virtual environments that has not been widely studied is the impact of threat on learning in a navigational task. In fact, to our knowledge, the combined assessment of learning during mental rotation trials and learning in an ecologically valid virtual reality-based navigational environment (that has both high and low threat zones) has not been adequately studied. Results followed expectation: 1) learning occurred in the virtual reality based mental rotation test. Although there was a relation between route learning and practice, a primacy effect was observed as participants performed more poorly when going from the first zone to the last.},
keywords = {DoD, MedVR, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Visuospatial function and performance in interactions between humans and computers involve the human identification and manipulation of computer generated stimuli and their location. The impact of learning on mental rotation has been demonstrated in studies relating everyday spatial activities and spatial abilities. An aspect of visuospatial learning in virtual environments that has not been widely studied is the impact of threat on learning in a navigational task. In fact, to our knowledge, the combined assessment of learning during mental rotation trials and learning in an ecologically valid virtual reality-based navigational environment (that has both high and low threat zones) has not been adequately studied. Results followed expectation: 1) learning occurred in the virtual reality based mental rotation test. Although there was a relation between route learning and practice, a primacy effect was observed as participants performed more poorly when going from the first zone to the last.
Huang, Yu-Jen; Bolas, Mark; Suma, Evan
Fusing Depth, Color, and Skeleton Data for Enhanced Real-Time Hand Segmentation Proceedings Article
In: ACM Symposium on Spatial User Interaction, 2013.
Links | BibTeX | Tags: MxR, UARC
@inproceedings{huang_fusing_2013,
title = {Fusing Depth, Color, and Skeleton Data for Enhanced Real-Time Hand Segmentation},
author = {Yu-Jen Huang and Mark Bolas and Evan Suma},
url = {http://ict.usc.edu/pubs/Fusing%20Depth,%20Color,%20and%20Skeleton%20Data%20for%20Enhanced%20Real-Time%20Hand%20Segmentation.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {ACM Symposium on Spatial User Interaction},
keywords = {MxR, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Nagano, Koki; Jones, Andrew; Liu, Jing; Busch, Jay; Yu, Xueming; Bolas, Mark; Debevec, Paul
An Autostereoscopic Projector Array Optimized for 3D Facial Display Proceedings Article
In: SIGGRAPH 2013 Emerging Technologies, 2013.
Abstract | Links | BibTeX | Tags: Graphics, UARC
@inproceedings{nagano_autostereoscopic_2013,
title = {An Autostereoscopic Projector Array Optimized for 3D Facial Display},
author = {Koki Nagano and Andrew Jones and Jing Liu and Jay Busch and Xueming Yu and Mark Bolas and Paul Debevec},
url = {http://ict.usc.edu/pubs/An%20Autostereoscopic%20Projector%20Array%20Optimized%20for%203D%20Facial%20Display%20.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {SIGGRAPH 2013 Emerging Technologies},
abstract = {Video projectors are rapidly shrinking in size, power consumption, and cost. Such projectors provide unprecedented flexibility to stack, arrange, and aim pixels without the need for moving parts. This dense projector display is optimized in size and resolution to display an autostereoscopic life-sized 3D human face. It utilizes 72 Texas Instruments PICO projectors to illuminate a 30 cm x 30 cm anisotropic screen with a wide 110-degree field of view. The demonstration includes both live scanning of subjects and virtual animated characters.},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Video projectors are rapidly shrinking in size, power consumption, and cost. Such projectors provide unprecedented flexibility to stack, arrange, and aim pixels without the need for moving parts. This dense projector display is optimized in size and resolution to display an autostereoscopic life-sized 3D human face. It utilizes 72 Texas Instruments PICO projectors to illuminate a 30 cm x 30 cm anisotropic screen with a wide 110-degree field of view. The demonstration includes both live scanning of subjects and virtual animated characters.
Talbot, Thomas
Playing with Biology: Making medical games that appear lifelike Journal Article
In: International Journal of Gaming and Computer Mediated Simulations, vol. 5, no. 3, pp. 83–96, 2013.
Abstract | Links | BibTeX | Tags: MedVR, UARC
@article{talbot_playing_2013,
title = {Playing with Biology: Making medical games that appear lifelike},
author = {Thomas Talbot},
url = {http://www.igi-global.com/article/playing-with-biology/93030},
year = {2013},
date = {2013-07-01},
journal = {International Journal of Gaming and Computer Mediated Simulations},
volume = {5},
number = {3},
pages = {83–96},
abstract = {Game-based medical simulations differ from other training modalities in that life processes must be simulated as part of the experience. Biological fidelity is the degree to which character anatomical appearance and physiology behavior are represented within a game or simulation. Methods to achieve physiological fidelity include computational physiology engines, complex state machines, simple state machines and kinetic models. Traditional games also employ health scores that can also be employed for medical gaming. The selection of technique to is dependent upon the goals of the simulation, the types of input expected of the user, the amount of development work possible and the level of fidelity required. Apparent biological fidelity, responsiveness to user inputs and the ability to correct mistakes is often more important than actual biological fidelity.},
keywords = {MedVR, UARC},
pubstate = {published},
tppubtype = {article}
}
Game-based medical simulations differ from other training modalities in that life processes must be simulated as part of the experience. Biological fidelity is the degree to which character anatomical appearance and physiology behavior are represented within a game or simulation. Methods to achieve physiological fidelity include computational physiology engines, complex state machines, simple state machines and kinetic models. Traditional games also employ health scores that can also be employed for medical gaming. The selection of technique to is dependent upon the goals of the simulation, the types of input expected of the user, the amount of development work possible and the level of fidelity required. Apparent biological fidelity, responsiveness to user inputs and the ability to correct mistakes is often more important than actual biological fidelity.
Rosenbloom, Paul S.; Demski, Abram; Han, Teawon; Ustun, Volkan
Learning via Gradient Descent in Sigma Proceedings Article
In: International Conference on Cognitive Modeling, Ottawa, Canada, 2013.
Abstract | Links | BibTeX | Tags: CogArch, Cognitive Architecture, UARC, Virtual Humans
@inproceedings{rosenbloom_learning_2013,
title = {Learning via Gradient Descent in Sigma},
author = {Paul S. Rosenbloom and Abram Demski and Teawon Han and Volkan Ustun},
url = {http://ict.usc.edu/pubs/Learning%20via%20Gradient%20Descent%20in%20Sigma.pdf},
year = {2013},
date = {2013-07-01},
booktitle = {International Conference on Cognitive Modeling},
address = {Ottawa, Canada},
abstract = {Integrating a gradient-descent learning mechanism at the core of the graphical models upon which the Sigma cognitive architecture/system is built yields learning behaviors that span important forms of both procedural learning (e.g., action and reinforcement learning) and declarative learning (e.g., supervised and unsupervised concept formation), plus several additional forms of learning (e.g., distribution tracking and map learning) relevant to cognitive systems/modeling. The core result presented here is this breadth of cognitive learning behaviors that is producible in this uniform manner.},
keywords = {CogArch, Cognitive Architecture, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Integrating a gradient-descent learning mechanism at the core of the graphical models upon which the Sigma cognitive architecture/system is built yields learning behaviors that span important forms of both procedural learning (e.g., action and reinforcement learning) and declarative learning (e.g., supervised and unsupervised concept formation), plus several additional forms of learning (e.g., distribution tracking and map learning) relevant to cognitive systems/modeling. The core result presented here is this breadth of cognitive learning behaviors that is producible in this uniform manner.
Alexander, Oleg; Fyffe, Graham; Busch, Jay; Yu, Xueming; Ichikari, Ryosuke; Jones, Andrew; Debevec, Paul; Jimenez, Jorge; Danvoye, Etienne; Antionazzi, Bernardo; Eheler, Mike; Kysela, Zybnek; Pahlen, Javier
Digital Ira: Creating a Real-Time Photoreal Digital Actor Proceedings Article
In: SIGGRAPH Real Time Live!, Anaheim, CA, 2013, ISBN: 978-1-4503-2342-0.
Abstract | Links | BibTeX | Tags: Graphics, UARC
@inproceedings{alexander_digital_2013,
title = {Digital Ira: Creating a Real-Time Photoreal Digital Actor},
author = {Oleg Alexander and Graham Fyffe and Jay Busch and Xueming Yu and Ryosuke Ichikari and Andrew Jones and Paul Debevec and Jorge Jimenez and Etienne Danvoye and Bernardo Antionazzi and Mike Eheler and Zybnek Kysela and Javier Pahlen},
url = {http://dl.acm.org/citation.cfm?doid=2503385.2503387},
doi = {10.1145/2503385.2503387},
isbn = {978-1-4503-2342-0},
year = {2013},
date = {2013-07-01},
booktitle = {SIGGRAPH Real Time Live!},
address = {Anaheim, CA},
abstract = {In 2008, the "Digital Emily" project [Alexander et al. 2009] showed how a set of high-resolution facial expressions scanned in a light stage could be rigged into a real-time photoreal digital character and driven with video-based facial animation techniques. However, Digital Emily was rendered offline, involved just the front of the face, and was never seen in a tight closeup. In this collaboration between Activision and USC ICT shown at SIGGRAPH 2013's Real-Time Live venue, we endeavoured to create a real-time, photoreal digital human character which could be seen from any viewpoint, in any lighting, and could perform realistically from video performance capture even in a tight closeup. In addition, we wanted this to run in a real-time game-ready production pipeline, ultimately achieving 180 frames per second for a full-screen character on a two-year old graphics card.},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
In 2008, the "Digital Emily" project [Alexander et al. 2009] showed how a set of high-resolution facial expressions scanned in a light stage could be rigged into a real-time photoreal digital character and driven with video-based facial animation techniques. However, Digital Emily was rendered offline, involved just the front of the face, and was never seen in a tight closeup. In this collaboration between Activision and USC ICT shown at SIGGRAPH 2013's Real-Time Live venue, we endeavoured to create a real-time, photoreal digital human character which could be seen from any viewpoint, in any lighting, and could perform realistically from video performance capture even in a tight closeup. In addition, we wanted this to run in a real-time game-ready production pipeline, ultimately achieving 180 frames per second for a full-screen character on a two-year old graphics card.
Tunwattanapong, Borom; Fyffe, Graham; Graham, Paul; Busch, Jay; Yu, Xueming; Ghosh, Abhijeet; Debevec, Paul
Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination Journal Article
In: ACM Transactions on Graphics, vol. 32, no. 4, 2013, ISSN: 07300301.
Abstract | Links | BibTeX | Tags: Graphics, UARC
@article{tunwattanapong_acquiring_2013,
title = {Acquiring Reflectance and Shape from Continuous Spherical Harmonic Illumination},
author = {Borom Tunwattanapong and Graham Fyffe and Paul Graham and Jay Busch and Xueming Yu and Abhijeet Ghosh and Paul Debevec},
url = {http://ict.usc.edu/pubs/Acquiring%20Re%ef%ac%82ectance%20and%20Shape%20from%20Continuous%20Spherical%20Harmonic%20Illumination.pdf},
doi = {10.1145/2461912.2461944},
issn = {07300301},
year = {2013},
date = {2013-07-01},
journal = {ACM Transactions on Graphics},
volume = {32},
number = {4},
abstract = {We present a novel technique for acquiring the geometry and spatially-varying reflectance properties of 3D objects by observing them under continuous spherical harmonic illumination conditions. The technique is general enough to characterize either entirely specular or entirely diffuse materials, or any varying combination across the surface of the object. We employ a novel computational illumination setup consisting of a rotating arc of controllable LEDs which sweep out programmable spheres of incident illumination during 1-second exposures. We illuminate the object with a succession of spherical harmonic illumination conditions, as well as photographed environmental lighting for validation. From the response of the object to the harmonics, we can separate diffuse and specular reflections, estimate world-space diffuse and specular normals, and compute anisotropic roughness parameters for each view of the object. We then use the maps of both diffuse and specular reflectance to form correspondences in a multiview stereo algorithm, which allows even highly specular surfaces to be corresponded across views. The algorithm yields a complete 3D model and a set of merged reflectance maps. We use this technique to digitize the shape and reflectance of a variety of objects difficult to acquire with other techniques and present validation renderings which match well to photographs in similar lighting.},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {article}
}
We present a novel technique for acquiring the geometry and spatially-varying reflectance properties of 3D objects by observing them under continuous spherical harmonic illumination conditions. The technique is general enough to characterize either entirely specular or entirely diffuse materials, or any varying combination across the surface of the object. We employ a novel computational illumination setup consisting of a rotating arc of controllable LEDs which sweep out programmable spheres of incident illumination during 1-second exposures. We illuminate the object with a succession of spherical harmonic illumination conditions, as well as photographed environmental lighting for validation. From the response of the object to the harmonics, we can separate diffuse and specular reflections, estimate world-space diffuse and specular normals, and compute anisotropic roughness parameters for each view of the object. We then use the maps of both diffuse and specular reflectance to form correspondences in a multiview stereo algorithm, which allows even highly specular surfaces to be corresponded across views. The algorithm yields a complete 3D model and a set of merged reflectance maps. We use this technique to digitize the shape and reflectance of a variety of objects difficult to acquire with other techniques and present validation renderings which match well to photographs in similar lighting.
Alexander, Oleg; Busch, Jay; Graham, Paul; Tunwattanapong, Borom; Jones, Andrew; Nagano, Koki; Ichikari, Ryosuke; Debevec, Paul; Fyffe, Graham
Digital Ira: High-Resolution Facial Performance Playback Proceedings Article
In: SIGGRAPH 2013 Real-Time Live! The 40th International Conference and Exhibition on Computer Graphics and Interactive Techniques, Anaheim, CA, 2013.
Abstract | Links | BibTeX | Tags: Graphics, UARC
@inproceedings{alexander_digital_2013-1,
title = {Digital Ira: High-Resolution Facial Performance Playback},
author = {Oleg Alexander and Jay Busch and Paul Graham and Borom Tunwattanapong and Andrew Jones and Koki Nagano and Ryosuke Ichikari and Paul Debevec and Graham Fyffe},
url = {http://gl.ict.usc.edu/Research/DigitalIra/},
year = {2013},
date = {2013-07-01},
booktitle = {SIGGRAPH 2013 Real-Time Live! The 40th International Conference and Exhibition on Computer Graphics and Interactive Techniques},
address = {Anaheim, CA},
abstract = {In this collaboration between Activision and USC ICT, we tried to create a real-time, photoreal digital human character which could be seen from any viewpoint, any lighting, and could perform realistically from video performance capture even in a tight closeup. In addition, we needed this to run in a game-ready production pipeline. To achieve this, we scanned the actor in thirty high-resolution expressions using the USC ICT's new Light Stage X system [Ghosh et al. SIGGRAPHAsia2011] and chose eight expressions for the real-time performance rendering. To record the performance, we shot multi-view 30fps video of the actor performing improvised lines using the same multi-camera rig. We used a new tool called Vuvuzela to interactively and precisely correspond all expression (u,v)'s to the neutral expression, which was retopologized to an artist mesh. Our new offline animation solver works by creating a performance graph representing dense GPU optical flow between the video frames and the eight expressions. This graph gets pruned by analyzing the correlation between the video frames and the expression scans over twelve facial regions. The algorithm then computes dense optical flow and 3D triangulation yielding per-frame spatially varying blendshape weights approximating the performance.},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
In this collaboration between Activision and USC ICT, we tried to create a real-time, photoreal digital human character which could be seen from any viewpoint, any lighting, and could perform realistically from video performance capture even in a tight closeup. In addition, we needed this to run in a game-ready production pipeline. To achieve this, we scanned the actor in thirty high-resolution expressions using the USC ICT's new Light Stage X system [Ghosh et al. SIGGRAPHAsia2011] and chose eight expressions for the real-time performance rendering. To record the performance, we shot multi-view 30fps video of the actor performing improvised lines using the same multi-camera rig. We used a new tool called Vuvuzela to interactively and precisely correspond all expression (u,v)'s to the neutral expression, which was retopologized to an artist mesh. Our new offline animation solver works by creating a performance graph representing dense GPU optical flow between the video frames and the eight expressions. This graph gets pruned by analyzing the correlation between the video frames and the expression scans over twelve facial regions. The algorithm then computes dense optical flow and 3D triangulation yielding per-frame spatially varying blendshape weights approximating the performance.
DeVault, David; Traum, David
A method for the approximation of incremental understanding of explicit utterance meaning using predictive models in finite domains Proceedings Article
In: Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{devault_method_2013,
title = {A method for the approximation of incremental understanding of explicit utterance meaning using predictive models in finite domains},
author = {David DeVault and David Traum},
url = {http://ict.usc.edu/pubs/A%20method%20for%20the%20approximation%20of%20incremental%20understanding%20of%20explicit%20utterance%20meaning%20using%20predictive%20models%20in%20%EF%AC%81nite%20domains.pdf},
year = {2013},
date = {2013-06-01},
booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
address = {Atlanta, GA},
abstract = {This paper explores the relationship between explicit and predictive models of incremental speech understanding in a dialogue system that supports a finite set of user utterance meanings. We present a method that enables the approximation of explicit understanding using information implicit in a predictive understanding model for the same domain. We show promising performance for this method in a corpus evaluation, and discuss its practical application and annotation costs in relation to some alternative approaches.⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
This paper explores the relationship between explicit and predictive models of incremental speech understanding in a dialogue system that supports a finite set of user utterance meanings. We present a method that enables the approximation of explicit understanding using information implicit in a predictive understanding model for the same domain. We show promising performance for this method in a corpus evaluation, and discuss its practical application and annotation costs in relation to some alternative approaches.⬚⬚⬚
Morie, Jacquelyn; Kang, Sin-Hwa
What Can Your Avatar Tell the Doctor? Proceedings Article
In: The CyberPsychology & CyberTherapy Conference, Brussels, Belgium, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans, Virtual Worlds
@inproceedings{morie_what_2013,
title = {What Can Your Avatar Tell the Doctor?},
author = {Jacquelyn Morie and Sin-Hwa Kang},
url = {http://ict.usc.edu/pubs/What%20Can%20Your%20Avatar%20Tell%20the%20Doctor.pdf},
year = {2013},
date = {2013-06-01},
booktitle = {The CyberPsychology & CyberTherapy Conference},
address = {Brussels, Belgium},
abstract = {The goal of our study is to explore a functioning virtual self that can present salient information about our health. We investigate several types of connectivity between a user and his/her animated character representation (also referred to as an avatar). We aim to find various forms of avatar connection with its user to address the value of avatars in health care applications. We propose an additional variation on an existing categorization of avatar-self associations. We specifically examine the potential of the biologically instrumented end of the avatar connectivity continuum.},
keywords = {UARC, Virtual Humans, Virtual Worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
The goal of our study is to explore a functioning virtual self that can present salient information about our health. We investigate several types of connectivity between a user and his/her animated character representation (also referred to as an avatar). We aim to find various forms of avatar connection with its user to address the value of avatars in health care applications. We propose an additional variation on an existing categorization of avatar-self associations. We specifically examine the potential of the biologically instrumented end of the avatar connectivity continuum.
Song, Yale; Morency, Louis-Philippe; Davis, Randall
Action Recognition by Hierarchical Sequence Summarization Proceedings Article
In: Computer Vision and Pattern Recognition (CVPR 2013), Portland, OR, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{song_action_2013,
title = {Action Recognition by Hierarchical Sequence Summarization},
author = {Yale Song and Louis-Philippe Morency and Randall Davis},
url = {http://ict.usc.edu/pubs/Action%20Recognition%20by%20Hierarchical%20Sequence%20Summarization.pdf},
year = {2013},
date = {2013-06-01},
booktitle = {Computer Vision and Pattern Recognition (CVPR 2013)},
address = {Portland, OR},
abstract = {Recent progress has shown that learning from hierarchical feature representations leads to improvements in various computer vision tasks. Motivated by the observations that human activity data contains information at various temporal resolutions, we present a hierarchical sequence summarization approach for action recognition that learns multiple layers of discriminative feature representations. We build up a hierarchy dynamically and recursively by alternating sequence learning and sequence summarization. For sequence learning we use CRFs with latent variables to learn hidden spatio-temporal dynamics; for sequence summarization we group observations that have similar semantic meaning in the latent space. For each layer we learn an abstract feature representation through non-linear gate functions. This procedure is repeated to obtain a hierarchical sequence summary representation. We develop an efficient learning method to train our model and show that its complexity grows sublinearly with the size of the hierarchy. Experimental results show the effectiveness of our approach, achieving the best published results on the Ar-mGesture and Canal9 datasets.},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Recent progress has shown that learning from hierarchical feature representations leads to improvements in various computer vision tasks. Motivated by the observations that human activity data contains information at various temporal resolutions, we present a hierarchical sequence summarization approach for action recognition that learns multiple layers of discriminative feature representations. We build up a hierarchy dynamically and recursively by alternating sequence learning and sequence summarization. For sequence learning we use CRFs with latent variables to learn hidden spatio-temporal dynamics; for sequence summarization we group observations that have similar semantic meaning in the latent space. For each layer we learn an abstract feature representation through non-linear gate functions. This procedure is repeated to obtain a hierarchical sequence summary representation. We develop an efficient learning method to train our model and show that its complexity grows sublinearly with the size of the hierarchy. Experimental results show the effectiveness of our approach, achieving the best published results on the Ar-mGesture and Canal9 datasets.
Reisenzein, Rainer; Hudlicka, Eva; Dastani, Mehdi; Gratch, Jonathan; Hindriks, Koen; Lorini, Emiliano; Meyer, John-Jules
Computational Modeling of Emotion: Towards Improving the Inter- and Intradisciplinary Exchange Journal Article
In: IEEE Transactions on Affective Computing, vol. 4, no. 3, pp. 246–266, 2013, ISSN: 1949-3045.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@article{reisenzein_computational_2013,
title = {Computational Modeling of Emotion: Towards Improving the Inter- and Intradisciplinary Exchange},
author = {Rainer Reisenzein and Eva Hudlicka and Mehdi Dastani and Jonathan Gratch and Koen Hindriks and Emiliano Lorini and John-Jules Meyer},
url = {http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6517842},
doi = {10.1109/T-AFFC.2013.14},
issn = {1949-3045},
year = {2013},
date = {2013-05-01},
journal = {IEEE Transactions on Affective Computing},
volume = {4},
number = {3},
pages = {246–266},
abstract = {The past years have seen increasing cooperation between psychology and computer science in the field of computational modeling of emotion. However, to realize its potential, the exchange between the two disciplines, as well as the intradisciplinary coordination, should be further improved. We make three proposals for how this could be achieved. The proposals refer to: (1) systematizing and classifying the assumptions of psychological emotion theories; (2) formalizing emotion theories in implementation-independent formal languages (set theory; agent logics); and (3) modeling emotions using general cognitive architectures (such as Soar and ACT-R), general agent architectures (such as the BDI architecture) or generalpurpose affective agent architectures. These proposal share two overarching themes. The first is a proposal for modularization: deconstruct emotion theories into basic assumptions; modularize architectures. The second is a proposal for unification and standardization: Translate different emotion theories into a common informal conceptual system or a formal language, or implement them in a common architecture.⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {article}
}
The past years have seen increasing cooperation between psychology and computer science in the field of computational modeling of emotion. However, to realize its potential, the exchange between the two disciplines, as well as the intradisciplinary coordination, should be further improved. We make three proposals for how this could be achieved. The proposals refer to: (1) systematizing and classifying the assumptions of psychological emotion theories; (2) formalizing emotion theories in implementation-independent formal languages (set theory; agent logics); and (3) modeling emotions using general cognitive architectures (such as Soar and ACT-R), general agent architectures (such as the BDI architecture) or generalpurpose affective agent architectures. These proposal share two overarching themes. The first is a proposal for modularization: deconstruct emotion theories into basic assumptions; modularize architectures. The second is a proposal for unification and standardization: Translate different emotion theories into a common informal conceptual system or a formal language, or implement them in a common architecture.⬚
Scherer, Stefan; Pestian, John; Morency, Louis-Philippe
Investigating the Speech Characteristics of Suicidal Adolescents Proceedings Article
In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{scherer_investigating_2013,
title = {Investigating the Speech Characteristics of Suicidal Adolescents},
author = {Stefan Scherer and John Pestian and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Investigating%20the%20Speech%20Characteristics%20of%20Suicidal%20Adolescents.pdf},
year = {2013},
date = {2013-05-01},
booktitle = {International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
address = {Vancouver, Canada},
abstract = {Suicide is a very serious problem. In the United states it ranks as the second most frequent cause of death among teenagers between the ages of 12 and 17. In this work, we investigate speech characteristics of prosody as well as voice quality in a dyadic interview corpus with suicidal and non-suicidal adolescents. In these interviews the adolescents answer specifically designed questions. Based on this limited dataset, we reveal statistically significant differences in the speech patterns of suicidal adolescents within the investigated interview corpus. Further, we investigate the classification capabilities of machine learning approaches both on an utterance as well as an interview level. The work shows promising results in a speaker independent classification experiment based on only a dozen speech features. We believe that once the algorithms are refined and integrated with other methods, they may be of value to the clinician.⬚⬚⬚⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
Suicide is a very serious problem. In the United states it ranks as the second most frequent cause of death among teenagers between the ages of 12 and 17. In this work, we investigate speech characteristics of prosody as well as voice quality in a dyadic interview corpus with suicidal and non-suicidal adolescents. In these interviews the adolescents answer specifically designed questions. Based on this limited dataset, we reveal statistically significant differences in the speech patterns of suicidal adolescents within the investigated interview corpus. Further, we investigate the classification capabilities of machine learning approaches both on an utterance as well as an interview level. The work shows promising results in a speaker independent classification experiment based on only a dozen speech features. We believe that once the algorithms are refined and integrated with other methods, they may be of value to the clinician.⬚⬚⬚⬚⬚
Weninger, Felix; Wagner, Claudia; Wollmer, Martin; Schuller, Bjorn; Morency, Louis-Philippe
Speaker Trait Characterization in Web Videos: Uniting Speech, Language, and Facial Features Proceedings Article
In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Vancouver, Canada, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{weninger_speaker_2013,
title = {Speaker Trait Characterization in Web Videos: Uniting Speech, Language, and Facial Features},
author = {Felix Weninger and Claudia Wagner and Martin Wollmer and Bjorn Schuller and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Speaker%20Trait%20Characterization%20in%20Web%20Videos-%20Uniting%20Speech,%20Language,%20and%20Facial%20Features.pdf},
year = {2013},
date = {2013-05-01},
booktitle = {International Conference on Acoustics, Speech, and Signal Processing (ICASSP)},
address = {Vancouver, Canada},
abstract = {We present a multi-modal approach to speaker characterization using acoustic, visual and linguistic features. Full realism is provided by evaluation on a database of real-life web videos and automatic feature extraction including face and eye detection, and automatic speech recognition. Different segmentations are evaluated for the audio and video streams, and the statistical relevance of Linguistic Inquiry and Word Count (LIWC) features is confirmed. In the result, late multimodal fusion delivers 73, 92 and 73 % average recall in binary age, gender and race classification on unseen test subjects, outperforming the best single modalities for age and race.⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
We present a multi-modal approach to speaker characterization using acoustic, visual and linguistic features. Full realism is provided by evaluation on a database of real-life web videos and automatic feature extraction including face and eye detection, and automatic speech recognition. Different segmentations are evaluated for the audio and video streams, and the statistical relevance of Linguistic Inquiry and Word Count (LIWC) features is confirmed. In the result, late multimodal fusion delivers 73, 92 and 73 % average recall in binary age, gender and race classification on unseen test subjects, outperforming the best single modalities for age and race.⬚
Hatch, Henry J.; Cherry, W. Peter; Glimcher, Paul W.; Hill, Randall W.; Keesee, Robin L.; Kieff, Elliot D.; Macmillan, Jean; Melvin, William L.; Paul, Richard R.; Pew, Richard; Rose, M. Frank; Sciarretta, Albert A.; Speed, Ann; Yakovac, Joseph
Making the Soldier Decisive on Future Battlefields Book
National Academies Press, Washington D.C., 2013.
Links | BibTeX | Tags: DoD, UARC
@book{hatch_making_2013,
title = {Making the Soldier Decisive on Future Battlefields},
author = {Henry J. Hatch and W. Peter Cherry and Paul W. Glimcher and Randall W. Hill and Robin L. Keesee and Elliot D. Kieff and Jean Macmillan and William L. Melvin and Richard R. Paul and Richard Pew and M. Frank Rose and Albert A. Sciarretta and Ann Speed and Joseph Yakovac},
url = {http://www.nap.edu/catalog.php?record_id=18321},
year = {2013},
date = {2013-05-01},
publisher = {National Academies Press},
address = {Washington D.C.},
keywords = {DoD, UARC},
pubstate = {published},
tppubtype = {book}
}
Suma, Evan; Krum, David M.; Bolas, Mark
Redirected Walking in Mixed Reality Training Applications Book Section
In: Human Walking in Virtual Environments: Perception, Technology, and Applications, Springer, 2013, ISBN: 1-4419-8431-3.
Abstract | Links | BibTeX | Tags: MxR, UARC
@incollection{suma_redirected_2013,
title = {Redirected Walking in Mixed Reality Training Applications},
author = {Evan Suma and David M. Krum and Mark Bolas},
url = {http://www.amazon.com/Human-Walking-Virtual-Environments-Applications/dp/1441984313/ref=sr_1_1},
isbn = {1-4419-8431-3},
year = {2013},
date = {2013-05-01},
booktitle = {Human Walking in Virtual Environments: Perception, Technology, and Applications},
publisher = {Springer},
edition = {2013},
abstract = {To create effective immersive training experiences, it is important to provide intuitive interfaces that allow users to move around and interact with virtual content in a manner that replicates real world experiences. However, natural loco- motion remains an implementation challenge because the dimensions of the phys- ical tracking space restrict the size of the virtual environment that users can walk through. To relax these limitations, redirected walking techniques may be employed to enable walking through immersive virtual environments that are substantially larger than the physical tracking area. In this chapter, we present practical design considerations for employing redirected walking in immersive training applications and recent research evaluating the impact on spatial orientation. Additionally, we also describe an alternative implementation of redirection that is more appropriate for mixed reality environments. Finally, we discuss challenges and future directions for research in redirected walking with the goal of transitioning these techniques into practical training simulators.},
keywords = {MxR, UARC},
pubstate = {published},
tppubtype = {incollection}
}
To create effective immersive training experiences, it is important to provide intuitive interfaces that allow users to move around and interact with virtual content in a manner that replicates real world experiences. However, natural loco- motion remains an implementation challenge because the dimensions of the phys- ical tracking space restrict the size of the virtual environment that users can walk through. To relax these limitations, redirected walking techniques may be employed to enable walking through immersive virtual environments that are substantially larger than the physical tracking area. In this chapter, we present practical design considerations for employing redirected walking in immersive training applications and recent research evaluating the impact on spatial orientation. Additionally, we also describe an alternative implementation of redirection that is more appropriate for mixed reality environments. Finally, we discuss challenges and future directions for research in redirected walking with the goal of transitioning these techniques into practical training simulators.
Rosenbloom, Paul S.
The Sigma cognitive architecture and system Journal Article
In: AISB Quarterly, pp. 4–13, 2013.
Links | BibTeX | Tags: CogArch, Cognitive Architecture, UARC, Virtual Humans
@article{rosenbloom_sigma_2013,
title = {The Sigma cognitive architecture and system},
author = {Paul S. Rosenbloom},
url = {http://ict.usc.edu/pubs/The%20Sigma%20cognitive%20architecture%20and%20system.pdf},
year = {2013},
date = {2013-05-01},
journal = {AISB Quarterly},
pages = {4–13},
keywords = {CogArch, Cognitive Architecture, UARC, Virtual Humans},
pubstate = {published},
tppubtype = {article}
}
Wienberg, Christopher; Roemmele, Melissa; Gordon, Andrew S.
Content-Based Similarity Measures of Weblog Authors Proceedings Article
In: ACM Web Science Conference, Paris, France, 2013.
Abstract | Links | BibTeX | Tags: The Narrative Group, UARC
@inproceedings{wienberg_content-based_2013,
title = {Content-Based Similarity Measures of Weblog Authors},
author = {Christopher Wienberg and Melissa Roemmele and Andrew S. Gordon},
url = {http://ict.usc.edu/pubs/Content-Based%20Similarity%20Measures%20of%20Weblog%20Authors.PDF},
year = {2013},
date = {2013-05-01},
booktitle = {ACM Web Science Conference},
address = {Paris, France},
abstract = {With recent research interest in the confounding roles of homophily and contagion in studies of social influence, there is a strong need for reliable content-based measures of the similarity between people. In this paper, we investigate the use of text similarity measures as a way of predicting the similarity of prolific weblog authors. We describe a novel method of collecting human judgments of overall similarity between two authors, as well as demographic, political, cultural, religious, values, hobbies/interests, personality, and writing style similarity. We then apply a range of automated textual similarity measures based on word frequency counts, and calculate their statistical correlation with human judgments. Our findings indicate that commonly used text similarity measures do not correlate well with human judgments of author similarity. However, various measures that pay special attention to personal pronouns and their context correlate significantly with different facets of similarity.},
keywords = {The Narrative Group, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
With recent research interest in the confounding roles of homophily and contagion in studies of social influence, there is a strong need for reliable content-based measures of the similarity between people. In this paper, we investigate the use of text similarity measures as a way of predicting the similarity of prolific weblog authors. We describe a novel method of collecting human judgments of overall similarity between two authors, as well as demographic, political, cultural, religious, values, hobbies/interests, personality, and writing style similarity. We then apply a range of automated textual similarity measures based on word frequency counts, and calculate their statistical correlation with human judgments. Our findings indicate that commonly used text similarity measures do not correlate well with human judgments of author similarity. However, various measures that pay special attention to personal pronouns and their context correlate significantly with different facets of similarity.
Graham, Paul; Tunwattanapong, Borom; Busch, Jay; Yu, Xueming; Jones, Andrew; Debevec, Paul; Ghosh, Abhijeet
Measurement-based Synthesis of Facial Microgeometry Proceedings Article
In: EUROGRAPHICS, Girona, Spain, 2013.
Abstract | Links | BibTeX | Tags: Graphics, UARC
@inproceedings{graham_measurement-based_2013,
title = {Measurement-based Synthesis of Facial Microgeometry},
author = {Paul Graham and Borom Tunwattanapong and Jay Busch and Xueming Yu and Andrew Jones and Paul Debevec and Abhijeet Ghosh},
url = {http://ict.usc.edu/pubs/Measurement-based%20Synthesis%20of%20Facial%20Microgeometry.pdf},
year = {2013},
date = {2013-05-01},
booktitle = {EUROGRAPHICS},
address = {Girona, Spain},
abstract = {Current scanning techniques record facial mesostructure with submillimeter precision showing pores, wrinkles, and creases. However, surface roughness continues to shape specular reflection at the level of microstructure: micron scale structures. Here, we present an approach to increase the resolution of mesostructure-level facial scans using microstructure examples digitized about the face. We digitize the skin patches using polarized gradient illumination and 10 mm resolution macro photography, and observe point-source reflectance measurements to characterize the specular reflectance lobe at this smaller scale. We then perform constrained texture synthesis to create appropriate surface microstructure per facial region, blending the regions to cover the whole entire face. We show that renderings of microstructure-augmented facial models preserve the original scanned mesostructure and exhibit surface reflections which are qualitatively more consistent with real photographs.},
keywords = {Graphics, UARC},
pubstate = {published},
tppubtype = {inproceedings}
}
Current scanning techniques record facial mesostructure with submillimeter precision showing pores, wrinkles, and creases. However, surface roughness continues to shape specular reflection at the level of microstructure: micron scale structures. Here, we present an approach to increase the resolution of mesostructure-level facial scans using microstructure examples digitized about the face. We digitize the skin patches using polarized gradient illumination and 10 mm resolution macro photography, and observe point-source reflectance measurements to characterize the specular reflectance lobe at this smaller scale. We then perform constrained texture synthesis to create appropriate surface microstructure per facial region, blending the regions to cover the whole entire face. We show that renderings of microstructure-augmented facial models preserve the original scanned mesostructure and exhibit surface reflections which are qualitatively more consistent with real photographs.
Levesque, Julien-Charles; Morency, Louis-Philippe; Gagne, Christian
Sequential Emotion Recognition using Latent-Dynamic Conditional Neural Fields Proceedings Article
In: IEEE Conference on Automatic Face and Gesture Recognition, Shanghai, China, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans
@inproceedings{levesque_sequential_2013,
title = {Sequential Emotion Recognition using Latent-Dynamic Conditional Neural Fields},
author = {Julien-Charles Levesque and Louis-Philippe Morency and Christian Gagne},
url = {http://ict.usc.edu/pubs/Sequential%20Emotion%20Recognition%20using%20Latent-Dynamic%20Conditional%20Neural%20Fields.pdf},
year = {2013},
date = {2013-04-01},
booktitle = {IEEE Conference on Automatic Face and Gesture Recognition},
address = {Shanghai, China},
abstract = {A wide number of problems in face and gesture analysis involve the labeling of temporal sequences. In this paper, we introduce a discriminative model for such sequence labeling tasks. This model involves two layers of latent dynamics, each with their separate roles. The first layer, the neural network or gating layer, aims to extract non-linear relationships between input data and output labels. The second layer, the hidden-states layer, aims to model temporal substructure in the sequence by learning hidden-states and their transition dynamics. A new regularization term is proposed for the training of this model, encouraging diversity between hidden-states. We evaluate the performance of this model on an audiovisual dataset of emotion recognition and compare it against other popular methods for sequence labeling.⬚⬚},
keywords = {UARC, Virtual Humans},
pubstate = {published},
tppubtype = {inproceedings}
}
A wide number of problems in face and gesture analysis involve the labeling of temporal sequences. In this paper, we introduce a discriminative model for such sequence labeling tasks. This model involves two layers of latent dynamics, each with their separate roles. The first layer, the neural network or gating layer, aims to extract non-linear relationships between input data and output labels. The second layer, the hidden-states layer, aims to model temporal substructure in the sequence by learning hidden-states and their transition dynamics. A new regularization term is proposed for the training of this model, encouraging diversity between hidden-states. We evaluate the performance of this model on an audiovisual dataset of emotion recognition and compare it against other popular methods for sequence labeling.⬚⬚
Kang, Sin-Hwa; Morie, Jacquelyn
Users’ Socially Desirable Responding with Computer Interviewers Proceedings Article
In: Computer-Human Interaction Conference, Paris, France, 2013.
Abstract | Links | BibTeX | Tags: UARC, Virtual Humans, Virtual Worlds
@inproceedings{kang_users_2013,
title = {Users’ Socially Desirable Responding with Computer Interviewers},
author = {Sin-Hwa Kang and Jacquelyn Morie},
url = {http://ict.usc.edu/pubs/Users%E2%80%99%20Socially%20Desirable%20Responding%20with%20Computer%20Interviewers.pdf},
year = {2013},
date = {2013-04-01},
booktitle = {Computer-Human Interaction Conference},
address = {Paris, France},
abstract = {In this paper, we explore how different types of computer interviewers and the amount of self- disclosure from the interviewers affect the quantity of socially desirable responses displayed by interviewees. Online surveys were delivered by computer interviewers. The computer interviewers included a text-based interface and an anthropomorphic character interface. The interviewers’ self-disclosure presented their social norm violations. Interview questions were in the form of socially desirable response items representing impression management in this study. The experimental design was a 2 (Interviewers’ type) x 2 (Interviewers’ self-disclosure versus no self-disclosure) factorial between-subjects experiment. The main dependent variable was whether users’ socially desirable responses were affected by the type of interviewer and that amount of self-disclosure provided by the interviewer. The preliminary findings present the potential for self-disclosing anthropomorphic characters to reduce the social desirability bias present in interviewees with high public self-consciousness in their self-disclosure.},
keywords = {UARC, Virtual Humans, Virtual Worlds},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we explore how different types of computer interviewers and the amount of self- disclosure from the interviewers affect the quantity of socially desirable responses displayed by interviewees. Online surveys were delivered by computer interviewers. The computer interviewers included a text-based interface and an anthropomorphic character interface. The interviewers’ self-disclosure presented their social norm violations. Interview questions were in the form of socially desirable response items representing impression management in this study. The experimental design was a 2 (Interviewers’ type) x 2 (Interviewers’ self-disclosure versus no self-disclosure) factorial between-subjects experiment. The main dependent variable was whether users’ socially desirable responses were affected by the type of interviewer and that amount of self-disclosure provided by the interviewer. The preliminary findings present the potential for self-disclosing anthropomorphic characters to reduce the social desirability bias present in interviewees with high public self-consciousness in their self-disclosure.
Jones, J. Adam; II, J. Edward Swan; Bolas, Mark
Peripheral Stimulation and its Effect on Perceived Spatial Scale in Virtual Environments Journal Article
In: IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 4, pp. 701–710, 2013.
Abstract | Links | BibTeX | Tags: MxR, UARC
@article{jones_peripheral_2013,
title = {Peripheral Stimulation and its Effect on Perceived Spatial Scale in Virtual Environments},
author = {J. Adam Jones and J. Edward Swan II and Mark Bolas},
url = {http://ict.usc.edu/pubs/Peripheral%20Stimulation%20and%20its%20Effect%20on%20Perceived%20Spatial%20Scale%20in%20Virtual%20Environments.pdf},
doi = {10.1109/TVCG.2013.37},
year = {2013},
date = {2013-04-01},
journal = {IEEE Transactions on Visualization and Computer Graphics},
volume = {19},
number = {4},
pages = {701–710},
abstract = {The following series of experiments explore the effect of static peripheral stimulation on the perception of distance and spatial scale in a typical head-mounted virtual environment. It was found that applying constant white light in an observer’s far periphery enabled the observer to more accurately judge distances using blind walking. An effect of similar magnitude was also found when observers estimated the size of a virtual space using a visual scale task. The presence of the effect across multiple psychophysical tasks provided confidence that a perceptual change was, in fact, being invoked by the addition of the peripheral stimulation. These results were also compared to observer performance in a very large field of view virtual environment and in the real world. The subsequent findings raise the possibility that distance judgments in virtual environments might be considerably more similar to those in the real world than previous work has suggested.},
keywords = {MxR, UARC},
pubstate = {published},
tppubtype = {article}
}
The following series of experiments explore the effect of static peripheral stimulation on the perception of distance and spatial scale in a typical head-mounted virtual environment. It was found that applying constant white light in an observer’s far periphery enabled the observer to more accurately judge distances using blind walking. An effect of similar magnitude was also found when observers estimated the size of a virtual space using a visual scale task. The presence of the effect across multiple psychophysical tasks provided confidence that a perceptual change was, in fact, being invoked by the addition of the peripheral stimulation. These results were also compared to observer performance in a very large field of view virtual environment and in the real world. The subsequent findings raise the possibility that distance judgments in virtual environments might be considerably more similar to those in the real world than previous work has suggested.