Publications
Search
581.
Feng, Andrew; Huang, Yazhou; Xu, Yuyu; Shapiro, Ari
Fast, automatic character animation pipelines Journal Article
In: Computer Animation and Virtual Worlds, vol. 25, no. 1, pp. 3–16, 2014, ISSN: 15464261.
@article{feng_fast_2014,
title = {Fast, automatic character animation pipelines},
author = {Andrew Feng and Yazhou Huang and Yuyu Xu and Ari Shapiro},
url = {http://ict.usc.edu/pubs/Fast,%20automatic%20character%20animation%20pipelines.pdf},
doi = {10.1002/cav.1560},
issn = {15464261},
year = {2014},
date = {2014-01-01},
journal = {Computer Animation and Virtual Worlds},
volume = {25},
number = {1},
pages = {3–16},
abstract = {Humanoid three-dimensional (3D) models can be easily acquired through various sources, including through online marketplaces. The use of such models within a game or simulation environment requires human input and intervention in order to associate such a model with a relevant set of motions and control mechanisms. In this paper, we demonstrate a pipeline where humanoid 3D models can be incorporated within seconds into an animation system and infused with a wide range of capabilities, such as locomotion, object manipulation, gazing, speech synthesis and lip syncing. We offer a set of heuristics that can associate arbitrary joint names with canonical ones and describe a fast retargeting algorithm that enables us to instill a set of behaviors onto an arbitrary humanoid skeleton on-the-fly. We believe that such a system will vastly increase the use of 3D interactive characters due to the ease that new models can be animated. Copyright © 2013 John Wiley & Sons, Ltd.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Humanoid three-dimensional (3D) models can be easily acquired through various sources, including through online marketplaces. The use of such models within a game or simulation environment requires human input and intervention in order to associate such a model with a relevant set of motions and control mechanisms. In this paper, we demonstrate a pipeline where humanoid 3D models can be incorporated within seconds into an animation system and infused with a wide range of capabilities, such as locomotion, object manipulation, gazing, speech synthesis and lip syncing. We offer a set of heuristics that can associate arbitrary joint names with canonical ones and describe a fast retargeting algorithm that enables us to instill a set of behaviors onto an arbitrary humanoid skeleton on-the-fly. We believe that such a system will vastly increase the use of 3D interactive characters due to the ease that new models can be animated. Copyright © 2013 John Wiley & Sons, Ltd.
582.
Gratch, Jonathan; Kang, Sin-Hwa; Wang, Ning
Using Social Agents to Explore Theories of Rapport and Emotional Resonance Book Section
In: Social Emotions in Nature and Artifact, pp. 181 –195, 2014.
@incollection{gratch_using_2014,
title = {Using Social Agents to Explore Theories of Rapport and Emotional Resonance},
author = {Jonathan Gratch and Sin-Hwa Kang and Ning Wang},
url = {http://www.oxfordscholarship.com/view/10.1093/acprof:oso/9780195387643.001.0001/acprof-9780195387643-chapter-12},
year = {2014},
date = {2014-01-01},
booktitle = {Social Emotions in Nature and Artifact},
pages = {181 –195},
abstract = {We discuss several technical challenges must be overcome before realizing this vision. More importantly, success depends not on simply overcoming these challenges, but demonstrating that such interactivity has measurable and desirable consequences for human-computer interaction. In this chapter, we describe the Rapport Agent, an interactive agent and methodological tool designed to Emotions unfold in with bewildering complexity in face-to-face social interactions. Building computer programs that can engage people in this unfolding emotional dance is a fascinating prospect with potentially profound practical and scientific consequences. Computer agents that engage people in this manner could enhance our understanding of this fundamental social process and, more practically, have dramatic implications investigate the role of nonverbal patterning in human-computer and computer-mediated interaction. We outline a series of laboratory studies and resulting findings that give insight into how nonverbal patterns of behavior can influence both subjective perceptions (such as feelings of rapport or embarrassment) and ehavioural outcomes (such as speech fluency or intimate self-disclosure).},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
We discuss several technical challenges must be overcome before realizing this vision. More importantly, success depends not on simply overcoming these challenges, but demonstrating that such interactivity has measurable and desirable consequences for human-computer interaction. In this chapter, we describe the Rapport Agent, an interactive agent and methodological tool designed to Emotions unfold in with bewildering complexity in face-to-face social interactions. Building computer programs that can engage people in this unfolding emotional dance is a fascinating prospect with potentially profound practical and scientific consequences. Computer agents that engage people in this manner could enhance our understanding of this fundamental social process and, more practically, have dramatic implications investigate the role of nonverbal patterning in human-computer and computer-mediated interaction. We outline a series of laboratory studies and resulting findings that give insight into how nonverbal patterns of behavior can influence both subjective perceptions (such as feelings of rapport or embarrassment) and ehavioural outcomes (such as speech fluency or intimate self-disclosure).
583.
Faust, Lauren; Artstein, Ron
People hesitate more, talk less to virtual interviewers than to human interviewers Proceedings Article
In: Proceedings of the 17th Workshop on the Semantics and Pragmatics of Dialogue, Amsterdam, 2013.
@inproceedings{faust_people_2013,
title = {People hesitate more, talk less to virtual interviewers than to human interviewers},
author = {Lauren Faust and Ron Artstein},
url = {http://ict.usc.edu/pubs/People%20hesitate%20more,%20talk%20less%20to%20virtual%20interviewers%20than%20to%20human%20interviewers.pdf},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of the 17th Workshop on the Semantics and Pragmatics of Dialogue},
address = {Amsterdam},
abstract = {In a series of screening interviews for psychological distress, conducted separately by a human interviewer and by an animated virtual character controlled by a human, participants talked substantially less and produced twice as many filled pauses when talking to the virtual character. This contrasts with earlier findings, where people were less disfluent when talking to a computer dialogue system. The results suggest that the characteristics of computer-directed speech vary depending on the type of dialogue system used.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In a series of screening interviews for psychological distress, conducted separately by a human interviewer and by an animated virtual character controlled by a human, participants talked substantially less and produced twice as many filled pauses when talking to the virtual character. This contrasts with earlier findings, where people were less disfluent when talking to a computer dialogue system. The results suggest that the characteristics of computer-directed speech vary depending on the type of dialogue system used.
584.
Yu, Zhou; Scherer, Stefen; Devault, David; Gratch, Jonathan; Stratou, Giota; Morency, Louis-Philippe; Cassell, Justine
Multimodal Prediction of Psychological Disorders: Learning Verbal and Nonverbal Commonalities in Adjacency Pairs Proceedings Article
In: Semdial 2013 DialDam: Proceedings of the 17th Workshop on the Semantics and Pragmatics of Dialogue, pp. 160–169, Amsterdam, The Netherlands, 2013.
@inproceedings{yu_multimodal_2013,
title = {Multimodal Prediction of Psychological Disorders: Learning Verbal and Nonverbal Commonalities in Adjacency Pairs},
author = {Zhou Yu and Stefen Scherer and David Devault and Jonathan Gratch and Giota Stratou and Louis-Philippe Morency and Justine Cassell},
url = {http://www.cs.cmu.edu/afs/cs/user/zhouyu/www/semdial_2013_zhou.pdf},
year = {2013},
date = {2013-12-01},
booktitle = {Semdial 2013 DialDam: Proceedings of the 17th Workshop on the Semantics and Pragmatics of Dialogue},
pages = {160–169},
address = {Amsterdam, The Netherlands},
abstract = {Semi-structured interviews are widely used in medical settings to gather information from individuals about psychological disorders, such as depression or anxiety. These interviews typically consist of a series of question and response pairs, which we refer to as adjacency pairs. We pro-pose a computational model, the Multi-modal HCRF, that considers the commonalities among adjacency pairs and information from multiple modalities to infer the psychological states of the interviewees. We collect data and perform experiments on a human to virtual human interaction data set. Our multimodal approach gives a significant advantage over conventional holistic approaches which ignore the adjacency pair context in predicting depression from semi-structured inter- views.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Semi-structured interviews are widely used in medical settings to gather information from individuals about psychological disorders, such as depression or anxiety. These interviews typically consist of a series of question and response pairs, which we refer to as adjacency pairs. We pro-pose a computational model, the Multi-modal HCRF, that considers the commonalities among adjacency pairs and information from multiple modalities to infer the psychological states of the interviewees. We collect data and perform experiments on a human to virtual human interaction data set. Our multimodal approach gives a significant advantage over conventional holistic approaches which ignore the adjacency pair context in predicting depression from semi-structured inter- views.
585.
Baltrušaitis, Tadas; Robinson, Peter; Morency, Louis-Philippe
Constrained local neural fields for robust facial landmark detection in the wild Proceedings Article
In: Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on, pp. 354–361, IEEE, Sydney, Australia, 2013.
@inproceedings{baltrusaitis_constrained_2013,
title = {Constrained local neural fields for robust facial landmark detection in the wild},
author = {Tadas Baltrušaitis and Peter Robinson and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Constrained%20local%20neural%20fields%20for%20robust%20facial%20landmark%20detection%20in%20the%20wild.pdf},
year = {2013},
date = {2013-12-01},
booktitle = {Computer Vision Workshops (ICCVW), 2013 IEEE International Conference on},
pages = {354–361},
publisher = {IEEE},
address = {Sydney, Australia},
abstract = {Facial feature detection algorithms have seen great progress over the recent years. However, they still struggle in poor lighting conditions and in the presence of extreme pose or occlusions. We present the Constrained Local Neural Field model for facial landmark detection. Our model includes two main novelties. First, we introduce a probabilistic patch expert (landmark detector) that can learn non-linear and spatial relationships between the input pixels and the probability of a landmark being aligned. Secondly, our model is optimised using a novel Non-uniform Regularised Landmark Mean-Shift optimisation technique, which takes into account the reliabilities of each patch expert. We demonstrate the benefit of our approach on a number of publicly available datasets over other state-of-the-art approaches when performing landmark detection in unseen lighting conditions and in the wild.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Facial feature detection algorithms have seen great progress over the recent years. However, they still struggle in poor lighting conditions and in the presence of extreme pose or occlusions. We present the Constrained Local Neural Field model for facial landmark detection. Our model includes two main novelties. First, we introduce a probabilistic patch expert (landmark detector) that can learn non-linear and spatial relationships between the input pixels and the probability of a landmark being aligned. Secondly, our model is optimised using a novel Non-uniform Regularised Landmark Mean-Shift optimisation technique, which takes into account the reliabilities of each patch expert. We demonstrate the benefit of our approach on a number of publicly available datasets over other state-of-the-art approaches when performing landmark detection in unseen lighting conditions and in the wild.
586.
Kok, Iwan; Heylen, Dirk; Morency, Louis-Philippe
Speaker-Adaptive Multimodal Prediction Model for Listener Responses Proceedings Article
In: pp. 51–58, ACM Press, Sydney, Australia, 2013, ISBN: 978-1-4503-2129-7.
@inproceedings{de_kok_speaker-adaptive_2013,
title = {Speaker-Adaptive Multimodal Prediction Model for Listener Responses},
author = {Iwan Kok and Dirk Heylen and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Speaker-adaptive%20multimodal%20prediction%20model%20for%20listener%20responses.pdf},
doi = {10.1145/2522848.2522866},
isbn = {978-1-4503-2129-7},
year = {2013},
date = {2013-12-01},
pages = {51–58},
publisher = {ACM Press},
address = {Sydney, Australia},
abstract = {The goal of this paper is to acknowledge and model the variability in speaking styles in dyadic interactions and build a predictive algorithm for listener responses that is able to adapt to these different styles. The end result of this research will be a virtual human able to automatically respond to a human speaker with proper listener responses (e.g., head nods). Our novel speaker-adaptive prediction model is created from a corpus of dyadic interactions where speaker variability is analyzed to identify a subset of prototypical speaker styles. During a live interaction our prediction model automatically identifies the closest prototypical speaker style and predicts listener responses based on this communicative style. Central to our approach is the idea of "speaker profile" which uniquely identify each speaker and enables the matching between prototypical speakers and new speakers. The paper shows the merits of our speaker-adaptive listener response prediction model by showing improvement over a state-of-the-art approach which does not adapt to the speaker. Besides the merits of speaker-adaptation, our experiments highlights the importance of using multimodal features when comparing speakers to select the closest prototypical speaker style.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
The goal of this paper is to acknowledge and model the variability in speaking styles in dyadic interactions and build a predictive algorithm for listener responses that is able to adapt to these different styles. The end result of this research will be a virtual human able to automatically respond to a human speaker with proper listener responses (e.g., head nods). Our novel speaker-adaptive prediction model is created from a corpus of dyadic interactions where speaker variability is analyzed to identify a subset of prototypical speaker styles. During a live interaction our prediction model automatically identifies the closest prototypical speaker style and predicts listener responses based on this communicative style. Central to our approach is the idea of "speaker profile" which uniquely identify each speaker and enables the matching between prototypical speakers and new speakers. The paper shows the merits of our speaker-adaptive listener response prediction model by showing improvement over a state-of-the-art approach which does not adapt to the speaker. Besides the merits of speaker-adaptation, our experiments highlights the importance of using multimodal features when comparing speakers to select the closest prototypical speaker style.
587.
Mahmoud, Marwa; Morency, Louis-Philippe; Robinson, Peter
Automatic Multimodal Descriptors of Rhythmic Body Movement Proceedings Article
In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 429–436, ACM, 2013.
@inproceedings{mahmoud_automatic_2013,
title = {Automatic Multimodal Descriptors of Rhythmic Body Movement},
author = {Marwa Mahmoud and Louis-Philippe Morency and Peter Robinson},
url = {http://ict.usc.edu/pubs/Automatic%20multimodal%20descriptors%20of%20rhythmic%20body%20movement.pdf},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of the 15th ACM on International conference on multimodal interaction},
pages = {429–436},
publisher = {ACM},
abstract = {Prolonged durations of rhythmic body gestures were proved to be correlated with different types of psychological disorders. To-date, there is no automatic descriptor that can robustly detect those behaviours. In this paper, we propose a cyclic gestures descriptor that can detect and localise rhythmic body movements by taking advantage of both colour and depth modalities. We show experimentally how our rhythmic descriptor can successfully localise the rhythmic gestures as: hands fidgeting, legs fidgeting or rocking, significantly higher than the majority vote classification baseline. Our experiments also demonstrate the importance of fusing both modalities, with a significant increase in performance when compared to individual modalities.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Prolonged durations of rhythmic body gestures were proved to be correlated with different types of psychological disorders. To-date, there is no automatic descriptor that can robustly detect those behaviours. In this paper, we propose a cyclic gestures descriptor that can detect and localise rhythmic body movements by taking advantage of both colour and depth modalities. We show experimentally how our rhythmic descriptor can successfully localise the rhythmic gestures as: hands fidgeting, legs fidgeting or rocking, significantly higher than the majority vote classification baseline. Our experiments also demonstrate the importance of fusing both modalities, with a significant increase in performance when compared to individual modalities.
588.
Mohammadi, Gelareh; Park, Sunghyun; Sagae, Kenji; Vinciarelli, Alessandro; Morency, Louis-Philippe
Who Is Persuasive? The Role of Perceived Personality and Communication Modality in Social Multimedia Proceedings Article
In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 19–26, ACM Press, New York, NY, 2013, ISBN: 978-1-4503-2129-7.
@inproceedings{mohammadi_who_2013,
title = {Who Is Persuasive? The Role of Perceived Personality and Communication Modality in Social Multimedia},
author = {Gelareh Mohammadi and Sunghyun Park and Kenji Sagae and Alessandro Vinciarelli and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Who%20is%20persuasive%20-%20the%20role%20of%20perceived%20personality%20and%20communication%20modality%20in%20social%20multimedia.pdf},
doi = {10.1145/2522848.2522857},
isbn = {978-1-4503-2129-7},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of the 15th ACM on International conference on multimodal interaction},
pages = {19–26},
publisher = {ACM Press},
address = {New York, NY},
abstract = {Persuasive communication is part of everyone's daily life. With the emergence of social websites like YouTube, Facebook and Twitter, persuasive communication is now seen online on a daily basis. This paper explores the effect of multi-modality and perceived personality on persuasiveness of social multimedia content. The experiments are performed over a large corpus of movie review clips from Youtube which is presented to online annotators in three different modalities: only text, only audio and video. The annotators evaluated the persuasiveness of each review across different modalities and judged the personality of the speaker. Our detailed analysis confirmed several research hypotheses designed to study the relationships between persuasion, perceived personality and communicative channel, namely modality. Three hypotheses are designed: the first hypothesis studies the effect of communication modality on persuasion, the second hypothesis examines the correlation between persuasion and personality perception and finally the third hypothesis, derived from the first two hypotheses explores how communication modality influence the personality perception.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Persuasive communication is part of everyone's daily life. With the emergence of social websites like YouTube, Facebook and Twitter, persuasive communication is now seen online on a daily basis. This paper explores the effect of multi-modality and perceived personality on persuasiveness of social multimedia content. The experiments are performed over a large corpus of movie review clips from Youtube which is presented to online annotators in three different modalities: only text, only audio and video. The annotators evaluated the persuasiveness of each review across different modalities and judged the personality of the speaker. Our detailed analysis confirmed several research hypotheses designed to study the relationships between persuasion, perceived personality and communicative channel, namely modality. Three hypotheses are designed: the first hypothesis studies the effect of communication modality on persuasion, the second hypothesis examines the correlation between persuasion and personality perception and finally the third hypothesis, derived from the first two hypotheses explores how communication modality influence the personality perception.
589.
Miller, Chreston; Quek, Francis; Morency, Louis-Philippe
Interactive Relevance Search and Modeling: Support for Expert-Driven Analysis of Multimodal Data Proceedings Article
In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 149–156, ACM, Sydney, Australia, 2013.
@inproceedings{miller_interactive_2013,
title = {Interactive Relevance Search and Modeling: Support for Expert-Driven Analysis of Multimodal Data},
author = {Chreston Miller and Francis Quek and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Interactive%20relevance%20search%20and%20modeling%20-%20Support%20for%20expert-driven%20analysis%20of%20multimodal%20data.pdf},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of the 15th ACM on International conference on multimodal interaction},
pages = {149–156},
publisher = {ACM},
address = {Sydney, Australia},
abstract = {In this paper we present the findings of three longitudinal case studies in which a new method for conducting multimodal analysis of human behavior is tested. The focus of this new method is to engage a researcher integrally in the analysis process and allow them to guide the identification and discovery of relevant behavior instances within multimodal data. The case studies resulted in the creation of two analysis strategies: Single-Focus Hypothesis Testing and Multi-Focus Hypothesis Testing. Each were shown to be beneficial to multimodal analysis through supporting either a single focused deep analysis or analysis across multiple angles in unison. These strategies exemplified how challenging questions can be answered for multimodal datasets. The new method is described and the case studies’ findings are presented detailing how the new method supports multimodal analysis and opens the door for a new breed of analysis methods. Two of the three case studies resulted in publishable results for the respective participants.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper we present the findings of three longitudinal case studies in which a new method for conducting multimodal analysis of human behavior is tested. The focus of this new method is to engage a researcher integrally in the analysis process and allow them to guide the identification and discovery of relevant behavior instances within multimodal data. The case studies resulted in the creation of two analysis strategies: Single-Focus Hypothesis Testing and Multi-Focus Hypothesis Testing. Each were shown to be beneficial to multimodal analysis through supporting either a single focused deep analysis or analysis across multiple angles in unison. These strategies exemplified how challenging questions can be answered for multimodal datasets. The new method is described and the case studies’ findings are presented detailing how the new method supports multimodal analysis and opens the door for a new breed of analysis methods. Two of the three case studies resulted in publishable results for the respective participants.
590.
Scherer, Stefan; Stratou, Giota; Morency, Louis-Philippe
Audiovisual Behavior Descriptors for Depression Assessment Proceedings Article
In: Proceedings of ICMI'13, pp. 135–140, ACM Press, Sydney, Australia, 2013, ISBN: 978-1-4503-2129-7.
@inproceedings{scherer_audiovisual_2013,
title = {Audiovisual Behavior Descriptors for Depression Assessment},
author = {Stefan Scherer and Giota Stratou and Louis-Philippe Morency},
url = {http://ict.usc.edu/pubs/Audiovisual%20behavior%20descriptors%20for%20depression%20assessment.pdf},
doi = {10.1145/2522848.2522886},
isbn = {978-1-4503-2129-7},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of ICMI'13},
pages = {135–140},
publisher = {ACM Press},
address = {Sydney, Australia},
abstract = {We investigate audiovisual indicators, in particular measures of reduced emotional expressivity and psycho-motor retardation, for depression within semi-structured virtual human interviews. Based on a standard self-assessment depression scale we investigate the statistical discriminative strength of the audiovisual features on a depression/no-depression basis. Within subject-independent unimodal and multimodal classification experiments we find that early feature-level fusion yields promising results and confirms the statistical findings. We further correlate the behavior descriptors with the assessed depression severity and find considerable correlation. Lastly, a joint multimodal factor analysis reveals two prominent factors within the data that show both statistical discriminative power as well as strong linear correlation with the depression severity score. These preliminary results based on a standard factor analysis are promising and motivate us to investigate this approach further in the future, while incorporating additional modalities.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We investigate audiovisual indicators, in particular measures of reduced emotional expressivity and psycho-motor retardation, for depression within semi-structured virtual human interviews. Based on a standard self-assessment depression scale we investigate the statistical discriminative strength of the audiovisual features on a depression/no-depression basis. Within subject-independent unimodal and multimodal classification experiments we find that early feature-level fusion yields promising results and confirms the statistical findings. We further correlate the behavior descriptors with the assessed depression severity and find considerable correlation. Lastly, a joint multimodal factor analysis reveals two prominent factors within the data that show both statistical discriminative power as well as strong linear correlation with the depression severity score. These preliminary results based on a standard factor analysis are promising and motivate us to investigate this approach further in the future, while incorporating additional modalities.
591.
Song, Yale; Morency, Louis-Philippe; Davis, Randall
Learning a Sparse Codebook of Facial and Body Microexpressions for Emotion Recognition Proceedings Article
In: Proceedings of the 15th ACM on International conference on multimodal interaction, pp. 237–244, ACM Press, 2013, ISBN: 978-1-4503-2129-7.
@inproceedings{song_learning_2013,
title = {Learning a Sparse Codebook of Facial and Body Microexpressions for Emotion Recognition},
author = {Yale Song and Louis-Philippe Morency and Randall Davis},
url = {http://ict.usc.edu/pubs/Learning%20a%20sparse%20codebook%20of%20facial%20and%20body%20microexpressions%20for%20emotion%20recognition.pdf},
doi = {10.1145/2522848.2522851},
isbn = {978-1-4503-2129-7},
year = {2013},
date = {2013-12-01},
booktitle = {Proceedings of the 15th ACM on International conference on multimodal interaction},
pages = {237–244},
publisher = {ACM Press},
abstract = {Obtaining a compact and discriminative representation of facial and body expressions is a difficult problem in emotion recognition. Part of the difficulty is capturing microexpressions, i.e., short, involuntary expressions that last for only a fraction of a second: at a micro-temporal scale, there are so many other subtle face and body movements that do not convey semantically meaningful information. We present a novel approach to this problem by exploiting the sparsity of the frequent micro-temporal motion patterns. Local space-time features are extracted over the face and body region for a very short time period, e.g., few milliseconds. A codebook of microexpressions is learned from the data and used to encode the features in a sparse manner. This allows us to obtain a representation that captures the most salient motion patterns of the face and body at a micro-temporal scale. Experiments performed on the AVEC 2012 dataset show our approach achieving the best published performance on the expectation dimension based solely on visual features. We also report experimental results on audio-visual emotion recognition, comparing early and late data fusion techniques.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Obtaining a compact and discriminative representation of facial and body expressions is a difficult problem in emotion recognition. Part of the difficulty is capturing microexpressions, i.e., short, involuntary expressions that last for only a fraction of a second: at a micro-temporal scale, there are so many other subtle face and body movements that do not convey semantically meaningful information. We present a novel approach to this problem by exploiting the sparsity of the frequent micro-temporal motion patterns. Local space-time features are extracted over the face and body region for a very short time period, e.g., few milliseconds. A codebook of microexpressions is learned from the data and used to encode the features in a sparse manner. This allows us to obtain a representation that captures the most salient motion patterns of the face and body at a micro-temporal scale. Experiments performed on the AVEC 2012 dataset show our approach achieving the best published performance on the expectation dimension based solely on visual features. We also report experimental results on audio-visual emotion recognition, comparing early and late data fusion techniques.
592.
Xu, Yuyu; Feng, Andrew W.; Marsella, Stacy C.; Shapiro, Ari
A Practical and Configurable Lip Sync Method for Games Proceedings Article
In: ACM SIGGRAPH Motion in Games, Dublin, Ireland, 2013.
@inproceedings{xu_practical_2013,
title = {A Practical and Configurable Lip Sync Method for Games},
author = {Yuyu Xu and Andrew W. Feng and Stacy C. Marsella and Ari Shapiro},
url = {http://ict.usc.edu/pubs/A%20Practical%20and%20Configurable%20Lip%20Sync%20Method%20for%20Games.pdf},
year = {2013},
date = {2013-11-01},
booktitle = {ACM SIGGRAPH Motion in Games},
address = {Dublin, Ireland},
abstract = {We demonstrate a lip animation (lip sync) algorithm for real-time applications that can be used to generate synchronized facial movements with audio generated from natural speech or a text-to-speech engine. Our method requires an animator to construct animations using a canonical set of visemes for all pairwise combinations of a reduced phoneme set (phone bigrams). These animations are then stitched together to construct the final animation, adding velocity and lip-pose constraints. This method can be applied to any character that uses the same, small set of visemes. Our method can operate efficiently in multiple languages by reusing phone bigram animations that are shared among languages, and specific word sounds can be identified and changed on a per-character basis. Our method uses no machine learning, which offers two advantages over techniques that do: 1) data can be generated for non-human characters whose faces can not be easily retargeted from a human speaker’s face, and 2) the specific facial poses or shapes used for animation can be specified during the setup and rigging stage, and before the lip animation stage, thus making it suitable for game pipelines or circumstances where the speech targets poses are predetermined, such as after acquisition from an online 3D marketplace.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
We demonstrate a lip animation (lip sync) algorithm for real-time applications that can be used to generate synchronized facial movements with audio generated from natural speech or a text-to-speech engine. Our method requires an animator to construct animations using a canonical set of visemes for all pairwise combinations of a reduced phoneme set (phone bigrams). These animations are then stitched together to construct the final animation, adding velocity and lip-pose constraints. This method can be applied to any character that uses the same, small set of visemes. Our method can operate efficiently in multiple languages by reusing phone bigram animations that are shared among languages, and specific word sounds can be identified and changed on a per-character basis. Our method uses no machine learning, which offers two advantages over techniques that do: 1) data can be generated for non-human characters whose faces can not be easily retargeted from a human speaker’s face, and 2) the specific facial poses or shapes used for animation can be specified during the setup and rigging stage, and before the lip animation stage, thus making it suitable for game pipelines or circumstances where the speech targets poses are predetermined, such as after acquisition from an online 3D marketplace.
593.
Gratch, Jonathan; Marsella, Stacy C. (Ed.)
Social Emotions in Nature and Artifact Book
Oxford University Press, 2013, ISBN: 0-19-538764-3.
@book{gratch_social_2013,
title = {Social Emotions in Nature and Artifact},
editor = {Jonathan Gratch and Stacy C. Marsella},
url = {http://www.amazon.com/Social-Emotions-Artifact-Cognitive-Architectures/dp/0195387643},
isbn = {0-19-538764-3},
year = {2013},
date = {2013-11-01},
publisher = {Oxford University Press},
series = {Oxford Series on Cognitive Models and Architectures},
abstract = {Recent years have seen the rise of a remarkable partnership between the social and computational sciences on the phenomena of emotions. Rallying around the term Affective Computing, this research can be seen as revival of the cognitive science revolution, albeit garbed in the cloak of affect, rather than cognition. Traditional cognitive science research, to the extent it considered emotion at all, cases it as at best a heuristic but more commonly a harmful bias to cognition. More recent scholarship in the social sciences has upended this view. Increasingly, emotions are viewed as a form of information processing that serves a functional role in human cognition and social interactions. Emotions shape social motives and communicate important information to social partners. When communicating face-to-face, people can rapidly detect nonverbal affective cues, make inferences about the other party's mental state, and respond in ways that co-construct an emotional trajectory between participants. Recent advances in biometrics and artificial intelligence are allowing computer systems to engage in this nonverbal dance, on the one hand opening a wealth of possibilities for human-machine systems, and on the other, creating powerful new tools for behavioral science research. Social Emotions in Nature and Artifact reports on the state-of-the-art in both social science theory and computational methods, and illustrates how these two fields, together, can both facilitate practical computer/robotic applications and illuminate human social processes.},
keywords = {},
pubstate = {published},
tppubtype = {book}
}
Recent years have seen the rise of a remarkable partnership between the social and computational sciences on the phenomena of emotions. Rallying around the term Affective Computing, this research can be seen as revival of the cognitive science revolution, albeit garbed in the cloak of affect, rather than cognition. Traditional cognitive science research, to the extent it considered emotion at all, cases it as at best a heuristic but more commonly a harmful bias to cognition. More recent scholarship in the social sciences has upended this view. Increasingly, emotions are viewed as a form of information processing that serves a functional role in human cognition and social interactions. Emotions shape social motives and communicate important information to social partners. When communicating face-to-face, people can rapidly detect nonverbal affective cues, make inferences about the other party's mental state, and respond in ways that co-construct an emotional trajectory between participants. Recent advances in biometrics and artificial intelligence are allowing computer systems to engage in this nonverbal dance, on the one hand opening a wealth of possibilities for human-machine systems, and on the other, creating powerful new tools for behavioral science research. Social Emotions in Nature and Artifact reports on the state-of-the-art in both social science theory and computational methods, and illustrates how these two fields, together, can both facilitate practical computer/robotic applications and illuminate human social processes.
594.
Lane, H. Chad; Hays, Matthew Jensen; Core, Mark G.; Auerbach, Daniel
Learning intercultural communication skills with virtual humans: Feedback and fidelity. Journal Article
In: Journal of Educational Psychology, vol. 105, no. 4, pp. 1026–1035, 2013, ISSN: 1939-2176, 0022-0663.
@article{lane_learning_2013,
title = {Learning intercultural communication skills with virtual humans: Feedback and fidelity.},
author = {H. Chad Lane and Matthew Jensen Hays and Mark G. Core and Daniel Auerbach},
url = {http://doi.apa.org/getdoi.cfm?doi=10.1037/a0031506},
doi = {10.1037/a0031506},
issn = {1939-2176, 0022-0663},
year = {2013},
date = {2013-11-01},
journal = {Journal of Educational Psychology},
volume = {105},
number = {4},
pages = {1026–1035},
abstract = {In the context of practicing intercultural communication skills, we investigated the role of fidelity in a game-based, virtual learning environment as well as the role of feedback delivered by an intelligent tutoring system. In 2 experiments, we compared variations on the game interface, use of the tutoring system, and the form of the feedback. Our findings suggest that for learning basic intercultural communicative skills, a 3-dimensional (3-D) interface with animation and sound produced equivalent learning to a more static 2-D interface. However, learners took significantly longer to analyze and respond to the actions of animated virtual humans, suggesting a deeper engagement. We found large gains in learning across conditions. There was no differential effect with the tutor engaged, but it was found to have a positive impact on learner success in a transfer task. This difference was most pronounced when the feedback was delivered in a more general form versus a concrete style.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In the context of practicing intercultural communication skills, we investigated the role of fidelity in a game-based, virtual learning environment as well as the role of feedback delivered by an intelligent tutoring system. In 2 experiments, we compared variations on the game interface, use of the tutoring system, and the form of the feedback. Our findings suggest that for learning basic intercultural communicative skills, a 3-dimensional (3-D) interface with animation and sound produced equivalent learning to a more static 2-D interface. However, learners took significantly longer to analyze and respond to the actions of animated virtual humans, suggesting a deeper engagement. We found large gains in learning across conditions. There was no differential effect with the tutor engaged, but it was found to have a positive impact on learner success in a transfer task. This difference was most pronounced when the feedback was delivered in a more general form versus a concrete style.
595.
Nouri, Elnaz
Does History Help? An Experiment on How Context Affects Crowdsourcing Dialogue Annotation Proceedings Article
In: Proceedings of the Human Computation Workshop on Scaling Speech, Language Understanding and Dialogue through Crowdsourcing, Palm Springs, CA, 2013.
@inproceedings{nouri_does_2013,
title = {Does History Help? An Experiment on How Context Affects Crowdsourcing Dialogue Annotation},
author = {Elnaz Nouri},
url = {http://ict.usc.edu/pubs/Does%20History%20Help%20-%20An%20Experiment%20on%20How%20Context%20Affects%20Crowdsourcing%20Dialogue%20Annotation.pdf},
year = {2013},
date = {2013-11-01},
booktitle = {Proceedings of the Human Computation Workshop on Scaling Speech, Language Understanding and Dialogue through Crowdsourcing},
address = {Palm Springs, CA},
abstract = {Crowds of people can potentially solve some problems faster than individuals. Crowd sourced data can be leveraged to benefit the crowd by providing information or solutions faster than traditional means. Many tasks needed for developing dialogue systems such as annotation can benefit from crowdsourcing as well. We investigate how to outsource dialogue data annotation through Amazon Mechanical Turk. We are in particular interested in empirically analyzing how much context from previous parts of the dialogue (e.g. previous dialogue turns) is needed to be provided before the target part (dialogue turn) is presented to the annotator. The answer to this question is essentially important for leveraging crowd sourced data for appropriate and efficient response and coordination. We study the effect of presenting different numbers of previous data (turns) to the Turkers in annotating sentiments of dyadic negotiation dialogs on the inter annotator reliability and comparison to the gold standard.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
Crowds of people can potentially solve some problems faster than individuals. Crowd sourced data can be leveraged to benefit the crowd by providing information or solutions faster than traditional means. Many tasks needed for developing dialogue systems such as annotation can benefit from crowdsourcing as well. We investigate how to outsource dialogue data annotation through Amazon Mechanical Turk. We are in particular interested in empirically analyzing how much context from previous parts of the dialogue (e.g. previous dialogue turns) is needed to be provided before the target part (dialogue turn) is presented to the annotator. The answer to this question is essentially important for leveraging crowd sourced data for appropriate and efficient response and coordination. We study the effect of presenting different numbers of previous data (turns) to the Turkers in annotating sentiments of dyadic negotiation dialogs on the inter annotator reliability and comparison to the gold standard.
596.
Morbini, Fabrizio; DeVault, David; Sagae, Kenji; Gerten, Jillian; Nazarian, Angela; Traum, David
FLoReS: A Forward Looking, Reward Seeking, Dialogue Manager Book Section
In: Natural Interaction with Robots, Knowbots and Smartphones - Putting Spoken Dialog Systems into Practice, pp. 313–325, Springer New York, 2013.
@incollection{morbini_flores_2013,
title = {FLoReS: A Forward Looking, Reward Seeking, Dialogue Manager},
author = {Fabrizio Morbini and David DeVault and Kenji Sagae and Jillian Gerten and Angela Nazarian and David Traum},
url = {http://www.amazon.com/Natural-Interaction-Robots-Knowbots-Smartphones/dp/1461482798/ref=sr_1_1?s=books&ie=UTF8&qid=1409179426&sr=1-1&keywords=Natural+Interaction+with+Robots%2C+Knowbots+and+Smartphones},
year = {2013},
date = {2013-11-01},
booktitle = {Natural Interaction with Robots, Knowbots and Smartphones - Putting Spoken Dialog Systems into Practice},
pages = {313–325},
publisher = {Springer New York},
keywords = {},
pubstate = {published},
tppubtype = {incollection}
}
597.
Ito, Jonathan Y.; Marsella, Stacy C.
Modeling Framing Effects Comparing an Appraisal-Based Model with Existing Models Proceedings Article
In: ACII 2013, pp. 381–386, IEEE Computer Society, 2013.
@inproceedings{ito_modeling_2013,
title = {Modeling Framing Effects Comparing an Appraisal-Based Model with Existing Models},
author = {Jonathan Y. Ito and Stacy C. Marsella},
url = {http://ict.usc.edu/pubs/Modeling%20Framing%20Effects%20Comparing%20an%20Appraisal-Based%20Model%20with%20Existing%20Models.pdf},
year = {2013},
date = {2013-09-01},
booktitle = {ACII 2013},
pages = {381–386},
publisher = {IEEE Computer Society},
abstract = {One significant challenge in creating accurate models of human decision behavior is accounting for the effects of context. Research shows that seemingly minor changes in the presentation of a decision can lead to shifts in behavior; phenomena collectively referred to as framing effects. This work presents a computational modeling analysis comparing the effectiveness of Context Dependent Utility, an appraisal-based approach to modeling the multi-dimensional effects of context on decision behavior, against Cumulative Prospect Theory, Security-Potential/Aspiration Theory, the Transfer of Attention Exchange model, and a power-based utility function. To contrast model performance, a non-linear least-squares analysis and subsequent calculation of Akaike Information Criterion scores, which take into account goodness of fit while penalizing for model complexity, are employed. Results suggest that multi-dimensional models of context and framing, such as Context Dependent Utility, can be much more accurate in modeling decisions which similarly involve multi-dimensional considerations of context. Furthermore, this work demonstrates the effectiveness of employing affective constructs, such as appraisal, for encoding and evaluation of context within decision-theoretic frameworks to better model and predict human decision behavior.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
One significant challenge in creating accurate models of human decision behavior is accounting for the effects of context. Research shows that seemingly minor changes in the presentation of a decision can lead to shifts in behavior; phenomena collectively referred to as framing effects. This work presents a computational modeling analysis comparing the effectiveness of Context Dependent Utility, an appraisal-based approach to modeling the multi-dimensional effects of context on decision behavior, against Cumulative Prospect Theory, Security-Potential/Aspiration Theory, the Transfer of Attention Exchange model, and a power-based utility function. To contrast model performance, a non-linear least-squares analysis and subsequent calculation of Akaike Information Criterion scores, which take into account goodness of fit while penalizing for model complexity, are employed. Results suggest that multi-dimensional models of context and framing, such as Context Dependent Utility, can be much more accurate in modeling decisions which similarly involve multi-dimensional considerations of context. Furthermore, this work demonstrates the effectiveness of employing affective constructs, such as appraisal, for encoding and evaluation of context within decision-theoretic frameworks to better model and predict human decision behavior.
598.
Khooshabeh, Peter; Dehghani, Morteza; Nazarian, Angela; Gratch, Jonathan
The Cultural Influence Model: When Accented Natural Language Spoken by Virtual Characters Matters Journal Article
In: Journal of Artificial Intelligence and Society, vol. 29, 2013.
@article{khooshabeh_cultural_2013,
title = {The Cultural Influence Model: When Accented Natural Language Spoken by Virtual Characters Matters},
author = {Peter Khooshabeh and Morteza Dehghani and Angela Nazarian and Jonathan Gratch},
url = {http://ict.usc.edu/pubs/The%20Cultural%20Influence%20Model-%20When%20Accented%20Natural%20Language%20Spoken%20by%20Virtual%20Characters%20Matters.pdf},
year = {2013},
date = {2013-09-01},
journal = {Journal of Artificial Intelligence and Society},
volume = {29},
abstract = {Advances in Artificial Intelligence (AI) and computer graphics digital technologies have contributed to a relative increase of realism in virtual characters. Preserving virtual characters’ communicative realism, in particular, joined the ranks of the improvements in natural language technology and animation algorithms. This paper focuses on culturally relevant paralinguistic cues in nonverbal communication. We model the effects of an English speaking digital character with different accents on human interactants (i.e., users). Our cultural influence model proposes that paralinguistic realism, in the form of accented speech, is effective in promoting culturally congruent cognition only when it is self-relevant to users. For example, a Chinese or Middle Eastern English accent may be perceived as foreign to individuals who do not share the same ethnic cultural background with members of those cultures. However, for individuals who are familiar and affiliate with those cultures (i.e., in-group members who are bicultural), accent not only serves as a motif of shared social identity, it also primes them to adopt culturally appropriate interpretive frames that influence their decision making.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Advances in Artificial Intelligence (AI) and computer graphics digital technologies have contributed to a relative increase of realism in virtual characters. Preserving virtual characters’ communicative realism, in particular, joined the ranks of the improvements in natural language technology and animation algorithms. This paper focuses on culturally relevant paralinguistic cues in nonverbal communication. We model the effects of an English speaking digital character with different accents on human interactants (i.e., users). Our cultural influence model proposes that paralinguistic realism, in the form of accented speech, is effective in promoting culturally congruent cognition only when it is self-relevant to users. For example, a Chinese or Middle Eastern English accent may be perceived as foreign to individuals who do not share the same ethnic cultural background with members of those cultures. However, for individuals who are familiar and affiliate with those cultures (i.e., in-group members who are bicultural), accent not only serves as a motif of shared social identity, it also primes them to adopt culturally appropriate interpretive frames that influence their decision making.
599.
Park, Sunghyun; Scherer, Stefan; Gratch, Jonathan; Carnevale, Peter; Morency, Louis-Philippe
Mutual Behaviors during Dyadic Negotiation: Automatic Prediction of Respondent Reactions Proceedings Article
In: Affective Computing and Intelligent Interaction, pp. 423–428, Geneva, Switzerland, 2013, ISBN: 978-0-7695-5048-0.
@inproceedings{park_mutual_2013,
title = {Mutual Behaviors during Dyadic Negotiation: Automatic Prediction of Respondent Reactions},
author = {Sunghyun Park and Stefan Scherer and Jonathan Gratch and Peter Carnevale and Louis-Philippe Morency},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6681467},
doi = {10.1109/ACII.2013.76},
isbn = {978-0-7695-5048-0},
year = {2013},
date = {2013-09-01},
booktitle = {Affective Computing and Intelligent Interaction},
pages = {423–428},
address = {Geneva, Switzerland},
abstract = {In this paper, we analyze face-to-face negotiation interactions with the goal of predicting the respondent’s immediate reaction (i.e., accept or reject) to a negotiation offer. Supported by the theory of social rapport, we focus on mutual behaviors which are defined as nonverbal characteristics that occur due to interactional influence. These patterns include behavioral symmetry (e.g., synchronized smiles) as well as asymmetry (e.g., opposite postures) between the two negotiators. In addition, we put emphasis on finding audio- visual mutual behaviors that can be extracted automatically, with the vision of a real-time decision support tool. We introduce a dyadic negotiation dataset consisting of 42 face-to- face interactions and show experiments confirming the importance of multimodal and mutual behaviors.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we analyze face-to-face negotiation interactions with the goal of predicting the respondent’s immediate reaction (i.e., accept or reject) to a negotiation offer. Supported by the theory of social rapport, we focus on mutual behaviors which are defined as nonverbal characteristics that occur due to interactional influence. These patterns include behavioral symmetry (e.g., synchronized smiles) as well as asymmetry (e.g., opposite postures) between the two negotiators. In addition, we put emphasis on finding audio- visual mutual behaviors that can be extracted automatically, with the vision of a real-time decision support tool. We introduce a dyadic negotiation dataset consisting of 42 face-to- face interactions and show experiments confirming the importance of multimodal and mutual behaviors.
600.
Stratou, Giota; Scherer, Stefan; Gratch, Jonathan; Morency, Louis-Philippe
Automatic Nonverbal Behavior Indicators of Depression and PTSD: Exploring Gender Differences Proceedings Article
In: Affective Computing and Intelligent Interaction, pp. 147–152, IEEE, Geneva, Switzerland, 2013, ISBN: 978-0-7695-5048-0.
@inproceedings{stratou_automatic_2013,
title = {Automatic Nonverbal Behavior Indicators of Depression and PTSD: Exploring Gender Differences},
author = {Giota Stratou and Stefan Scherer and Jonathan Gratch and Louis-Philippe Morency},
url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6681422},
doi = {10.1109/ACII.2013.31},
isbn = {978-0-7695-5048-0},
year = {2013},
date = {2013-09-01},
booktitle = {Affective Computing and Intelligent Interaction},
pages = {147–152},
publisher = {IEEE},
address = {Geneva, Switzerland},
abstract = {In this paper, we show that gender plays an important role in the automatic assessment of psychological conditions such as depression and post-traumatic stress disorder (PTSD). We identify a directly interpretable and intuitive set of predictive indicators, selected from three general categories of nonverbal behaviors: affect, expression variability and motor variability. For the analysis, we introduce a semi-structured virtual human interview dataset which includes 53 video recorded interactions. Our experiments on automatic classification of psychological conditions show that a gender-dependent approach significantly improves the performance over a gender agnostic one.},
keywords = {},
pubstate = {published},
tppubtype = {inproceedings}
}
In this paper, we show that gender plays an important role in the automatic assessment of psychological conditions such as depression and post-traumatic stress disorder (PTSD). We identify a directly interpretable and intuitive set of predictive indicators, selected from three general categories of nonverbal behaviors: affect, expression variability and motor variability. For the analysis, we introduce a semi-structured virtual human interview dataset which includes 53 video recorded interactions. Our experiments on automatic classification of psychological conditions show that a gender-dependent approach significantly improves the performance over a gender agnostic one.
Filter
Sorry, no publications matched your criteria.