Publications
Search
221.
Feng, Andrew; Rosenberg, Evan Suma; Shapiro, Ari
Just-in-time, viable, 3D avatars from scans Journal Article
In: Computer Animation and Virtual Worlds (Special Issue on Computer Animation and Social Agents), vol. 28, no. 3-4, 2017.
@article{feng_just–time_2017,
title = {Just-in-time, viable, 3D avatars from scans},
author = {Andrew Feng and Evan Suma Rosenberg and Ari Shapiro},
url = {http://onlinelibrary.wiley.com/doi/10.1002/cav.1769/epdf},
doi = {10.1002/cav.1769},
year = {2017},
date = {2017-05-01},
journal = {Computer Animation and Virtual Worlds (Special Issue on Computer Animation and Social Agents)},
volume = {28},
number = {3-4},
abstract = {We demonstrate a system that can generate a photorealistic, interactive 3-D character from a human subject that is capable of movement, emotion, speech, and gesture in less than 20 min without the need for 3-D artist intervention or specialized technical knowledge through a near automatic process. Our method uses mostly commodity or off-the-shelf hardware. We demonstrate the just-in-time use of generating such 3-D models for virtual and augmented reality, games, simulation, and communication. We anticipate that the inexpensive generation of such photorealistic models will be useful in many venues where a just-in-time 3-D reconstructions of digital avatars that resemble particular human subjects is necessary.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We demonstrate a system that can generate a photorealistic, interactive 3-D character from a human subject that is capable of movement, emotion, speech, and gesture in less than 20 min without the need for 3-D artist intervention or specialized technical knowledge through a near automatic process. Our method uses mostly commodity or off-the-shelf hardware. We demonstrate the just-in-time use of generating such 3-D models for virtual and augmented reality, games, simulation, and communication. We anticipate that the inexpensive generation of such photorealistic models will be useful in many venues where a just-in-time 3-D reconstructions of digital avatars that resemble particular human subjects is necessary.
222.
Feng, Andrew; Rosenberg, Evan Suma; Shapiro, Ari
Just-in-time, viable, 3D avatars from scans Journal Article
In: Computer Animation and Virtual Worlds (Special Issue on Computer Animation and Social Agents), vol. 28, no. 3-4, 2017.
@article{feng_just--time_2017,
title = {Just-in-time, viable, 3D avatars from scans},
author = {Andrew Feng and Evan Suma Rosenberg and Ari Shapiro},
url = {http://onlinelibrary.wiley.com/doi/10.1002/cav.1769/epdf},
doi = {10.1002/cav.1769},
year = {2017},
date = {2017-05-01},
journal = {Computer Animation and Virtual Worlds (Special Issue on Computer Animation and Social Agents)},
volume = {28},
number = {3-4},
abstract = {We demonstrate a system that can generate a photorealistic, interactive 3-D character from a human subject that is capable of movement, emotion, speech, and gesture in less than 20 min without the need for 3-D artist intervention or specialized technical knowledge through a near automatic process. Our method uses mostly commodity or off-the-shelf hardware. We demonstrate the just-in-time use of generating such 3-D models for virtual and augmented reality, games, simulation, and communication. We anticipate that the inexpensive generation of such photorealistic models will be useful in many venues where a just-in-time 3-D reconstructions of digital avatars that resemble particular human subjects is necessary.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We demonstrate a system that can generate a photorealistic, interactive 3-D character from a human subject that is capable of movement, emotion, speech, and gesture in less than 20 min without the need for 3-D artist intervention or specialized technical knowledge through a near automatic process. Our method uses mostly commodity or off-the-shelf hardware. We demonstrate the just-in-time use of generating such 3-D models for virtual and augmented reality, games, simulation, and communication. We anticipate that the inexpensive generation of such photorealistic models will be useful in many venues where a just-in-time 3-D reconstructions of digital avatars that resemble particular human subjects is necessary.
223.
Stratou, Giota; Morency, Louis-Philippe
MultiSense—Context-Aware Nonverbal Behavior Analysis Framework: A Psychological Distress Use Case Journal Article
In: IEEE Transactions on Affective Computing, vol. 8, no. 2, pp. 190–203, 2017, ISSN: 1949-3045.
@article{stratou_multisensecontext-aware_2017,
title = {MultiSense—Context-Aware Nonverbal Behavior Analysis Framework: A Psychological Distress Use Case},
author = {Giota Stratou and Louis-Philippe Morency},
url = {http://ieeexplore.ieee.org/document/7579221/},
doi = {10.1109/TAFFC.2016.2614300},
issn = {1949-3045},
year = {2017},
date = {2017-04-01},
journal = {IEEE Transactions on Affective Computing},
volume = {8},
number = {2},
pages = {190–203},
abstract = {During face-to-face interactions, people naturally integrate nonverbal behaviors such as facial expressions and body postures as part of the conversation to infer the communicative intent or emotional state of their interlocutor. The interpretation of these nonverbal behaviors will often be contextualized by interactional cues such as the previous spoken question, the general discussion topic or the physical environment. A critical step in creating computers able to understand or participate in this type of social face-to-face interactions is to develop a computational platform to synchronously recognize nonverbal behaviors as part of the interactional context. In this platform, information for the acoustic and visual modalities should be carefully synchronized and rapidly processed. At the same time, contextual and interactional cues should be remembered and integrated to better interpret nonverbal (and verbal) behaviors. In this article, we introduce a real-time computational framework, MultiSense, which offers flexible and efficient synchronization approaches for context-based nonverbal behavior analysis. MultiSense is designed to utilize interactional cues from both interlocutors (e.g., from the computer and the human participant) and integrate this contextual information when interpreting nonverbal behaviors. MultiSense can also assimilate behaviors over a full interaction and summarize the observed affective states of the user. We demonstrate the capabilities of the new framework with a concrete use case from the mental health domain where MultiSense is used as part of a decision support tool to assess indicators of psychological distress such as depression and post-traumatic stress disorder (PTSD). In this scenario, MultiSense not only infers psychological distress indicators from nonverbal behaviors but also broadcasts the user state in real-time to a virtual agent (i.e., a digital interviewer) designed to conduct semi-structured interviews with human participants. Our experiments show the added value of our multimodal synchronization approaches and also demonstrate the importance of MultiSense contextual interpretation when inferring distress indicators.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
During face-to-face interactions, people naturally integrate nonverbal behaviors such as facial expressions and body postures as part of the conversation to infer the communicative intent or emotional state of their interlocutor. The interpretation of these nonverbal behaviors will often be contextualized by interactional cues such as the previous spoken question, the general discussion topic or the physical environment. A critical step in creating computers able to understand or participate in this type of social face-to-face interactions is to develop a computational platform to synchronously recognize nonverbal behaviors as part of the interactional context. In this platform, information for the acoustic and visual modalities should be carefully synchronized and rapidly processed. At the same time, contextual and interactional cues should be remembered and integrated to better interpret nonverbal (and verbal) behaviors. In this article, we introduce a real-time computational framework, MultiSense, which offers flexible and efficient synchronization approaches for context-based nonverbal behavior analysis. MultiSense is designed to utilize interactional cues from both interlocutors (e.g., from the computer and the human participant) and integrate this contextual information when interpreting nonverbal behaviors. MultiSense can also assimilate behaviors over a full interaction and summarize the observed affective states of the user. We demonstrate the capabilities of the new framework with a concrete use case from the mental health domain where MultiSense is used as part of a decision support tool to assess indicators of psychological distress such as depression and post-traumatic stress disorder (PTSD). In this scenario, MultiSense not only infers psychological distress indicators from nonverbal behaviors but also broadcasts the user state in real-time to a virtual agent (i.e., a digital interviewer) designed to conduct semi-structured interviews with human participants. Our experiments show the added value of our multimodal synchronization approaches and also demonstrate the importance of MultiSense contextual interpretation when inferring distress indicators.
224.
Kang, Sin-Hwa; Krum, David M.; Khooshabeh, Peter; Phan, Thai; Chang, Chien-Yen; Amir, Ori; Lin, Rebecca
Social influence of humor in virtual human counselor's self-disclosure Journal Article
In: Computer Animation and Virtual Worlds, vol. 28, no. 3-4, 2017, ISSN: 15464261.
@article{kang_social_2017,
title = {Social influence of humor in virtual human counselor's self-disclosure},
author = {Sin-Hwa Kang and David M. Krum and Peter Khooshabeh and Thai Phan and Chien-Yen Chang and Ori Amir and Rebecca Lin},
url = {http://doi.wiley.com/10.1002/cav.1763},
doi = {10.1002/cav.1763},
issn = {15464261},
year = {2017},
date = {2017-04-01},
journal = {Computer Animation and Virtual Worlds},
volume = {28},
number = {3-4},
abstract = {We explored the social influence of humor in a virtual human counselor's selfdisclosure while also varying the ethnicity of the virtual counselor. In a 2 × 3 experiment (humor and ethnicity of the virtual human counselor), participants experienced counseling interview interactions via Skype on a smartphone. We measured user responses to and perceptions of the virtual human counselor. The results demonstrate that humor positively affects user responses to and perceptions of a virtual counselor. The results further suggest that matching styles of humor with a virtual counselor's ethnicity influences user responses and perceptions. The results offer insight into the effective design and development of realistic and believable virtual human counselors. Furthermore, they illuminate the potential use of humor to enhance self‐disclosure in human–agent interactions.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We explored the social influence of humor in a virtual human counselor's selfdisclosure while also varying the ethnicity of the virtual counselor. In a 2 × 3 experiment (humor and ethnicity of the virtual human counselor), participants experienced counseling interview interactions via Skype on a smartphone. We measured user responses to and perceptions of the virtual human counselor. The results demonstrate that humor positively affects user responses to and perceptions of a virtual counselor. The results further suggest that matching styles of humor with a virtual counselor's ethnicity influences user responses and perceptions. The results offer insight into the effective design and development of realistic and believable virtual human counselors. Furthermore, they illuminate the potential use of humor to enhance self‐disclosure in human–agent interactions.
225.
Yang, Chao; Lu, Xin; Lin, Zhe; Shechtman, Eli; Wang, Oliver; Li, Hao
High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis Journal Article
In: arXiv preprint arXiv:1611.09969v2, 2017.
@article{yang_high-resolution_2017,
title = {High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesis},
author = {Chao Yang and Xin Lu and Zhe Lin and Eli Shechtman and Oliver Wang and Hao Li},
url = {https://arxiv.org/pdf/1611.09969},
year = {2017},
date = {2017-04-01},
journal = {arXiv preprint arXiv:1611.09969v2},
abstract = {Recent advances in deep learning have shown exciting promise in filling large holes in natural images with semantically plausible and context aware details, impacting fundamental image manipulation tasks such as object removal. While these learning-based methods are significantly more effective in capturing high-level features than prior techniques, hey can only handle very low-resolution inputs due to memory limitations and difficulty in training. Even for slightly larger images, the inpainted regions would appear blurry and unpleasant boundaries become visible. We propose a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network. We evaluate our method on the ImageNet and Paris Streetview datasets and achieved state-of-theart inpainting accuracy. We show our approach produces sharper and more coherent results than prior methods, especially for high-resolution images.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Recent advances in deep learning have shown exciting promise in filling large holes in natural images with semantically plausible and context aware details, impacting fundamental image manipulation tasks such as object removal. While these learning-based methods are significantly more effective in capturing high-level features than prior techniques, hey can only handle very low-resolution inputs due to memory limitations and difficulty in training. Even for slightly larger images, the inpainted regions would appear blurry and unpleasant boundaries become visible. We propose a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network. We evaluate our method on the ImageNet and Paris Streetview datasets and achieved state-of-theart inpainting accuracy. We show our approach produces sharper and more coherent results than prior methods, especially for high-resolution images.
226.
Dehghani, Morteza; Boghrati, Reihane; Man, Kingson; Hoover, Joseph; Gimbel, Sarah; Vaswani, Ashish; Zevin, Jason; Immordino, Mary Helen; Gordon, Andrew; Damasio, Antonio; Kaplan, Jonas T.
Decoding the Neural Representation of Story Meanings across Languages Journal Article
In: Human Brain Mapping, vol. 38, no. 12, 2017.
@article{dehghani_decoding_2017,
title = {Decoding the Neural Representation of Story Meanings across Languages},
author = {Morteza Dehghani and Reihane Boghrati and Kingson Man and Joseph Hoover and Sarah Gimbel and Ashish Vaswani and Jason Zevin and Mary Helen Immordino and Andrew Gordon and Antonio Damasio and Jonas T. Kaplan},
url = {https://psyarxiv.com/qrpp3/},
doi = {10.1002/hbm.23814},
year = {2017},
date = {2017-03-01},
journal = {Human Brain Mapping},
volume = {38},
number = {12},
abstract = {Drawing from a common lexicon of semantic units, humans fashion narratives whose meaning transcends that of their individual utterances. However, while brain regions that represent lower-level semantic units, such as words and sentences, have been identified, questions remain about the neural representation of narrative comprehension, which involves inferring cumulative meaning. To address these questions, we exposed English, Mandarin and Farsi native speakers to native language translations of the same stories during fMRI scanning. Using a new technique in natural language processing, we calculated the distributed representations of these stories (capturing the meaning of the stories in high-dimensional semantic space), and demonstrate that using these representations we can identify the specific story a participant was reading from the neural data. Notably, this was possible even when the distributed representations were calculated using stories in a different language than the participant was reading. Relying on over 44 billion classifications, our results reveal that identification relied on a collection of brain regions most prominently located in the default mode network. These results demonstrate that neuro-semantic encoding of narratives happens at levels higher than individual semantic units and that this encoding is systematic across both individuals and languages.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Drawing from a common lexicon of semantic units, humans fashion narratives whose meaning transcends that of their individual utterances. However, while brain regions that represent lower-level semantic units, such as words and sentences, have been identified, questions remain about the neural representation of narrative comprehension, which involves inferring cumulative meaning. To address these questions, we exposed English, Mandarin and Farsi native speakers to native language translations of the same stories during fMRI scanning. Using a new technique in natural language processing, we calculated the distributed representations of these stories (capturing the meaning of the stories in high-dimensional semantic space), and demonstrate that using these representations we can identify the specific story a participant was reading from the neural data. Notably, this was possible even when the distributed representations were calculated using stories in a different language than the participant was reading. Relying on over 44 billion classifications, our results reveal that identification relied on a collection of brain regions most prominently located in the default mode network. These results demonstrate that neuro-semantic encoding of narratives happens at levels higher than individual semantic units and that this encoding is systematic across both individuals and languages.
227.
Parra, Federico; Miljkovitch, Raphaële; Persiaux, Gwenaelle; Morales, Michelle; Scherer, Stefan
The Multimodal Assessment of Adult Attachment Security: Developing the Biometric Attachment Test Journal Article
In: Journal of Medical Internet Research, vol. 19, no. 4, pp. e100, 2017, ISSN: 1438-8871.
@article{parra_multimodal_2017,
title = {The Multimodal Assessment of Adult Attachment Security: Developing the Biometric Attachment Test},
author = {Federico Parra and Raphaële Miljkovitch and Gwenaelle Persiaux and Michelle Morales and Stefan Scherer},
url = {http://www.jmir.org/2017/4/e100/},
doi = {10.2196/jmir.6898},
issn = {1438-8871},
year = {2017},
date = {2017-03-01},
journal = {Journal of Medical Internet Research},
volume = {19},
number = {4},
pages = {e100},
abstract = {Background: Attachment theory has been proven essential for mental health, including psychopathology, development, and interpersonal relationships. Validated psychometric instruments to measure attachment abound but suffer from shortcomings common to traditional psychometrics. Recent developments in multimodal fusion and machine learning pave the way for new automated and objective psychometric instruments for adult attachment that combine psychophysiological, linguistic, and behavioral analyses in the assessment of the construct.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Background: Attachment theory has been proven essential for mental health, including psychopathology, development, and interpersonal relationships. Validated psychometric instruments to measure attachment abound but suffer from shortcomings common to traditional psychometrics. Recent developments in multimodal fusion and machine learning pave the way for new automated and objective psychometric instruments for adult attachment that combine psychophysiological, linguistic, and behavioral analyses in the assessment of the construct.
228.
Nouri, Elnaz; Georgila, Kallirroi; Traum, David
Culture-specific models of negotiation for virtual characters: multi-attribute decision-making based on culture-specific values Journal Article
In: AI & SOCIETY, vol. 32, no. 1, pp. 51–63, 2017, ISSN: 0951-5666, 1435-5655.
@article{nouri_culture-specific_2017,
title = {Culture-specific models of negotiation for virtual characters: multi-attribute decision-making based on culture-specific values},
author = {Elnaz Nouri and Kallirroi Georgila and David Traum},
url = {http://link.springer.com/10.1007/s00146-014-0570-7},
doi = {10.1007/s00146-014-0570-7},
issn = {0951-5666, 1435-5655},
year = {2017},
date = {2017-02-01},
journal = {AI & SOCIETY},
volume = {32},
number = {1},
pages = {51–63},
abstract = {We posit that observed differences in negotiation performance across cultures can be explained by participants trying to optimize across multiple values, where the relative importance of values differs across cultures. We look at two ways for specifying weights on values for different cultures: one in which the weights of the model are hand-crafted, based on intuition interpreting Hofstede dimensions for the cultures, and one in which the weights of the model are learned from data using inverse reinforcement learning (IRL). We apply this model to the Ultimatum Game and integrate it into a virtual human dialog system. We show that weights learned from IRL surpass both a weak baseline with random weights and a strong baseline considering only one factor for maximizing gain in own wealth in accounting for the behavior of human players from four different cultures. Wealso show that the weights learned with our model for one culture outperform weights learned for other cultures when playing against opponents of the first culture.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We posit that observed differences in negotiation performance across cultures can be explained by participants trying to optimize across multiple values, where the relative importance of values differs across cultures. We look at two ways for specifying weights on values for different cultures: one in which the weights of the model are hand-crafted, based on intuition interpreting Hofstede dimensions for the cultures, and one in which the weights of the model are learned from data using inverse reinforcement learning (IRL). We apply this model to the Ultimatum Game and integrate it into a virtual human dialog system. We show that weights learned from IRL surpass both a weak baseline with random weights and a strong baseline considering only one factor for maximizing gain in own wealth in accounting for the behavior of human players from four different cultures. Wealso show that the weights learned with our model for one culture outperform weights learned for other cultures when playing against opponents of the first culture.
229.
Lucas, Gale M.; Gratch, Jonathan; Malandrakis, Nikolaos; Szablowski, Evan; Fessler, Eli; Nichols, Jeffrey
GOAALLL!: Using Sentiment in the World Cup to Explore Theories of Emotion Journal Article
In: Image and Vision Computing, 2017, ISSN: 02628856.
@article{lucas_goaalll_2017,
title = {GOAALLL!: Using Sentiment in the World Cup to Explore Theories of Emotion},
author = {Gale M. Lucas and Jonathan Gratch and Nikolaos Malandrakis and Evan Szablowski and Eli Fessler and Jeffrey Nichols},
url = {http://linkinghub.elsevier.com/retrieve/pii/S0262885617300148},
doi = {10.1016/j.imavis.2017.01.006},
issn = {02628856},
year = {2017},
date = {2017-01-01},
journal = {Image and Vision Computing},
abstract = {Sporting events evoke strong emotions amongst fans and thus act as natural laboratories to explore emotions and how they unfold in the wild. Computational tools, such as sentiment analysis, provide new ways to examine such dynamic emotional processes. In this article we use sentiment analysis to examine tweets posted during 2014 World Cup. Such analysis gives insight into how people respond to highly emotional events, and how these emotions are shaped by contextual factors, such as prior expectations, and how these emotions change as events unfold over time. Here we report on some preliminary analysis of a World Cup twitter corpus using sentiment analysis techniques. After performing initial tests of validation for sentiment analysis on data in this corpus, we show these tools can give new insights into existing theories of what makes a sporting match exciting. This analysis seems to suggest that, contrary to assumptions in sports economics, excitement relates to expressions of negative emotion. The results are discussed in terms of innovations in methodology and understanding the role of emotion for “tuning in” to real world events. We also discuss some challenges that such data present for existing sentiment analysis techniques and discuss future analysis.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Sporting events evoke strong emotions amongst fans and thus act as natural laboratories to explore emotions and how they unfold in the wild. Computational tools, such as sentiment analysis, provide new ways to examine such dynamic emotional processes. In this article we use sentiment analysis to examine tweets posted during 2014 World Cup. Such analysis gives insight into how people respond to highly emotional events, and how these emotions are shaped by contextual factors, such as prior expectations, and how these emotions change as events unfold over time. Here we report on some preliminary analysis of a World Cup twitter corpus using sentiment analysis techniques. After performing initial tests of validation for sentiment analysis on data in this corpus, we show these tools can give new insights into existing theories of what makes a sporting match exciting. This analysis seems to suggest that, contrary to assumptions in sports economics, excitement relates to expressions of negative emotion. The results are discussed in terms of innovations in methodology and understanding the role of emotion for “tuning in” to real world events. We also discuss some challenges that such data present for existing sentiment analysis techniques and discuss future analysis.
230.
Eini, Dalit Shefer; Ratzon, Navah Z.; Rizzo, Albert A.; Yeh, Shih-Ching; Lange, Belinda; Yaffe, Batia; Daich, Alexander; Weiss, Patrice L.; Kizony, Rachel
Camera-tracking gaming control device for evaluation of active wrist flexion and extension Journal Article
In: Journal of Hand Therapy, vol. 30, no. 1, pp. 89–96, 2017, ISSN: 08941130.
@article{shefer_eini_camera-tracking_2017,
title = {Camera-tracking gaming control device for evaluation of active wrist flexion and extension},
author = {Dalit Shefer Eini and Navah Z. Ratzon and Albert A. Rizzo and Shih-Ching Yeh and Belinda Lange and Batia Yaffe and Alexander Daich and Patrice L. Weiss and Rachel Kizony},
url = {http://linkinghub.elsevier.com/retrieve/pii/S0894113016301132},
doi = {10.1016/j.jht.2016.07.002},
issn = {08941130},
year = {2017},
date = {2017-01-01},
journal = {Journal of Hand Therapy},
volume = {30},
number = {1},
pages = {89–96},
abstract = {Study Design: Cross sectional. Introduction: Measuring wrist range of motion (ROM) is an essential procedure in hand therapy clinics. Purpose of the Study: To test the reliability and validity of a dynamic ROM assessment, the Camera WristTracker (CWT). Methods: Wrist flexion and extension ROM of 15 patients with distal radius fractures and 15 matchedcontrols were assessed with the CWT and with a universal goniometer. Results: One-way model intraclass correlation coefficient analysis indicated high test-retest reliability for extension (ICC ¼ 0.92) and moderate reliability for flexion (ICC ¼ 0.49). Standard error for extension was 2.45 and for flexion was 4.07 . Repeated-measures analysis revealed a significant main effect for group; ROM was greater in the control group (F[1, 28] ¼ 47.35; P textbackslashtextbackslashtextbackslashtextless.001). The concurrent validity of the CWT was partially supported. Conclusion: The results indicate that the CWT may provide highly reliable scores for dynamic wrist extension ROM, and moderately reliable scores for flexion, in people recovering from a distal radius fracture. Level of Evidence: N/A.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Study Design: Cross sectional. Introduction: Measuring wrist range of motion (ROM) is an essential procedure in hand therapy clinics. Purpose of the Study: To test the reliability and validity of a dynamic ROM assessment, the Camera WristTracker (CWT). Methods: Wrist flexion and extension ROM of 15 patients with distal radius fractures and 15 matchedcontrols were assessed with the CWT and with a universal goniometer. Results: One-way model intraclass correlation coefficient analysis indicated high test-retest reliability for extension (ICC ¼ 0.92) and moderate reliability for flexion (ICC ¼ 0.49). Standard error for extension was 2.45 and for flexion was 4.07 . Repeated-measures analysis revealed a significant main effect for group; ROM was greater in the control group (F[1, 28] ¼ 47.35; P textbackslashtextbackslashtextbackslashtextless.001). The concurrent validity of the CWT was partially supported. Conclusion: The results indicate that the CWT may provide highly reliable scores for dynamic wrist extension ROM, and moderately reliable scores for flexion, in people recovering from a distal radius fracture. Level of Evidence: N/A.
231.
Rizzo, Albert "Skip"
The Ultimate Skinner Box: Clinical Virtual Reality 1990-2016 Journal Article
In: Engadget, 2017.
@article{rizzo_ultimate_2017,
title = {The Ultimate Skinner Box: Clinical Virtual Reality 1990-2016},
author = {Albert "Skip" Rizzo},
url = {https://www.engadget.com/2017/01/10/the-ultimate-skinner-box-clinical-virtual-reality-1990-2016/},
year = {2017},
date = {2017-01-01},
journal = {Engadget},
abstract = {The last decade has given rise to a dramatic increase in the global adoption of innovative digital technologies. This can be seen in the rapid acceptance and growing demand for mobile devices, high speed network access, smart televisions, social media, hyper-realistic digital games, behavioral sensing devices, and now the 2nd coming of Virtual Reality! Such consumer driven technologies that were considered to be visionary just 10 years ago have now become common and increasingly essential fixtures in the current digital landscape},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
The last decade has given rise to a dramatic increase in the global adoption of innovative digital technologies. This can be seen in the rapid acceptance and growing demand for mobile devices, high speed network access, smart televisions, social media, hyper-realistic digital games, behavioral sensing devices, and now the 2nd coming of Virtual Reality! Such consumer driven technologies that were considered to be visionary just 10 years ago have now become common and increasingly essential fixtures in the current digital landscape
232.
Woolley, J. D.; Chuang, B.; Fussell, C.; Scherer, S.; Biagianti, B.; Fulford, D.; Mathalon, D. H.; Vinogradov, S.
Intranasal oxytocin increases facial expressivity, but not ratings of trustworthiness, in patients with schizophrenia and healthy controls Journal Article
In: Psychological Medicine, pp. 1–12, 2017, ISSN: 0033-2917, 1469-8978.
@article{woolley_intranasal_2017,
title = {Intranasal oxytocin increases facial expressivity, but not ratings of trustworthiness, in patients with schizophrenia and healthy controls},
author = {J. D. Woolley and B. Chuang and C. Fussell and S. Scherer and B. Biagianti and D. Fulford and D. H. Mathalon and S. Vinogradov},
url = {https://www.cambridge.org/core/product/identifier/S0033291716003433/type/journal_article},
doi = {10.1017/S0033291716003433},
issn = {0033-2917, 1469-8978},
year = {2017},
date = {2017-01-01},
journal = {Psychological Medicine},
pages = {1–12},
abstract = {Blunted facial affect is a common negative symptom of schizophrenia. Additionally, assessing the trustworthiness of faces is a social cognitive ability that is impaired in schizophrenia. Currently available pharmacological agents are ineffective at improving either of these symptoms, despite their clinical significance. The hypothalamic neuropeptide oxytocin has multiple prosocial effects when administered intranasally to healthy individuals and shows promise in decreasing negative symptoms and enhancing social cognition in schizophrenia. Although two small studies have investigated oxytocin's effects on ratings of facial trustworthiness in schizophrenia, its effects on facial expressivity have not been investigated in any population. We investigated the effects of oxytocin on facial emotional expressivity while participants performed a facial trustworthiness rating task in 33 individuals with schizophrenia and 35 age-matched healthy controls using a double-blind, placebo-controlled, cross-over design. Participants rated the trustworthiness of presented faces interspersed with emotionally evocative photographs while being video-recorded. Participants’ facial expressivity in these videos was quantified by blind raters using a well-validated manualized approach (i.e. the Facial Expression Coding System; FACES). While oxytocin administration did not affect ratings of facial trustworthiness, it significantly increased facial expressivity in individuals with schizophrenia (Z = −2.33},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Blunted facial affect is a common negative symptom of schizophrenia. Additionally, assessing the trustworthiness of faces is a social cognitive ability that is impaired in schizophrenia. Currently available pharmacological agents are ineffective at improving either of these symptoms, despite their clinical significance. The hypothalamic neuropeptide oxytocin has multiple prosocial effects when administered intranasally to healthy individuals and shows promise in decreasing negative symptoms and enhancing social cognition in schizophrenia. Although two small studies have investigated oxytocin's effects on ratings of facial trustworthiness in schizophrenia, its effects on facial expressivity have not been investigated in any population. We investigated the effects of oxytocin on facial emotional expressivity while participants performed a facial trustworthiness rating task in 33 individuals with schizophrenia and 35 age-matched healthy controls using a double-blind, placebo-controlled, cross-over design. Participants rated the trustworthiness of presented faces interspersed with emotionally evocative photographs while being video-recorded. Participants’ facial expressivity in these videos was quantified by blind raters using a well-validated manualized approach (i.e. the Facial Expression Coding System; FACES). While oxytocin administration did not affect ratings of facial trustworthiness, it significantly increased facial expressivity in individuals with schizophrenia (Z = −2.33
233.
Saito, Shunsuke; Wei, Lingyu; Hu, Liwen; Nagano, Koki; Li, Hao
Photorealistic Facial Texture Inference Using Deep Neural Networks Journal Article
In: arXiv preprint arXiv:1612.00523, 2016.
@article{saito_photorealistic_2016,
title = {Photorealistic Facial Texture Inference Using Deep Neural Networks},
author = {Shunsuke Saito and Lingyu Wei and Liwen Hu and Koki Nagano and Hao Li},
url = {https://arxiv.org/abs/1612.00523},
year = {2016},
date = {2016-12-01},
journal = {arXiv preprint arXiv:1612.00523},
abstract = {We present a data-driven inference method that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild. After an initial estimation of shape and low-frequency albedo, we compute a high-frequency partial texture map, without the shading component, of the visible face area. To extract the fine appearance details from this incomplete input, we introduce a multi-scale detail analysis technique based on midlayer feature correlations extracted from a deep convolutional neural network. We demonstrate that fitting a convex combination of feature correlations from a high-resolution face database can yield a semantically plausible facial detail description of the entire face. A complete and photorealistic texture map can then be synthesized by iteratively optimizing for the reconstructed feature correlations. Using these high-resolution textures and a commercial rendering framework, we can produce high-fidelity 3D renderings that are visually comparable to those obtained with state-of-theart multi-view face capture systems. We demonstrate successful face reconstructions from a wide range of low resolution input images, including those of historical figures. In addition to extensive evaluations, we validate the realism of our results using a crowdsourced user study.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We present a data-driven inference method that can synthesize a photorealistic texture map of a complete 3D face model given a partial 2D view of a person in the wild. After an initial estimation of shape and low-frequency albedo, we compute a high-frequency partial texture map, without the shading component, of the visible face area. To extract the fine appearance details from this incomplete input, we introduce a multi-scale detail analysis technique based on midlayer feature correlations extracted from a deep convolutional neural network. We demonstrate that fitting a convex combination of feature correlations from a high-resolution face database can yield a semantically plausible facial detail description of the entire face. A complete and photorealistic texture map can then be synthesized by iteratively optimizing for the reconstructed feature correlations. Using these high-resolution textures and a commercial rendering framework, we can produce high-fidelity 3D renderings that are visually comparable to those obtained with state-of-theart multi-view face capture systems. We demonstrate successful face reconstructions from a wide range of low resolution input images, including those of historical figures. In addition to extensive evaluations, we validate the realism of our results using a crowdsourced user study.
234.
Zadeh, Amir; Zellers, Rowan; Pincus, Eli; Morency, Louis-Philippe
Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages Journal Article
In: IEEE Intelligent Systems, vol. 31, no. 6, pp. 82–88, 2016, ISSN: 1541-1672.
@article{zadeh_multimodal_2016,
title = {Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages},
author = {Amir Zadeh and Rowan Zellers and Eli Pincus and Louis-Philippe Morency},
url = {http://ieeexplore.ieee.org/abstract/document/7742221/},
doi = {10.1109/MIS.2016.94},
issn = {1541-1672},
year = {2016},
date = {2016-11-01},
journal = {IEEE Intelligent Systems},
volume = {31},
number = {6},
pages = {82–88},
abstract = {People share their opinions, stories, and reviews through online video sharing websites every day. The automatic analysis of these online opinion videos is bringing new or understudied research challenges to the field of computational linguistics and multimodal analysis. Among these challenges is the fundamental question of exploiting the dynamics between visual gestures and verbal messages to be able to better model sentiment. This article addresses this question in four ways: introducing the first multimodal dataset with opinion-level sentiment intensity annotations; studying the prototypical interaction patterns between facial gestures and spoken words when inferring sentiment intensity; proposing a new computational representation, called multimodal dictionary, based on a language-gesture study; and evaluating the authors' proposed approach in a speaker-independent paradigm for sentiment intensity prediction. The authors' study identifies four interaction types between facial gestures and verbal content: neutral, emphasizer, positive, and negative interactions. Experiments show statistically significant improvement when using multimodal dictionary representation over the conventional early fusion representation (that is, feature concatenation).},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
People share their opinions, stories, and reviews through online video sharing websites every day. The automatic analysis of these online opinion videos is bringing new or understudied research challenges to the field of computational linguistics and multimodal analysis. Among these challenges is the fundamental question of exploiting the dynamics between visual gestures and verbal messages to be able to better model sentiment. This article addresses this question in four ways: introducing the first multimodal dataset with opinion-level sentiment intensity annotations; studying the prototypical interaction patterns between facial gestures and spoken words when inferring sentiment intensity; proposing a new computational representation, called multimodal dictionary, based on a language-gesture study; and evaluating the authors' proposed approach in a speaker-independent paradigm for sentiment intensity prediction. The authors' study identifies four interaction types between facial gestures and verbal content: neutral, emphasizer, positive, and negative interactions. Experiments show statistically significant improvement when using multimodal dictionary representation over the conventional early fusion representation (that is, feature concatenation).
235.
Traum, David
Using Dialogue System Technology to Support Interactive History Learning Journal Article
In: Journal of Japan Society of Artificial Intelligence (JSAI), vol. 31, no. 6, pp. 806, 2016.
@article{traum_using_2016,
title = {Using Dialogue System Technology to Support Interactive History Learning},
author = {David Traum},
url = {http://www.ai-gakkai.or.jp/en/en/vol31_no6/},
year = {2016},
date = {2016-11-01},
journal = {Journal of Japan Society of Artificial Intelligence (JSAI)},
volume = {31},
number = {6},
pages = {806},
abstract = {We describe the use of spoken dialogue technology to enhance informal history learning. We describe several uses for this technology, including allowing learners to engage in natural interactions at a historical site, allowing learners to talk with recreations of historical figures, and using oral history recordings of a witness to create a dialogue experience. Two projects are highlighted, one to give a guided experience of a historical location, and another, New Dimensions in Testimony, that allows an experience similar to face to face conversation with a Holocaust survivor. These techniques allow many of the bene ts of an intimate connection to historical places and people, through direct interaction and user initiative, but can also be delivered to a mass audience, formerly only reachable by broadcast, non-interactive media.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
We describe the use of spoken dialogue technology to enhance informal history learning. We describe several uses for this technology, including allowing learners to engage in natural interactions at a historical site, allowing learners to talk with recreations of historical figures, and using oral history recordings of a witness to create a dialogue experience. Two projects are highlighted, one to give a guided experience of a historical location, and another, New Dimensions in Testimony, that allows an experience similar to face to face conversation with a Holocaust survivor. These techniques allow many of the bene ts of an intimate connection to historical places and people, through direct interaction and user initiative, but can also be delivered to a mass audience, formerly only reachable by broadcast, non-interactive media.
236.
Rizzo, Albert; Scherer, Scherer; DeVault, David; Gratch, Jonathan; Artstein, Ronald; Hartholt, Arno; Lucas, Gale; Marsella, Stacy; Morbini, Fabrizio; Nazarian, Angela; Stratou, Giota; Traum, David; Wood, Rachel; Boberg, Jill; Morency, Louis Philippe
Detection and computational analysis of psychological signals using a virtual human interviewing agent Journal Article
In: Journal of Pain Management, pp. 311–321, 2016, ISSN: 1939-5914.
@article{rizzo_detection_2016,
title = {Detection and computational analysis of psychological signals using a virtual human interviewing agent},
author = {Albert Rizzo and Scherer Scherer and David DeVault and Jonathan Gratch and Ronald Artstein and Arno Hartholt and Gale Lucas and Stacy Marsella and Fabrizio Morbini and Angela Nazarian and Giota Stratou and David Traum and Rachel Wood and Jill Boberg and Louis Philippe Morency},
url = {http://www.icdvrat.org/2014/papers/ICDVRAT2014_S03N3_Rizzo_etal.pdf},
issn = {1939-5914},
year = {2016},
date = {2016-11-01},
journal = {Journal of Pain Management},
pages = {311–321},
abstract = {It has long been recognized that facial expressions, body posture/gestures and vocal parameters play an important role in human communication and the implicit signalling of emotion. Recent advances in low cost computer vision and behavioral sensing technologies can now be applied to the process of making meaningful inferences as to user state when a person interacts with a computational device. Effective use of this additive information could serve to promote human interaction with virtual human (VH) agents that may enhance diagnostic assessment. This paper will focus on our current research in these areas within the DARPA-funded "Detection and Computational Analysis of Psychological Signals" project, with specific attention to the SimSensei application use case. SimSensei is a virtual human interaction platform that is able to sense and interpret real-time audiovisual behavioral signals from users interacting with the system. It is specifically designed for health care support and leverages years of virtual human research and development at USC-ICT. The platform enables an engaging face-to-face interaction where the virtual human automatically reacts to the state and inferred intent of the user through analysis of behavioral signals gleaned from facial expressions, body gestures and vocal parameters. Akin to how non-verbal behavioral signals have an impact on human to human interaction and communication, SimSensei aims to capture and infer from user non-verbal communication to improve engagement between a VH and a user. The system can also quantify and interpret sensed behavioral signals longitudinally that can be used to inform diagnostic assessment within a clinical context.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
It has long been recognized that facial expressions, body posture/gestures and vocal parameters play an important role in human communication and the implicit signalling of emotion. Recent advances in low cost computer vision and behavioral sensing technologies can now be applied to the process of making meaningful inferences as to user state when a person interacts with a computational device. Effective use of this additive information could serve to promote human interaction with virtual human (VH) agents that may enhance diagnostic assessment. This paper will focus on our current research in these areas within the DARPA-funded "Detection and Computational Analysis of Psychological Signals" project, with specific attention to the SimSensei application use case. SimSensei is a virtual human interaction platform that is able to sense and interpret real-time audiovisual behavioral signals from users interacting with the system. It is specifically designed for health care support and leverages years of virtual human research and development at USC-ICT. The platform enables an engaging face-to-face interaction where the virtual human automatically reacts to the state and inferred intent of the user through analysis of behavioral signals gleaned from facial expressions, body gestures and vocal parameters. Akin to how non-verbal behavioral signals have an impact on human to human interaction and communication, SimSensei aims to capture and infer from user non-verbal communication to improve engagement between a VH and a user. The system can also quantify and interpret sensed behavioral signals longitudinally that can be used to inform diagnostic assessment within a clinical context.
237.
Pestian, John P.; Sorter, Michael; Connolly, Brian; Cohen, Kevin Bretonnel; McCullumsmith, Cheryl; Gee, Jeffry T.; Morency, Louis-Philippe; Scherer, Stefan; Rohlfs, Lesley
A Machine Learning Approach to Identifying the Thought Markers of Suicidal Subjects: A Prospective Multicenter Trial Journal Article
In: Suicide and Life-Threatening Behavior, 2016, ISSN: 03630234.
@article{pestian_machine_2016,
title = {A Machine Learning Approach to Identifying the Thought Markers of Suicidal Subjects: A Prospective Multicenter Trial},
author = {John P. Pestian and Michael Sorter and Brian Connolly and Kevin Bretonnel Cohen and Cheryl McCullumsmith and Jeffry T. Gee and Louis-Philippe Morency and Stefan Scherer and Lesley Rohlfs},
url = {http://doi.wiley.com/10.1111/sltb.12312},
doi = {10.1111/sltb.12312},
issn = {03630234},
year = {2016},
date = {2016-11-01},
journal = {Suicide and Life-Threatening Behavior},
abstract = {Death by suicide demonstrates profound personal suffering and societal failure. While basic sciences provide the opportunity to understand biological markers related to suicide, computer science provides opportunities to understand suicide thought markers. In this novel prospective, multimodal, multicenter, mixed demographic study, we used machine learning to measure and fuse two classes of suicidal thought markers: verbal and nonverbal. Machine learning algorithms were used with the subjects’ words and vocal characteristics to classify 379 subjects recruited from two academic medical centers and a rural community hospital into one of three groups: suicidal, mentally ill but not suicidal, or controls. By combining linguistic and acoustic characteristics, subjects could be classified into one of the three groups with up to 85% accuracy. The results provide insight into how advanced technology can be used for suicide assessment and prevention.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Death by suicide demonstrates profound personal suffering and societal failure. While basic sciences provide the opportunity to understand biological markers related to suicide, computer science provides opportunities to understand suicide thought markers. In this novel prospective, multimodal, multicenter, mixed demographic study, we used machine learning to measure and fuse two classes of suicidal thought markers: verbal and nonverbal. Machine learning algorithms were used with the subjects’ words and vocal characteristics to classify 379 subjects recruited from two academic medical centers and a rural community hospital into one of three groups: suicidal, mentally ill but not suicidal, or controls. By combining linguistic and acoustic characteristics, subjects could be classified into one of the three groups with up to 85% accuracy. The results provide insight into how advanced technology can be used for suicide assessment and prevention.
238.
Olszewski, Kyle; Lim, Joseph J.; Saito, Shunsuke; Li, Hao
High-fidelity facial and speech animation for VR HMDs Journal Article
In: ACM Transactions on Graphics, vol. 35, no. 6, pp. 1–14, 2016, ISSN: 07300301.
@article{olszewski_high-fidelity_2016,
title = {High-fidelity facial and speech animation for VR HMDs},
author = {Kyle Olszewski and Joseph J. Lim and Shunsuke Saito and Hao Li},
url = {http://dl.acm.org/citation.cfm?doid=2980179.2980252},
doi = {10.1145/2980179.2980252},
issn = {07300301},
year = {2016},
date = {2016-11-01},
journal = {ACM Transactions on Graphics},
volume = {35},
number = {6},
pages = {1–14},
abstract = {Several significant challenges currently prohibit expressive interaction in virtual reality (VR). The occlusion introduced by modern head-mounted displays (HMDs) makes most existing techniques for facial tracking intractable in this scenario. Furthermore, even state-of-the-art techniques used for real-time facial tracking in less constrained environments fail to capture subtle details of the user’s facial expressions that are essential for compelling speech animation. We introduce a novel system for HMD users to control a digital avatar in real-time while producing plausible speech animation and emotional expressions. Using a monocular camera attached to the front of an HMD, we record video sequences from multiple subjects performing a variety of facial expressions and speaking several phonetically-balanced sentences. These images are used with artist-generated animation data corresponding to these sequences to train a convolutional neural network (CNN) to regress images of a user’s mouth region to the parameters that control a digital avatar. To make training this system more tractable, we make use of audiobased alignment techniques to map images of multiple users making the same utterance to the corresponding animation parameters. We demonstrate that our regression technique is also feasible for tracking the expressions around the user’s eye region, including the eyebrows, with an infrared (IR) camera within the HMD, thereby enabling full facial tracking. This system requires no user-specific calibration, makes use of easily obtainable consumer hardware, and produces high-quality animations of both speech and emotional expressions. Finally, we demonstrate the quality of our system on a variety of subjects and evaluate its performance against state-of-the-art realtime facial tracking techniques.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Several significant challenges currently prohibit expressive interaction in virtual reality (VR). The occlusion introduced by modern head-mounted displays (HMDs) makes most existing techniques for facial tracking intractable in this scenario. Furthermore, even state-of-the-art techniques used for real-time facial tracking in less constrained environments fail to capture subtle details of the user’s facial expressions that are essential for compelling speech animation. We introduce a novel system for HMD users to control a digital avatar in real-time while producing plausible speech animation and emotional expressions. Using a monocular camera attached to the front of an HMD, we record video sequences from multiple subjects performing a variety of facial expressions and speaking several phonetically-balanced sentences. These images are used with artist-generated animation data corresponding to these sequences to train a convolutional neural network (CNN) to regress images of a user’s mouth region to the parameters that control a digital avatar. To make training this system more tractable, we make use of audiobased alignment techniques to map images of multiple users making the same utterance to the corresponding animation parameters. We demonstrate that our regression technique is also feasible for tracking the expressions around the user’s eye region, including the eyebrows, with an infrared (IR) camera within the HMD, thereby enabling full facial tracking. This system requires no user-specific calibration, makes use of easily obtainable consumer hardware, and produces high-quality animations of both speech and emotional expressions. Finally, we demonstrate the quality of our system on a variety of subjects and evaluate its performance against state-of-the-art realtime facial tracking techniques.
239.
Jones, J. Adam; Krum, David M.; Bolas, Mark T.
Vertical Field-of-View Extension and Walking Characteristics in Head-Worn Virtual Environments Journal Article
In: ACM Transactions on Applied Perception, vol. 14, no. 2, pp. 1–17, 2016, ISSN: 15443558.
@article{jones_vertical_2016,
title = {Vertical Field-of-View Extension and Walking Characteristics in Head-Worn Virtual Environments},
author = {J. Adam Jones and David M. Krum and Mark T. Bolas},
url = {http://dl.acm.org/citation.cfm?id=2983631},
doi = {10.1145/2983631},
issn = {15443558},
year = {2016},
date = {2016-10-01},
journal = {ACM Transactions on Applied Perception},
volume = {14},
number = {2},
pages = {1–17},
abstract = {In this article, we detail a series of experiments that examines the effect of vertical field-of-view extension and the addition of non-specific peripheral visual stimulation on gait characteristics and distance judgments in a head-worn virtual environment. Specifically, we examined four field-of-view configurations: a common 60° diagonal field of view (48° × 40°), a 60° diagonal field of view with the addition of a luminous white frame in the far periphery, a field of view with an extended upper edge, and a field of view with an extended lower edge. We found that extension of the field of view, either with spatially congruent or spatially non-informative visuals, resulted in improved distance judgments and changes in observed posture. However, these effects were not equal across all field-of-view configurations, suggesting that some configurations may be more appropriate than others when balancing performance, cost, and ergonomics.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
In this article, we detail a series of experiments that examines the effect of vertical field-of-view extension and the addition of non-specific peripheral visual stimulation on gait characteristics and distance judgments in a head-worn virtual environment. Specifically, we examined four field-of-view configurations: a common 60° diagonal field of view (48° × 40°), a 60° diagonal field of view with the addition of a luminous white frame in the far periphery, a field of view with an extended upper edge, and a field of view with an extended lower edge. We found that extension of the field of view, either with spatially congruent or spatially non-informative visuals, resulted in improved distance judgments and changes in observed posture. However, these effects were not equal across all field-of-view configurations, suggesting that some configurations may be more appropriate than others when balancing performance, cost, and ergonomics.
240.
Joshi, Himanshu; Rosenbloom, Paul S.; Ustun, Volkan
Continuous phone recognition in the Sigma cognitive architecture Journal Article
In: Biologically Inspired Cognitive Architectures, vol. 18, pp. 23–32, 2016, ISSN: 2212683X.
@article{joshi_continuous_2016,
title = {Continuous phone recognition in the Sigma cognitive architecture},
author = {Himanshu Joshi and Paul S. Rosenbloom and Volkan Ustun},
url = {http://linkinghub.elsevier.com/retrieve/pii/S2212683X16300652},
doi = {10.1016/j.bica.2016.09.001},
issn = {2212683X},
year = {2016},
date = {2016-10-01},
journal = {Biologically Inspired Cognitive Architectures},
volume = {18},
pages = {23–32},
abstract = {Spoken language processing is an important capability of human intelligence that has hitherto been unexplored by cognitive architectures. This reflects on both the symbolic and sub-symbolic nature of the speech problem, and the capabilities provided by cognitive architectures to model the latter and its rich interplay with the former. Sigma has been designed to leverage the state-of-the-art hybrid (discrete + continuous) mixed (symbolic + probabilistic) capability of graphical models to provide in a uniform non-modular fashion effective forms of, and integration across, both cognitive and sub-cognitive behavior. In this article, previous work on speaker dependent isolated word recognition has been extended to demonstrate Sigma’s feasibility to process a stream of fluent audio and recognize phones, in an online and incremental manner with speaker independence. Phone recognition is an important step in integrating spoken language processing into Sigma. This work also extends the acoustic front-end used in the previous work in service of speaker independence. All of the knowledge used in phone recognition was added supraarchitecturally – i.e. on top of the architecture – without requiring the addition of new mechanisms to the architecture.},
keywords = {},
pubstate = {published},
tppubtype = {article}
}
Spoken language processing is an important capability of human intelligence that has hitherto been unexplored by cognitive architectures. This reflects on both the symbolic and sub-symbolic nature of the speech problem, and the capabilities provided by cognitive architectures to model the latter and its rich interplay with the former. Sigma has been designed to leverage the state-of-the-art hybrid (discrete + continuous) mixed (symbolic + probabilistic) capability of graphical models to provide in a uniform non-modular fashion effective forms of, and integration across, both cognitive and sub-cognitive behavior. In this article, previous work on speaker dependent isolated word recognition has been extended to demonstrate Sigma’s feasibility to process a stream of fluent audio and recognize phones, in an online and incremental manner with speaker independence. Phone recognition is an important step in integrating spoken language processing into Sigma. This work also extends the acoustic front-end used in the previous work in service of speaker independence. All of the knowledge used in phone recognition was added supraarchitecturally – i.e. on top of the architecture – without requiring the addition of new mechanisms to the architecture.
Filter
Sorry, no publications matched your criteria.