Evaluation of Off-the-shelf Speech Recognizers Across Diverse Dialogue Domains (bibtex)
by Georgila, Kallirroi, Leuski, Anton, Yanov, Volodymyr and Traum, David
Abstract:
We evaluate several publicly available off-the-shelf (commercial and research) automatic speech recognition (ASR) systems across diverse dialogue domains (in US-English). Our evaluation is aimed at non-experts with limited experience in speech recognition. Our goal is not only to compare a variety of ASR systems on several diverse data sets but also to measure how much ASR technology has advanced since our previous large-scale evaluations on the same data sets. Our results show that the performance of each speech recognizer can vary significantly depending on the domain. Furthermore, despite major recent progress in ASR technology, current state-of-the-art speech recognizers perform poorly in domains that require special vocabulary and language models, and under noisy conditions. We expect that our evaluation will prove useful to ASR consumers and dialogue system designers.
Reference:
Evaluation of Off-the-shelf Speech Recognizers Across Diverse Dialogue Domains (Georgila, Kallirroi, Leuski, Anton, Yanov, Volodymyr and Traum, David), In Proceedings of the 12th Language Resources and Evaluation Conference, European Language Resources Association, 2020.
Bibtex Entry:
@inproceedings{georgila_evaluation_2020,
	address = {Marseille, France},
	title = {Evaluation of {Off}-the-shelf {Speech} {Recognizers} {Across} {Diverse} {Dialogue} {Domains}},
	url = {https://www.aclweb.org/anthology/2020.lrec-1.797/},
	abstract = {We evaluate several publicly available off-the-shelf (commercial and research) automatic speech recognition (ASR) systems across diverse dialogue domains (in US-English). Our evaluation is aimed at non-experts with limited experience in speech recognition. Our goal is not only to compare a variety of ASR systems on several diverse data sets but also to measure how much ASR technology has advanced since our previous large-scale evaluations on the same data sets. Our results show that the performance of each speech recognizer can vary significantly depending on the domain. Furthermore, despite major recent progress in ASR technology, current state-of-the-art speech recognizers perform poorly in domains that require special vocabulary and language models, and under noisy conditions. We expect that our evaluation will prove useful to ASR consumers and dialogue system designers.},
	booktitle = {Proceedings of the 12th {Language} {Resources} and {Evaluation} {Conference}},
	publisher = {European Language Resources Association},
	author = {Georgila, Kallirroi and Leuski, Anton and Yanov, Volodymyr and Traum, David},
	month = may,
	year = {2020},
	keywords = {Virtual Humans},
	pages = {6469--6476}
}
Powered by bibtexbrowser