Assessing Agreement in Human-Robot Dialogue Strategies: A Tale of TwoWizards (bibtex)
by Matthew Marge, Claire Bonial, Kimberly A. Pollard, Ron Artstein, Brendan Byrne, Susan G. Hill, Clare Voss, David Traum
Abstract:
The Wizard-of-Oz (WOz) method is a common experimental technique in virtual agent and human-robot dialogue research for eliciting natural communicative behavior from human partners when full autonomy is not yet possible. For the first phase of our research reported here, wizards play the role of dialogue manager, acting as a robot’s dialogue processing. We describe a novel step within WOz methodology that incorporates two wizards and control sessions: the wizards function much like corpus annotators, being asked to make independent judgments on how the robot should respond when receiving the same verbal commands in separate trials. We show that inter-wizard discussion after the control sessions and the resolution with a reconciled protocol for the follow-on pilot sessions successfully impacts wizard behaviors and significantly aligns their strategies. We conclude that, without control sessions, we would have been unlikely to achieve both the natural diversity of expression that comes with multiple wizards and a better protocol for modeling an automated system.
Reference:
Assessing Agreement in Human-Robot Dialogue Strategies: A Tale of TwoWizards (Matthew Marge, Claire Bonial, Kimberly A. Pollard, Ron Artstein, Brendan Byrne, Susan G. Hill, Clare Voss, David Traum), In Proceedings of The Sixteenth International Conference on Intelligent Virtual Agents (IVA 2016),, Springer, 2016.
Bibtex Entry:
@inproceedings{marge_assessing_2016,
	address = {Los Angeles, CA},
	title = {Assessing {Agreement} in {Human}-{Robot} {Dialogue} {Strategies}: {A} {Tale} of {TwoWizards}},
	url = {http://iva2016.ict.usc.edu/wp-content/uploads/Papers/100110460.pdf},
	abstract = {The Wizard-of-Oz (WOz) method is a common experimental technique in virtual agent and human-robot dialogue research for eliciting natural communicative behavior from human partners when full autonomy is not yet possible. For the first phase of our research reported here, wizards play the role of dialogue manager, acting as a robot’s dialogue processing. We describe a novel step within WOz methodology that incorporates two wizards and control sessions: the wizards function much like corpus annotators, being asked to make independent judgments on how the robot should respond when receiving the same verbal commands in separate trials. We show that inter-wizard discussion after the control sessions and the resolution with a reconciled protocol for the follow-on pilot sessions successfully impacts wizard behaviors and significantly aligns their strategies. We conclude that, without control sessions, we would have been unlikely to achieve both the natural diversity of expression that comes with multiple wizards and a better protocol for modeling an automated system.},
	booktitle = {Proceedings of {The} {Sixteenth} {International} {Conference} on {Intelligent} {Virtual} {Agents} ({IVA} 2016),},
	publisher = {Springer},
	author = {Marge, Matthew and Bonial, Claire and Pollard, Kimberly A. and Artstein, Ron and Byrne, Brendan and Hill, Susan G. and Voss, Clare and Traum, David},
	month = sep,
	year = {2016},
	keywords = {ARL, UARC, Virtual Humans}
}
Powered by bibtexbrowser