Towards a Multimodal Taxonomy of Dialogue Moves for Word-Guessing Games

May 27, 2014 | Reykjavik, Iceland

Speaker: Eli Pincus
Host: Multimodal Corpora workshop 2014

We develop a taxonomy for guesser and clue-giver dialogue moves in word guessing games. The taxonomy is designed to aid in the
construction of a computational agent capable of participating in these games. We annotate the word guessing game of the multimodal
Rapid Dialogue Game (RDG) corpus, RDG-Phrase, with this scheme. The scheme classifies clues, guesses, and other verbal actions
as well as non-verbal actions such as gestures into different types. Cohen kappa inter-annotator agreement statistics for clue/non-clue
and guess/non-guess are both approximately 76%, and the kappas for clue type and guess type are 59% and 75%, respectively. We
discuss phenomena and challenges we encounter during annotation of the videos such as co-speech gestures, gesture disambiguation,
and gesture discretization.