Ryan Baker: “Detecting and Adapting to When Students Game the System”

March 30, 2007 | USC ICT

Speaker: Ryan Baker

Students use interactive learning environments in a considerable
variety of ways. In this talk, I will present research on developing learning
environments that can automatically detect and adapt when a student is
“gaming the system”, attempting to succeed in a learning environment by
exploiting properties of the system rather than by learning the material and
trying to use that knowledge to answer correctly. My colleagues and I have
determined across several studies that gaming the system is replicably
associated with low learning in intelligent tutors, and that gaming has
different effects, depending on when and why students game.

In this talk, I will present a detector that reliably detects gaming, in order to
drive adaptive support. In order to predict both which students game, and when
a specific student is gaming, this detector was trained using a combination of
a psychometric modeling framework, Latent Response Models (Maris, 1995)
and a machine-learning space-searching technique, Fast
Correlation-Based Filtering (Yu and Liu, 2003), using a mixture of labeled
and unlabeled data at different grain-sizes. My colleagues and I have
validated that this detector transfers effectively between several
intelligent tutor lessons without re-training, despite the lessons varying
considerably in their subject matter and user interfaces.

The gaming detector has been used to develop a tutor which responds
to gaming. Within this system, a software agent (“Scooter the Tutor”)
indicates to the student and their teacher whether the student has
been gaming recently. Scooter also gives students supplemental
exercises, in order to offer the student a second chance to learn the
material he/she had gamed through. Scooter reduces the frequency of
gaming by over half, and Scooter’s supplementary exercises are
associated with substantially better learning; Scooter appears to have
had virtually no effect on students who do not game.