Panel Discussion: Next Gen Evaluation of VR Interfaces

March 26, 2015 | Provence, France

Speaker: Evan Suma
Host: 2015 IEEE Virtual Reality Conference
Much of the work on evaluating the usability of VR systems over the past 15 years has focused on fairly low-level tasks, mainly based on Bowman et al.’s so called “Big Five” basic tasks of object Selection, Manipulation, Navigation, Symbolic Input, and System Control. Some additions to this have been discussed, including a) adding Avatar Control, due to the emergence of low-cost body part tracking systems (e.g., Kinect, Leap Motion), b) combining Selection and Manipulation into one, as they are so closely related, and c) splitting Navigation into two, Travel and Wayfinding, since many solutions exist for each of these individually. Even with these tweaks, however, it could be argued that research into interaction has matured to such a point that many viable solutions to each of these tasks exist, and that while we should not abandon this low-level research effort, greater impact could be made more rapidly by shifting focus to higher-level tasks and topics. Also, studying the low-level tasks in isolation ignores the fact that applications require users to physically and mentally switch between tasks, and that studying multiple low-level interface solutions together, along with the required transitions between them, is vital to user acceptance. In this panel, we explore several possible lines of evaluation, in the hopes of encouraging researchers and practitioners to think more impactfully about designing and evaluating their systems. Some of the work can be classified as “Fielded Studies,” where VR has been introduced into traditional workflow settings (e.g., medical student training), and evaluation has focused on how results from such systems relate to traditional approaches. Another tack is to design and evaluate from a user experience (UX) perspective. One possible future use of VR as posited in many works of popular fiction (e.g., Neuromancer, Ready Player One) is that we will spend most of our time wearing VR headsets and input gloves. Well, why not try it now, using today’s technology? Long-term, multi-person exposure approaches are now well within reach of most research budgets. In particular, gaming has been driving VR-related technology advancements for more than a decade, however it is not until recently that VR researchers have begun to focus some effort on formally designing for gaming experiences.