Emerging Technologies: An Auto-Multiscopic Projector Array for Interactive Digital Humans

August 9, 2015 | Los Angeles, CA

Speaker: Andrew Jones, Koki Nagano, Jay Busch, Xueming Yu, Hsuan-Yueh Peng, Oleg Alexander, Paul Debevec
Host: The 42nd International Conference and Exhibition on Computer Graphics and Interactive Techniques

Automultiscopic 3D displays allow a large number of viewers to experience 3D content simultaneously without the hassle of special glasses or head gear. This display uses a dense array of 216 video projectors to generate images with high angular density over a wide field of view. As users move around the display, their eyes smoothly transition from one view to the next. The display is ideal for displaying life-size human subjects, as it allows for natural personal interactions with 3D cues such as eye-gaze and spatial hand gestures.

The installation presents “time-offset” interactions with recorded 3D human subjects. A large set of video statements was recorded for each subject, and users access these statements through natural conversation that mimics face-to-face interaction. Conversational reactions to user questions are retrieved through speech recognition and a statistical classifier that finds the best video response for a given question. Recordings of answers, listening, and idle behaviors are linked together to create a persistent visual image of the person throughout the interaction. This type of time-offset interaction can support a wide range of applications, from creating entertaining performances to recording historical figures for education.