University of Southern California

Visual Sensing

In order to maximize the impact of an immersive training environment, virtual humans in that environment must be aware of the trainee's actions. The Visual Sensing project develops techniques for such awareness by enabling visual communication between humans and virtual humans. By including awareness of the trainee's position, posture, and actions, these techniques serve to make interactions between a trainee and a synthetic agent appear more natural. The project supports the Integrated Virtual Human project.

This project differs from others in the following ways:

  • It detects and tracks human movement without markers or special clothing (most other research requires markers or special clothing to make detection easier)
  • It uses a dark SASO-ST environment for training (no other research has attempt detection of human shapes in darkened rooms)
  • It analyzes gestures that occur in normal human-to-human interactions (most other research in recognition of body gestures has been for explicit communication, such as sign language)
  • It uses 3-D motion and shape data inferred from 5 video camera sensors, enabling different viewpoints for body construction (most other research relies on 2-D analysis)

Tags: communications, human, support, virtual

View All Projects »

Goals

  • Locate the trainee in the environment and construct a 3-D shape of his/her body
  • Develop methods for recognizing pointing gestures (and the direction of pointing) and estimating head pose
  • Capture human body and limb motions and infer natural gestures that enable inference of trainee emotional state and supplement the verbal communication with nonverbal cues.