Events

Andrew Jones, Graham Fyffe, XueMing Yu, Wan-Chun Ma, Jay Busch, Ryosuke Ichikari, Mark Bolas, Paul Debevec: “Head-mounted Photometric Stereo for Performance Capture”

November 17, 2011 | London, UK

Speaker: Andrew Jones, Graham Fyffe, XueMing Yu, Wan-Chun Ma, Jay Busch, Ryosuke Ichikari, Mark Bolas, Paul Debevec
Host: 8th European Conference on Visual Media Production (CVMP 2011)

Head-mounted cameras are an increasingly important tool for capturing facial performances to drive virtual characters. They provide a fixed, unoccluded view of the face, useful for observing motion capture dots or as input to video analysis. However, the 2D imagery captured with these systems is typically affected by ambient light and generally fails to record subtle 3D shape changes as the face performs. We have developed a system that augments a head-mounted camera with LED-based photometric stereo. The system allows observation of the face independent of the ambient light and generates per-pixel surface normals so that the performance is recorded dynamically in 3D. The resulting data can be used for facial relighting or as better input to machine learning algorithms for driving an animated face.