Near-Instant Capture of High-Resolution Facial Geometry and Reflectance

May 12, 2016 | Lisbon, Portugal

Speaker: Graham Fyffe
Host: Eurographics 2016, the 37th Annual Conference of the European Association for Computer Graphics

Modeling realistic human characters is frequently done using 3D recordings of the shape and appearance of real people across a set of different facial expressions [Pighin et al. 1998; Alexander et al. 2010] to build blendshape facial models. Believable characters which cross the “Uncanny Valley” require high-quality geometry, texture maps, reflectance properties, and surface detail at the level of skin pores and fine wrinkles. Unfortunately, there has not yet been a technique for recording such datasets which is near-instantaneous and relatively low-cost. While some facial capture techniques are instantaneous and inexpensive [Beeler et al. 2010; Bradley et al. 2010], these do not generally provide lighting-independent texture maps, specular reflectance information, or high-resolution surface normal detail for relighting. In contrast, techniques which use multiple photographs from spherical lighting setups [Weyrich et al. 2006; Ghosh et al. 2011] do capture such reflectance properties, but this comes at the expense of longer capture times and complicated custom equipment.

In this paper, we present a near-instant facial capture technique which records high-quality facial geometry and reflectance using commodity hardware. We use a 24-camera DSLR photogrammetry setup similar to common commercial systems1 and use six ring flash units to light the face. However, instead of the usual process of firing all the flashes and cameras at once, each flash is fired sequentially with a subset of the cameras, with the exposures packed milliseconds apart for a total capture time of 66ms, which is faster than the blink reflex [Bixler et al. 1967]. This arrangement produces 24 independent specular reflection angles evenly distributed across the face, allowing a shape-from-specularity approach to obtain high-frequency surface detail. However, unlike other shapefrom- specularity techniques, our images are not taken from the same viewpoint. Hence, we compute an initial estimate of the facial geometry using passive stereo, and then refine the geometry using separated diffuse and specular photometric detail. The resulting system produces accurate, high-resolution facial geometry and reflectance with near-instant capture in a relatively low-cost setup.

The principal contributions of this work are:

  • A near-instantaneous photometric capture setup for measuring the geometry and diffuse and specular reflectance of faces.
  • A camera-flash arrangement pattern which produces evenlydistributed specular reflections over the face with a single photo per camera and fewer lighting conditions than cameras.
  • A novel per-pixel separation of diffuse and specular reflectance using multiview color-space analysis and novel photometric estimation of specular surface normals for geometry refinement.