Facial performance synthesis using deformation-driven polynomial displacement maps
Abstract We present a novel method for acquisition, modeling, compression, and synthesis of reali... more Abstract We present a novel method for acquisition, modeling, compression, and synthesis of realistic facial deformations using polynomial displacement maps. Our method consists of an analysis phase where the relationship between motion capture markers and detailed facial geometry is inferred, and a synthesis phase where novel detailed animated facial geometry is driven solely by a sparse set of motion capture markers. For analysis, we record the actor wearing facial markers while performing a set of training expression clips.
Abstract We propose a novel post-production facial performance relighting system for human actors... more Abstract We propose a novel post-production facial performance relighting system for human actors. Our system uses just a dataset of view-dependent facial appearances with a neutral expression, captured for a static subject using a Light Stage apparatus. For the actual performance, however, a potentially different actor is captured under known, but static, illumination.
Abstract We demonstrate a novel light probe which can estimate the full dynamic range of a scene ... more Abstract We demonstrate a novel light probe which can estimate the full dynamic range of a scene with multiple bright light sources. It places diffuse strips between mirrored spherical quadrants, effectively co-locating diffuse and mirrored probes to record the full dynamic range of illumination in a single exposure. From this image, we estimate the intensity of multiple saturated light sources by solving a linear system.
Abstract: We present a fully image-based approach for capturing and modeling real human locomotio... more Abstract: We present a fully image-based approach for capturing and modeling real human locomotion under varying illumination and viewpoint that overviews the techniques and results presented by [Einarsson et al, 2006]. An actor performs repeatable locomotive actions (walking/running) on a rotating treadmill while being filmed from a vertical array of 3 high-speed cameras under controlled rapidly changing lighting conditions.
Abstract In this article we propose a new framework for capturing light transport data of a real ... more Abstract In this article we propose a new framework for capturing light transport data of a real scene, based on the recently developed theory of compressive sensing. Compressive sensing offers a solid mathematical framework to infer a sparse signal from a limited number of nonadaptive measurements. Besides introducing compressive sensing for fast acquisition of light transport to computer graphics, we develop several innovations that address specific challenges for image-based relighting, and which may have broader implications.
ABSTRACT We present a method that uses measured scene radiance and global illumination in order t... more ABSTRACT We present a method that uses measured scene radiance and global illumination in order to add new objects to light-based models with correct lighting. The method uses a high dynamic range imagebased model of the scene, rather than synthetic light sources, to illuminate the new objects. To compute the illumination, the scene is considered as three components: the distant scene, the local scene, and the synthetic objects.
Abstract We present a facial motion retargeting method that enables the control of a blendshape r... more Abstract We present a facial motion retargeting method that enables the control of a blendshape rig according to marker-based motion capture data. The main purpose of the proposed technique is to allow a blendshape rig to create facial expressions, which conforms best to the current motion capture input, regardless the underlying blendshape poses.
This landmark book is the first to describe HDRI technology in its entirety and covers a wide-ran... more This landmark book is the first to describe HDRI technology in its entirety and covers a wide-range of topics, from capture devices to tone reproduction and image-based lighting. The techniques described enable you to produce images that have a dynamic range much closer to that found in the real world, leading to an unparalleled visual experience.
We present a technique for relighting an image such that different areas of the image are illumin... more We present a technique for relighting an image such that different areas of the image are illuminated with different combinations of lighting directions. The approach is to capture a 4D reflectance field using a light stage, calculate the radial basis function interpolation of light constraints specified by users, and render the calculated illumination result in real-time using the GPU.
Abstract: We present a fully image-based approach for capturing and modeling real human locomotio... more Abstract: We present a fully image-based approach for capturing and modeling real human locomotion under varying illumination and viewpoint that overviews the techniques and results presented by [Einarsson et al, 2006]. An actor performs repeatable locomotive actions (walking/running) on a rotating treadmill while being filmed from a vertical array of 3 high-speed cameras under controlled rapidly changing lighting conditions.
Abstract. This paper presents a technique for representing and displaying high dynamic-range text... more Abstract. This paper presents a technique for representing and displaying high dynamic-range texture maps (HDRTMs) using current graphics hardware. Dynamic range in real-world environments often far exceeds the range representable in 8-bit per-channel texture maps. The increased realism afforded by a highdynamic range representation provides improved fidelity and expressiveness for interactive visualization of image-based models.
Capturing and Simulating Physically Accurate Illumination in Computer Graphics
Anyone who has seen a recent summer blockbuster has seen the results of dramatic improvements in ... more Anyone who has seen a recent summer blockbuster has seen the results of dramatic improvements in the realism of computer-generated graphics. Visualeffects supervisors now report that bringing even the most challenging visions of film directors to the screen is no longer in question; with today's techniques, the only questions are time and cost.
To increase the interest and engagement of middle school students in science and technology, the ... more To increase the interest and engagement of middle school students in science and technology, the InterFaces project has created virtual museum guides that are in use at the Museum of Science, Boston. The characters use natural language interaction and have near photoreal appearance to increase and presents reports from museum staff on visitor reaction.
Abstract We present a practical technique for image-based relighting under environmental illumina... more Abstract We present a practical technique for image-based relighting under environmental illumination which greatly reduces the number of required photographs compared to traditional techniques, while still achieving high quality editable relighting results. The proposed method employs an optimization procedure to combine spherical harmonics, a global lighting basis, with a set of local lights.
Abstract A new method is presented that takes as an input a high dynamic range image and maps it ... more Abstract A new method is presented that takes as an input a high dynamic range image and maps it into a limited range of luminance values reproducible by a display device. There is significant evidence that a similar operation is performed by early stages of human visual system (HVS). Our approach follows functionality of HVS without attempting to construct its sophisticated model. The operation is performed in three steps. First, we estimate local adaptation luminance at each point in the image.
ABSTRACT In this paper we present a method for recovering the reflectance properties of all surfa... more ABSTRACT In this paper we present a method for recovering the reflectance properties of all surfaces in a real scene from a sparse set of photographs, taking into account both direct and indirect illumination. The result is a lighting-independent model of the scene's geometry and reflectance properties, which can be rendered with arbitrary modifications to structure and lighting via traditional rendering methods.
Abstract We present a technique for capturing the extreme dynamic range of natural illumination e... more Abstract We present a technique for capturing the extreme dynamic range of natural illumination environments that include the sun and sky, which has presented a challenge for traditional high dynamic range photography processes. We find that through careful selection of exposure times, aperture, and neutral density filters that this full range can be covered in seven exposures with a standard digital camera.
Abstract We overview an approach to modeling and rendering architectural scenes from a sparse set... more Abstract We overview an approach to modeling and rendering architectural scenes from a sparse set of photographs. The approach is designed to require no special hardware and to produce real-time renderings on standard graphics hardware. The modeling method is an interactive photogrammetric modeling tool for recovering geometric models of architectural scenes from photographs.
Abstract���In this paper we overview the technology and production processes used to create ���Th... more Abstract���In this paper we overview the technology and production processes used to create ���The Parthenon���, a short computer animation that visually reunites the Parthenon and its sculptural decorations, separated since the beginning of the 19th century.
Abstract. In this paper we introduce a novel technique for estimating surface normals from the fo... more Abstract. In this paper we introduce a novel technique for estimating surface normals from the four Stokes polarization parameters of specularly reflected light under a single spherical incident lighting condition that is either unpolarized or circularly polarized. We illustrate the practicality of our technique by estimating surface normals under uncontrolled outdoor illumination from just four observations from a fixed viewpoint.
Uploads
Papers by Paul Debevec