Matching Live-Action Background Plates
Many 3D programs have an option to view background plates within an animation window (an image plane or rotoscope background), showing how your subject will be aligned with the background. Once your 3D scene is aligned with the camera angles from the real-world shot, you face the challenge of matching the lighting from the real-world environment. Matching the direction, color, and tone of the light sources in the real scene is essential to integrating your 3D rendered passes with the photographed background plate.
A set of reflective and matte balls can be ideal reference objects to help measure the position and color of lights on a location. Mirrored balls are sold as lawn ornaments, ball bearings, and as housings for ceiling-mounted security cameras. For a ball with a matte finish, plaster is ideal, but a Styrofoam ball from a craft store could be more portable and affordable. You may need to paint the ball gray and to attach a piece of wire to hold it in place.
A photograph showing the matte ball in a lighting environment can be great for picking the color of light reaching your subject from each direction, as shown in Figure 11.35. Ideally, this image should be shot with the same camera as your final background plate, and digitized at the same time with the same settings.
Figure 11.35. A matte-finished gray ball is positioned between a fire and window to probe different colors in the scene.
For the most accurate color matches, pick specific RGB color values from the image of the ball, as shown in Figure 11.36. You can then assign these RGB colors directly as colors for your lights from corresponding directions.
Figure 11.36. RGB colors picked off the reference ball will give an accurate color match to 3D lights.
When developing the scene's lighting, you can import the ball image as a background in your 3D program, and create a 3D sphere in front of it. Using your 3D sphere as a reference, adjust infinite or directional lights from each direction to make the shading of the 3D sphere match the shading of the ball in the background plate.
Studying the colors reaching a point in a real-world environment is a great exercise for anyone working in 3D lighting. Even if you don't need to match the lighting of a background plate right now, this process could be worth trying a few times, just to get a better feel for the colors of real-world lights.
A picture of a reflective ball in an environment, like the one shown in Figure 11.37, helps you more precisely determine the angle and relative brightness of each light, and guides you in creating highlights and reflections for your object. It is best if the reflective ball is shot from the same camera position as your final background plate.
Figure 11.37. A mirror ball captures a reflected image of the surrounding environment for reference or for use as a reflection map.
As with the matte ball image, the reflective ball image can be brought into your 3D program. If you make a shiny 3D sphere, you should be able to see highlights from your brighter lights, and match these to the highlights in the reflective ball.
An added bonus to having a picture of a reflective sphere in your environment is that you can use it to develop a reflection map for your object, as shown in Figure 11.38. In many programs, the best way to project your acquired image is as a planar projection onto the side of a large sphere surrounding your 3D scene; make the sphere render reflections only.
Figure 11.38. The image from the mirror ball makes a perfect reflection map.
The traditional approach to matching natural light from a real environment is to use an array of infinite or directional lights, as described above. Once each light is properly adjusted to match the color and brightness from each direction, this approach can produce realistic renderings and seamless lighting matches.
Another approach to re-creating a real-world lighting environment is to use a light probe image recorded on the same location as your background plate. A light probe image captures the lighting from all angles around a subject, which can be created by photographing a reflective ball at the shooting location. The light probe image can then be used to illuminate objects with all the recorded colors and tones of real-world light. From only one light probe image, illumination will reach 3D objects from all angles, as though the light source were a giant sphere wrapped around the entire 3D scene.
Unlike ordinary photographs of a reflective ball, light probe images are high dynamic range images (HDRI), meaning that they can capture an exposure latitude greatly exceeding the range of one visible image. To photograph light probe images, cameras are programmed to shoot a series of images at different exposure settings, exposing for the brightest light sources all the way down to the darkest, as shown in Figure 11.39. Without using HDRI, all of the brighter lights in a scene might appear clipped as pure white highlights, with no record of their relative brightness or color. Using HDRI, a light probe image can accurately record the color and relative brightness of every light source.
Figure 11.39. A high dynamic range image will indicate accurate colors at multiple levels of exposure.
Other Approaches to Matching Lighting
You can't always use probes and reflective balls on the set, nor can you expect every production to stop and wait for you to set up special reference shots. Sometimes you won't even be able to visit the location where background plates were photographed.
Even if you do get to measure the lighting with different kinds of balls, the lighting in the scene may change without being remeasured. Also, balls in one location in a scene may fail to give you the information you need about lighting in another pointyou'd need an infinite number of probes to fully measure the light at every point in space.
If you can go to the set or shooting location, you can use other techniques to assist in matching the lighting:
If you cannot go to the shooting location, or your background plate comes from stock footage or other sources, you still can match the lighting using other techniques:
Every production will create different challenges, but with this basic set of tricks, you should be able to match the lights from any background plate.