Incorporating Your Green Screen Talent into Virtual Sets

April 23, 2015 by Jeff Foster

by Jeff Foster

Virtual sets are digitally created scenes into which you can composite green screen subjects. In most cases, they’re computer-generated 3D studios, such as you see on the evening news and special programming on CNN, Fox News, 60 Minutes and COSMOS: A Spacetime Odyssey or in the many various sports programs on network television or even motion pictures. In most cases, you may not even know you’re viewing a virtual set behind the on-screen talent – and that’s exactly the result you’re hoping to achieve!

But proper planning for shooting and lighting your green screen and positioning your talent will make your desired result more successful.

TriCaster and LiveSet Virtual Sets

Many of the TriCaster models include their library of LiveSet virtual set designs that you can use with your green screen subjects. You simply set up and light your green screen and subjects, position them properly using the built-in guides and the TriCaster places them into the virtual set in real-time.

FIG 02

With the TriCaster you can select several cameras positioned around your green screen subject and align the virtual set to switch from camera to camera. The TriCaster can place live, simulated shadows and reflections onto desk and background surfaces from the matted live video feed, providing a realistic composite.

FIG 03

It’s important to make sure your lighting is appropriate on your subjects so they’ll best match the type of scene you’re putting them into. This means that if the virtual set resembles a TV studio, then you should have full, even lighting on your subject. You must also be sure that your camera height while shooting your subject matches the angle of the environment they’re to be composited into. If they’re supposed to be seated at a desk, then it’s best to have them sit on a stool facing the correct direction, but watch their posture! Often times people tend to slouch when seated for a long period of time, or tend to rock/sway in their seat unknowingly.

Shooting for Pre-Made Virtual Sets

If you’re using any pre-made virtual set, whether you’re shooting live with the TriCaster or compositing in post-production software, it’s essential that you have the background shots available on set before you set up the lighting and positioning of your cameras and green screen subjects. This will ensure that you have the correct angles and scalability when you go to composite them into the scene.

In this example, we started with a lobby scene that was pre-rendered and layered objects such as tables and desks were already matted out for us. What I did with these students at the film school was to have them create some objects they could stand behind or sit on to properly position them for their intended roles in the scene. We made sure the angles and lighting all matched each position, so when we made the final composite, they were aligned perfectly to the virtual set.

FIG 05

FIG 06

Creating or Modifying Your Own 3D Virtual Set

If you have a desktop 3D software application in your tool set, then you have everything you need to build your own 3D virtual set—or at least the ability to modify an existing set if you can get the raw data files to work with and they’re compatible with your software.

I was able to obtain some very well-constructed and realistic scenes to use as sets from Jonny Kim at PhotonCraft (www.photoncraft.com). These are available to download and edit as you need, including changing or animating cameras to suit your foreground footage. I was able to use one of these scenes in Strata 3D (www.strata.com) and change camera angles to suit my needs. Notice in the figures below that the geometry of the models appears simple when viewed in the wireframe mode; but when the scene is rendered, you can see all the details of the textures, lighting, and reflections. This virtual set could be used for several interior and exterior scenes, depending on where you positioned the 3D camera in the software. The beauty of creating your own sets or using a pre-made set like this is that you can have total control of the scene and where your talent is going to be positioned, as well as camera angles.

FIG 08

In Strata 3D, I adjusted the camera angle to closely match the angle of the green screen footage clips I already had. In this case, I was only able to visually match up the camera focal length, distance, and height, but the results were still pretty close. Rendering quick wireframe views, I could match the clips and make adjustments as necessary before creating a high-resolution detailed render.

Using the wireframes or even pre-rendered camera angles, you could also set up your green screen shots with the TriCaster and use the Live Key mode to align your talent to the shots, and then record your green screen video straight for compositing later in post-processing software.

This workflow is used a lot with indie filmmakers that want to have a good alignment while shooting their green screen plates and preview how the scene is going to look when finally composited with layers of effects with a software compositor/animator in post. Such a process was used on the Indie film “Hero Punk” (http://www.heropunk.com) by filmmaker Kanen Flowers.

Since the entire film was to be shot on green screen, he had loose storyboards created to build his virtual sets and then used rough renders to overlay his green screen shots live to be sure his lighting and camera positioning was accurate. When he later did the final compositing in post production, he was assured that every scene would line up correctly as a result.

FIG 09