Audra Coulombe is the Marketing Manager for The Molecule, a VFX, Motion Graphics, and VR company located in New York and Los Angeles.
At VFX studios worldwide, virtual reality has been a growing part of our pipeline for some time now. The field continues to be a sort of arms race — companies are competing to create the next big thing that will make VR easier to produce, more immersive, or more inexpensive. VR conferences have popped up all over the world, and exhibitions have all but taken over well-known festivals such as Sundance and SXSW.
The reason why VFX companies in particular are in the thick of the VR revolution is that, in order to create a professional, high-quality virtual reality project, you need a visual effects team. Unlike traditional production, where you might have a VFX shot here or there, in VR production, every shot is a VFX shot.
In this rush to get the next project out the door, sometimes these VR undertakings can be rushed, and the stitching or finishing can be sloppily done. Molecule CEO Chris Healer is fond of saying, “It’s really easy to get it 80% of the way done, but the last 20% can be difficult and expensive.”
Despite its challenges, however, producing VR is not as different from traditional VFX as people imagine. There are some key differences, but none that are too extreme. This post will walk you through the basics of our pipeline, focusing on the steps that tend to trip people up and noting where you can relax (because if you’ve produced any kind of video project before, it’ll sound pretty familiar).
VR Pre-Production
To create a great VR project, some of the most important decisions need to be made in pre-production and production. This principle is very similar to any project that requires visual effects — if you plan and shoot it well, post-production won’t be as laborious and expensive.
A good rule to live by is to bring your post facility and camera team together at the very beginning. Let them consult on best practices for shooting so the stitching runs smoothly.
Tip: There are a bajillion rigs out there. Ask yourself early on what is most important to you — is it image quality? Perfect stitches? Quick turnaround? Small budget? What is on your personal list will dictate what camera configuration you go with.
Production for a Golf VR experience for Samsung/PGA
VR Production
As mentioned above, the shooting of VR is actually pretty similar to basic video production, with a few key differences. Bear in mind that you have multiple cameras and everything in their line of sight will be seen.
- More cameras = more memory cards. Plan ahead and make sure you have enough data storage and an organized system for managing it all.
- More cameras = nowhere to hide! There is no behind-the camera position, so if you can see the camera, it can see you. This goes for equipment, too. When certain equipment (like dolly rigs, for example) are unavoidable, they will need to be painted out in post.
The Edit
All right, this is where it gets a little tricky.
Ingest. Remember that bullet point you just read about needing more memory cards? Now think of how much footage you usually have on a regular shoot, then multiply that by however many cameras you have. Plan for a few days of copying your footage to a server so that you can begin work. Six hours of 4k footage from eight cameras? Do the math… then take a short vacation while your footage copies.
Match camera (or camera-rig prep) is when you teach your computer the specifics of your rig. There are a number of different programs (for example, Nuke or AutoPanoPro) that can help you with this. Based on your template (4 GoPros pointed in these directions, with this type of lens, etc), your computer will be able to give you a basic stitch.
Tip: You can tell your computer the details of your rig or you can let it guess, but it’s usually best to give it all the information.
Once you teach the software your camera setup, you can produce a dailies stitch — basically, a rough stitch of a single take. It won’t look polished, because the stitches haven’t been perfected, but it’s certainly good enough to edit with. This is when you start piecing together the narrative of your story. In our office, we export these stitches as 4k x 2k flat, rectangular, lat-long videos that look like this:
An example of a dailies stitch. It has a lot of imperfections, but it gets the point across so you can put an edit together.
The last step in the story editing process is called an orientation pass. Remember that we’re dealing with 360 degrees of video, so the viewer can look in any direction he or she wants. As the filmmakers, we really want to nudge the viewer to look in one particular direction. This is why, once we have our linear narrative in place, we rotate the video like a kitchen timer to the position where we want our viewer to primarily focus.
Finishing and Exporting Your Project
After we’ve completed our edit, it’s time to play cleanup. This is the “last 20%” that Chris Healer was talking about, and it can be more of a beast than you think. The reason why this step is so crucial is that when you’re watching a story in a VR headset, the slightest anomaly can take you completely out of the experience. Going through the final stitch with a fine-toothed comb ensures that the viewer is completely immersed in the storytelling. Without this final step, the reality we want the viewer to experience can easily dissolve.
To us, this is the most time-consuming, detailed, frustrating, and rewarding part of the entire process. Here are a few examples of what to look out for:
Right angles: You’ll probably have no trouble stitching together a cloudless blue sky, but think about a drop ceiling. One ceiling can have hundreds of right angles that, when stitched poorly, don’t connect and can distract your viewer.
This guy is an example of a bad stitch.
Moving objects: if a person or animal crosses over a stitch line, you want to make sure that the viewer does not see the split. If stitched poorly, the person will look like he is cut into two pieces that don’t connect. And that just doesn’t occur in the real world.
Crew and equipment: As mentioned in the Production section, you’ll need to paint out anything that is not a part of your story. This is also where you would add any additional VFX and CG compositing.
When you’re ready to export your project, just like your import, you’re going to want to budget a day or two for render time and file transfer. Go take a second short vacation.
Tip: You can never check your final edit too many times for small errors. This final render takes a long time, so look over your project multiple times to make sure it’s perfect.
Of course, you’ll want to color grade your export, and you’ll want to add in your final audio mix. Both color and audio for VR warrant their own blog posts, so we’ll save that for another day, but this is the point at which you would add in those steps.
A finished stitch from the PGA/Samsung project, shown above.
Above is an example of what your finished lat-long image will look like. Feed this into any VR headset, and enjoy the fruits of your labor!