If you shoot video, you try to control as much as possible in order to create that perfect shot. You plan what, when, and where to shoot — even the composition, shot movements, and camera angles are meticulously thought out. But there are times when you really can’t prepare for everything, especially if you’re shooting in a public place. In this street footage by contributor StockElements, there are several distracting elements that would be good to remove.
Reviewing the footage reveals that doing this could be a bit tricky, because the elements we’ll want to remove sometimes go in and out of the frame.
Start with a Still
The first thing to do is export a still frame from the clip. We will then import this in Photoshop, where you can remove the unwanted areas. Go to a frame you like. Then go to Composition > Save Frame As. I’m exporting frame 148 of the clip because it has all the elements I want to remove.
You can change the file format of the still frame, the name, and where you want to save it.
Import the still frame in Photoshop and clean it up using any of your favorite tools, like the Clone Stamp, Healing Brush, or Content-Aware tool.
If you choose the the Clone Stamp, press “S” to select the tool and zoom in to your image. Next press, Alt on a PC or Option on a Mac, then click on area you want to copy. I started by removing the graffiti.
Here’s my cleaned-up up image. You can see I cloned out the side of the truck, the trash, the branding on the mud flaps, and the box with a patch.
Bring your clean image back in to After Effects and put it on top of your clip. Before we do any tracking, we must first isolate the parts of the image we will use.
Select your clean image and press “G” to use the mask tool. Then draw a mask on the side of the truck.
After creating your mask, go to Layer > Precompose. The Precompose window will pop-up. Give it a name, then choose “Move all attributes into the new composition.”
Repeat this process for all the other parts that you need to replace, like the patch, branding, and trash. Mask them out and precompose them, as well. This is how the composition should look.
Move into Mocha
Doing this alone with After Effects would be a bit tricky, because the areas that we want to remove go in and out the frame. In order to track this more efficiently, we’ll use Mocha AE by Imagineers Systems, which comes free with After Effects.
Turn off all the other layers except the truck-side precomp. Select the clip on the timeline, then go to Animation Track in Mocha AE. This will launch Mocha AE.
Mocha will automatically import your clip and set up the project for you, so all you need to do is to change the file name and the save directory. Press “OK” when you’re done.
Mocha is a planar tracker — this means that it tracks planes instead of points. We will need to track our clip several times, because the objects we want to fix are on different planes.
Mocha has two types of splines: X Splines and Bézier Splines.
a. X Splines
b. Bézier Splines
c. Track Module
d. User drawn spline (red line) and surface (blue square). The spline serves as the search region for Mocha while it’s tracking. The surface represents the 4 points of the corner pin that will be placed in After Effects. (Go to View > Overlays > Surface to turn this on.)
e. Mocha determines the best number of pixels to look for. You can bring the percentage up to increase accuracy. Note that this will also increase tracking time, however. (The default settings usually work fine.)
f. This controls what motion is being looked at when tracking.
g. Track forward and backward.
h. Allows you to jump to keyframes.
i. Selected layer. It’s always a good practice to rename your layers.
What’s great with Mocha is that the splines you create are not the tracking data — they are, however, linked to the tracking data. What this means is that you can create a spline or add several splines on a layer, and Mocha will then use them splines as search areas for tracking.
Select the X spline and draw on the side of the truck. I don’t need to draw on the entire side of the truck, since the front part will go out of the frame. You just need a portion of the area to suffice.
You can also turn on the surface tool and align it on the corners of the truck. If you like, you can turn on the grid beside the surface tool, as well. This is a great way to monitor your track and see if it’s slipping.
Proceed by pressing the forward or backward button.
Your track can also be adjusted via the Adjust Track module; this is beyond the scope of this tutorial, but there are a lot of tutorials on the Imagineers website on this.
Once the tracking is done, click on “Export Tracking Data” on the lower right corner of the screen.
This will open up the Export Tracking Data window. Choose “After Effects Corner Pin” then “Copy to Clipboard.”
In After Effects, go to the beginning of the timeline. Select the Truck Side pre-comp and go to Edit > Paste.
This applies the corner-pin effect to the layer with all the keyframes needed. The track is perfect. However, this is not the result that we want. Aligning the surface on the corners of the truck would only work if our image was fully rectangular. In this case, our image is skewed because it’s from a still frame of the clip. This situation happens very often in the real world. So we have to go through a few more steps to make sure that everything is in order.
Undo the last action or delete the corner-pin effect that was applied to your layer.
Let’s go back to Mocha to correct our problem. For this to work, you need to know which frame of the video you exported and cleaned. In this case, I exported frame 148 of the video.
Go to frame 148 by dragging the playhead on the timeline, or by typing the number in the Current Frame field.
Once you are on the correct frame, choose “Align Selected Surfaces.” This will fit the surface to the full dimensions of the footage.
The surface before alignment:
And after aligning the surface:
Repeat the process from before. Export the tracking data, then choose After Effects Corner Pin. Go back to AE, select the truck-side pre-comp, and go to Edit > Paste to apply the tracking data.
We now have the correct tracking data applied to our clean image.
Repeat these steps for all the other areas. The process will be the same — draw an X Spline, track, align the surface, export the tracking data, and paste the data in AE. Make sure to always properly label your layers, both in AE and Mocha. Do the tracking of the trash last, because it needs some extra steps.
In Mocha, draw a spline around the trash.
Next, click on “Add X Spline to Layer.” This will add another spline to you current layer.
Draw another spline, this time around the box.
Another technique is to track the pole and use that as a hold-out matte for your trash track. But for now, let’s just use this technique, as it’s much simpler. We now have two splines that Mocha will use for tracking.
Technically, the objects we’re tracking are not exactly on the same plane, so we’re actually cheating. But the thing you have to remember with visual effects is that sometimes it’s not what is right — it’s what looks right. And in this case, we can really get away with it, because the snow around the trash doesn’t have a noticeable pattern, plus it’s not the subject of the clip.
Before tracking, toggle the Eye and Cog on the other layers. This will make tracking faster, because turning off the Cog on the other layers ensures that Mocha won’t track those layers again.
Proceed with tracking. Don’t forget to align the surface before exporting the tracking data in AE.
This is our finished product:
The life of a visual effects artist will always involve removing things on a shot. It could be a wire, a reflection, a billboard — almost anything. This technique can help you tackle this and take your shot from good to great. AE is a great program for doing visual effects but combining it with Mocha makes it even more powerful and makes your life a lot easier. If you can track a shot, then you can do almost anything in AE.