Stories can be told in many different ways, using many different media types and image-capturing devices. Sometimes you may want to incorporate still images in your project, or the producer may call for a piece of smartphone footage to be cut into your edit. Chances are that these types of additions won’t match perfectly within your 16:9, 24-fps timeline and you’ll need to adjust your clips and settings so that it’s not too jarring of a departure visually or emotionally.
To do this, you’ll need to be wary of the problems that can arise when you’re working with different file types and frame rates, like image cropping, black bars/letterboxing, dropped frames, image stretching, and other things that aren’t coming across correctly in your vision for the piece. You also need to know how to fix them appropriately — especially if you’re working with tons of different types of files.
To Conform or Not to Conform
The first thing you need to figure out is if you actually want to have everything match within your edited sequence (frame rate, resolution, codec, etc). In my experience, I’ve almost always converted everything beforehand to identical settings so that I won’t have to worry about any aberrations in my footage, or any performance issues from my editing software.
You can use any video-conversion software (I use MPEG Streamclip for nearly everything), or you can do a batch export from AE, Premiere, or Final Cut and get everything to match perfectly (excluding vertical, 3D, and VR video, of course). I recommend conforming your media beforehand, as well, especially for things like demo reels or documentaries — but if you don’t care to do it before, you can use your editing software if it allows it.
Premiere, for instance, will let you add multiple file types, frame rates, and aspect ratios all in the same sequence (it will even ask if you want to change the settings to match the clip, but you’ll want to select “Keep Existing Settings”). Then you simply export out the settings you want and it will make everything conform automatically. However, if you have a lot of different files in there, it can cause some lag and, in severe cases, can cause the program to crash. They definitely are constantly working on this function, but just be mindful that you may have issues with too many different files.
The last thing I’ll say in this section is that there are a LOT of ways you can go just in this initial stage. You may want everything to be in the same frame rate, but you may not care that you have a UHD clip or two in there among a 1080p sequence. Or maybe you’ve got a bunch of clips that are 29.97 fps and a few that are 60 fps. As long as you’re aware that it may look different when you export, or if that’s a certain look you’re going for, then you’re fine just editing as is.
Setting Up a Sequence
Your sequence’s settings will determine how your project looks and how the added media will react, so you need to decide what your piece’s final look will be when you create your sequence. Usually, you can create a sequence from a single clip, so keep that clip’s settings in mind if you go this route, because it may not be the same as the rest of your files. You’ll want to choose a file that best illustrates the settings you’re going for. Once you start importing other clips, keep an eye on the program window to make sure nothing looks off and that there aren’t black bars or a cropped image (more on fixing those later).
As I said earlier, Premiere will ask if you want to keep the existing settings or change them to match the new clip (only if it’s different than your current sequence), so click “Keep Existing Settings” if you want to keep the sequence the same and adjust the new clip around it.
Determining Frame Rate
First a little background on frame rates: Options for your frame rate while shooting include (but aren’t limited to): 23.976, 23.98, 24, 25 (PAL), 29.97, 30, 48, 50 (PAL), 59.97, 60, 96, 120, 240, etc. In editing, however, you hardly see anything over 60 fps. All frame rates have their drawbacks and benefits, but the differences in the look of the video are pretty noticeable even to untrained eyes, so keep that in mind when making your project.
Most feature films are 24 fps and have been for decades; most web video is 30 fps; and the Hobbit movies (48 fps) and a few other ambitious examples have been shown at higher frame rates, but haven’t been received all that well. It’s actually fairly rare to see a sequence or video that is playing back at anything over 30 fps, so give it some thought when you’re picking your frame rate (because it does matter). Here’s a trailer for the latest Fast and Furious at 24p:
Now watch this trailer in 120 fps and you’ll notice the difference:
Once you’ve picked your frame rate, start importing your footage. Anything that is different from your sequence will be adjusted (so long as you click “Keep Existing Settings”), whether it’s a 3:2 pulldown or another really technical thing I can’t really explain. The clips will be adjusted by dropping, adding, and/or blending frames together to make them fit together in the sequence. If you haven’t conformed them, watch out for anything looking off.
*Dropped frames can look like a sudden jump or lag at an edit point, so watch your video thoroughly and see anything catches your eye.
Choosing an Aspect Ratio
I won’t go into much background on aspect ratio and all the standard ratios used today, because most of the time footage is shot and delivered in 16:9 — but I will say that the aspect ratio not only affects the aesthetic, but the feel of a video, as well. Wes Anderson famously used three different aspect ratios for The Grand Budapest Hotel, all noting different time periods and having different styles. And in big, epic films, 2.35:1 aspect ratios are often used to show the, well, big, epic scope of things.
However, with all that being said, since most video is 16:9 these days, and almost all televisions and monitors are 16:9, that should be your baseline, and you can work from there. Here are the basic issues and solutions. (For this example, imagine that we’re on a 16:9 project.)
Cropping: Any time you bring in a different aspect ratio than that of your current sequence, pixels outside of the 16:9 frame are going to get cropped/cut off. For instance, if you’re using 4096×2160 footage but your project is 3840×2160, you’re losing 256 horizontal pixels in the frame and may cut out important details. You’ll need to position the clip the best way possible by scaling or moving its position to where it works best. You may get black bars, however, which I’ll go over next.
Pillarboxing: This is the official term for those vertical black bars you see on the side of a video. This typically happens when the clip’s aspect ratio is 4:3, and especially when the video is vertical. In the case of normal 4:3 video, the best bet is to leave the black bars (think of Grand Budapest Hotel), or put a graphic overlay on the sides to cover the empty space.
For vertical video (please don’t shoot vertical video if you can avoid it), the standard method is to duplicate the layer, put a blur on the “bottom” layer, then scale it up to fill the empty space on the sides.
Letterboxing: This is the term for black bars on the top and bottom of a video. This usually happens when the footage has a larger aspect ratio than the sequence. Widescreen, 2:35:1, 2.40:1, and 21:9 are all going to have black bars when put into our baseline 16:9 sequence. Sometimes it’s not really that noticeable, and is often intentional (as shown by the video I posted above), but other times, it can cause the footage to be scaled too far. Your only option really is to scale it up and cut off the side pixels, or conform it from the start. Either way, you’re losing pixels, so be prepared to adjust accordingly.
Resolution
When working with different resolutions, your main caution should be going from lower resolution into a higher resolution project. A 1080p clip in a 4K sequence is much smaller, and an SD clip in a 4K sequence is even smaller. Trying to scale those up can have disastrous results, so you may want to try converting them beforehand.
If you’re doing the opposite, however, then you won’t have much to worry about with losing video quality. Just scale it to fit.
Codecs and Containers
This doesn’t really fit with aspect ratios or frame rates, but another couple of things to consider are the codec (h264, ProRes, etc.) and container (mp4, mov, m4v) of your files. Mixing up files with different codecs and containers can use a lot of your computer’s horsepower, and some programs can’t even use multiple types. The best thing to do here is save often and be on the lookout for any workflow issues. Most of the time you shouldn’t have much trouble, but it’s always best to save often and, if things get crazy with multiple codecs, just conform them and re-link them to minimize the risk.
Exporting
When it’s all said and done and your video is ready to go, the final thing to do is export it. With all these file types and frame rates and aspect ratios together in one place, you also still have another chance to make final adjustments here.
If your entire sequence has been edited in 4K but you need to deliver in 1080, you can do that here without having to conform your media. If you need the whole thing to be in 24 fps but all the footage is different frame rates, it will change that in the export. The export function allows you to set all your parameters for the final output, so at the very least, it’s the simplest step to making everything match in your sequence.
There may still be some clips that look funny, but take note of them and see why they’re not working. You may have to convert them prior to exporting.
Have questions about any of this information or other advice to share? Tell us in the comments!
Top image: Still from Video Editing Software Going Through the Timeline Frame by Frame by Kinomasterskaya