Guide to Great Web Video: Preprocessing Video

The Why, How and When’s of De-interlacing

Interlaced video is designed for television viewing, where each frame is displayed as two consecutive fields, each containing half the lines of the frame in an alternating fashion. If you were to look at a video signal over time, as shown in Figure 2, you would see the offset of lines in each field. Computer monitors display progressively scanned video, meaning all lines from one frame are displayed in order, also illustrated in Figure 2. When interlaced video is shown on a progressive scan monitor, the fields are pushed together, without regard to the motion that might have occurred between them. This results in a "finger" effect on moving edges, also shown in Figure 2. These fingers are unnecessary detail, which codecs will attempt to encode, using precious bits that would be better spent on the image details, textures and shadows of the moving objects.

Figure 2

If you capture video at 320x240, then you have automatically deinterlaced it. The capture drops every other line, or one whole field, if you look again at Figure 2. The major drawback to this is that details that are only one line wide can be lost. If you do want to archive your content for future generations of codecs, it’s always best to capture full screen video.

With full screen sources it’s important to be conscious of the interlaced artifacts, and you can use the operations built into applications like VirtualDub to eliminate them. As an example, our FASH.AVI source is an interlaced DV source captured at 720x480. For our initial codec tests, we chose not to deinterlace at all, but simply applied a Resize 2:1 (High Quality) filter. The resize filter sees moving diagonals, which are offset in time, as jagged lines, which are blurred during the filter operation. This will be most noticeable on diagonals that are less than 15o in slope, as highlighted in Figure 3a. In VirtualDub you can configure the DeInterlace Filter with the options to blend fields together, or duplicate one field and drop the other. When you blend the fields prior to resizing, the blurring effect is softer, resulting in fewer remnants of the jagged diagonals being fed to the codec. If you duplicate one field, the jagged edge is accentuated and the resize filter cannot eliminate it at all.

Figure 3a

Since VirtualDub is a kind of do-it-yourself type application, it’s important to note that the order of these operations is important as well. If you de-interlace with the blend operation after resizing, you are taking an already blurred image and blurring it more vertically, resulting in the image shown in Figure 3d. If you duplicate a field in the resized frames, you simply create more jagged edges that were somewhat removed by the resize filter itself, as shown in Figure 3e. Both of these options introduce even more artifacts to the moving details of the video, and thus would tax the video codec in terms of required bit rate. Other encoding applications such as Adobe Premiere or Discreet's Cleaner 5, will set this order for you automatically.

Figure 3d

Figure 3e

Another note that may confuse even the proficient compressionist is that VirtualDub advises blending two fields for the best quality. We’ve shown here an example to illustrate this, but why does Cleaner 5 recommend dropping a field instead? The answer lies in the way they are carried out in the application. If you chose the drop field option in VirtualDub, it actually removes those lines from the video, so they are not available to the next filter or processing step. So, our 720x480 image would become 720x240 in this case. Now apply a 2:1 resize, which operates in both horizontal and vertical directions, and you end up with a 360x120 image, distorted greatly from the original source. Cleaner 5, as well as other processing applications, knows it will drop the fields, but applies the filters together to achieve the desired results.

Frame Rate Conversions and Inverse Telecine

Reducing the frame rate is a prime method for lowering the bit rate requirements of your video encodes. The simplest way to do this is by dropping frames, so we always chose an output frame rate that is an integer divisor of the original. If you are starting with 30 frame per second (fps) video (or 29.97 in the broadcast world), you will be looking at 15, 10 or 7.5 fps for your output. In VirtualDub it is accomplished through the Video->Frame Rate dialog box shown in Figure 4.

Figure 4

The first function in the frame rate control window is Frame Rate conversion. Although it sounds like what you need, this will not accomplish the drop frame operation. Instead, it changes the way the frames are played back, so if you entered 15 in the "Change frame rate to" edit box, and you’re source was 30 fps, then the movie would play back at half speed, for twice as long as the original. In other words, all the frames are still there, they just play back slower.Keep the radio button on "No change" in this section.

The mathematical term that VirtualDub uses for the frame dropping operation is decimation, in the second section of the window shown in Figure 4. Decimate by 2 takes a 30 fps movie to 15 fps and Decimate by 3 takes it to 10 fps. Applying these to a PAL movie at 25 fps yields 12.5 and 8.33 fps, respectively. Don’t worry about the fractional frame rate, you are simply asking the application to drop every other frame, or every 2 out of 3 frames. You can also enter a number in the Decimate by edit box, and so can get to 7.5 fps from 30 using a factor of 4.

Tip: When encoding streams at frame rates that are lower than the source frame rate in VirtualDub, you need to compensate for the applications interpretation of bits/frame. If you decimate by 2, double the target bit rate. Decimate by 3 and you need to triple it, etc. Note that this is an imprecise science that will only get you in the ballpark for your target.

The last section of the frame rate control window is Inverse Telecine. We didn’t have a chance to test these options, but they are worth mentioning here. If you are not working with sources that originated from film, you should use the default setting "None (progressive)."

If your film source was converted to video, then its original frame rate of 24 fps was increased to 30 fps by repeating fields in a 3:2 pattern. The field repeat is simply extraneous video information that should not be fed to your codec. VirtualDub offers an automatic field-repeat detection scheme that can reset itself if scene changes occurred to disrupt the regular 3:2 pattern of the telecine process. You can also chose a manual scheme, whereby the algorithm simply assumes a regular pattern across the entire sequence. This might be useful for converting a segment of your video content without affecting the portion of the video that did not originate as film.

Tip: The help file in VirtualDub states that inverse telecine is not compatible with its frame decimation process. You will need to save a new source that has undergone this operation, resulting in a full resolution video at 24 fps. Frame decimation can then be carried out at integer divisors of this rate, such as 12 fps (decimate by 2) or 8 fps (decimate by 3).

Page 2: Size Matters >>