High-End Video Transcoder Shootout: Elemental Vs. Telestream
To be fair, both systems were designed for speed; for example, the Lightspeed Server system scaled the source files once to each target resolution, and produced all versions of that resolution from the single scaled file. Both systems produced the MP4 files first, and then transmuxed the MP4 files to the HLS output, which is much faster and much less CPU/GPU-dependent than encoding the HLS output from scratch. Any way you slice it, however, the results are pretty impressive.
Note that late in the review process, Elemental Technologies found a technique it thought might improve performance by up to 30 percent without impacting quality. We will update the results on our website if there are any changes.
Quality Overview
To assess quality, I ran three tests using two test files. The first two tests used the 360p_low and 720p_low presets shown in Table 1, which were among the most aggressive in the adaptive group as measured by bits-per-pixel-value. I also encoded one 720p test file to 800Kbps, more for vestigial purposes, since this is the file and test parameters I’ve used for years.
Before making any file comparisons, I verified that all test files were within 5 percent of the target data rate. I also double-checked each file to verify that I applied the correct encoding parameters and gave the vendors access to the finished files, so they could perform their own checks. Then, for subjective comparisons, I loaded the files into a Premiere Pro sequence that allows me to compare the files frame by frame. I also played each file back in real time to identify any motion-related compression artifacts.
In addition to these golden-eye type comparisons, for this review, I added three objective quality metrics, peak signal-to-noise ratio (PSNR), the Structured Similarity Index (SSIM), and Video Quality Matrix (VQM). By way of background, all objective quality metrics compare the compressed and original frames and compute a score based on differences in the compressed frame.
In the past, I’ve resisted using objective quality metrics, because PSNR, and to a lesser degree, SSIM, didn’t produce a score that correlated well with how real eyes viewed the image. For example, PSNR measures all differences between the compressed and original frame, even those too minor for the eye to see. If there were many instances when the color of one pixel varied from the original by one bit of the color value—say from one value of red to the adjacent value of red—the eye wouldn’t perceive it, but the PSNR score might be dismal.
In contrast, VQM was designed to detect and measure common motion and still image artifacts such as blurriness, erratic motion, noise, blockiness, and color distortion. In his paper “Survey of Objective Video Quality Measurements,” EMC Corp. employee Yubing Wang found a 95 percent correlation between the VQM score and subjective quality observations, meaning that VQM predicted subjective scores 95 percent of the time, which is exceptionally high. While I supplemented these scores with good old-fashioned eyeball tests, VQM provides a nice objective metric that’s very fast and easy to calculate.
In this regard, I used the Moscow State University Video Quality Measurement Tool (VQMT) to produce the objective scores, and compare the quality of files produced by our two encoders. Table 3 shows the results of the objective tests. For the record, smaller numbers are better for the VQM test, while larger numbers are better for the SSIM and PSNR.
Table 3. Smaller numbers are better for the VQM, while larger is better for SSIM/PSNR.
The top two groupings are the most relevant, since these were actual files produced in our adaptive group. As you can see (drum roll please), the numbers are very, very close, and subjective golden-eye comparisons similarly showed very little quality difference.
In the third test, encoded at a very aggressive bits-per-pixel value of 0.029, Telestream pulled ahead with a substantially better VQM score. Subjective comparisons verified a slight advantage for Telestream, with slightly more detail in many frames, particularly high-motion frames. Interestingly, the difference was so slight that you could seldom see it in side-by-side comparisons. However, when I loaded the Telestream video directly on top of the Elemental video at full resolution on the same Premiere Pro timeline, and switched Telestream video on and off, there were noticeable differences in many frames.
That said, noticeable does not mean significant; while Telestream showed a bit more detail, it wasn’t enough to turn ugly video into good-looking video. Figure 2 is an example; while Telestream shows slightly more detail in the face, it’s not something any viewer would notice without side-by-side comparisons, which, of course, they never have.
Figure 2. Telestream shows slightly more detail in the face, but nothing a viewer would notice without side-by-side comparisons.
In my view, Telestream’s most significant quality-related advantage is that Lightspeed Server uses the x264 codec and the system provides access to the codec’s presets and tuning parameters. Telestream used the medium preset for these tests, but if you need to optimize quality at the sacrifice of encoding speed, you can ratchet up the preset all the way to placebo, as well as select tuning parameters for film and other content.
Elemental takes the “simpler is better” approach and doesn’t provide access to the equivalent configuration parameters. For the vast majority of users, this will prove acceptable if not preferable, since if you don’t know what you’re doing, you could produce files incompatible with your target platforms. Still, if your goal is to produce the absolute best quality at the smallest possible file size, Telestream has an advantage.
In other tests, I analyzed the files produced for the quality tests in Bitrate Viewer to see how closely each encoder adhered to the 125 percent constrained VBR target. Figure 3 shows the results for the 600Kbps file, which should have a peak of 750Kbps. As you can see, while Elemental had much more variation in the data rate, the peak data rate was 932Kbps, about 180Kbps over the target. In contrast, Telestream peaked at 1614Kbps. While transient, this might affect playback compatibility, particularly for mobile viewers connecting over fixed bandwidth connections. This was the only test file that showed a significant difference between the files; the other two were very close.
Figure 3. Telestream shows a much higher peak data rate, more than twice the 750Kbps target.
I did check both the 600Kbps and 2900Kbps files for HLS-compliance in Apple’s Media Stream Validator tool, and they passed with flying colors. As shown in Table 4, Elemental’s files had slightly less overhead than Telestream, indicating a more efficient MPEG-2 transport stream, though the minor difference doesn’t translate to a real advantage.
Table 4. HLS overhead as reported by Apple’s Media Stream Validator tool
Overall, while a reviewer always prefers that his extensive tests reveal some profound differences between the products, this review is more like a dual physical exam that found both patients extremely healthy. Both Elemental Server and Telestream’s Lightspeed Server are exceptionally fast systems that produce nearly identical quality in the most common use cases. If you’re looking for the absolute fastest encoding time, Elemental has a slight advantage; if absolute quality at a low bitrate is your thing, Vantage has a slight edge. Otherwise, for most buyers, factors other than performance and quality will likely be decisive.
This article appears in the November/December 2014 issue of Streaming Media magazine as “High-End Transcoder Shootout.”
Related Articles
With VOD Producer DAI, content owners can automatically create DAI-complaint files within their existing cable infrastructure.
03 Apr 2015
At NAB, Elemental will show a bandwidth-saving solution that lets operators simplify creating multi-codec multi-delivery video.
25 Mar 2015
When an encoding job is highly demanding, it takes powerful gear to create and judge all the files. Here are the right tools for the job.
05 Dec 2014
For broadcasters and high-volume producers, Amazon's Elastic Transcoder has too many limitations. For everyone else, it's an appealing, if flawed, solution.
25 Jun 2014
The newest version of this popular live video production tool offers significant improvements and is a no-brainer upgrade for existing customers.
15 Jan 2014
Elemental is a strong choice for high-volume video publishers or those already using Elemental hardware, but it doesn't come at a bargain price.
11 Oct 2013