HEVC in HLS: How Does It Affect Device Performance?
If you're considering adding HEVC to your HTTP Live Streaming (HLS) videos, one concern has to be the CPU required to play the videos, as this affects both playback frame rate and battery life. To prepare for a Streaming Media West pre-conference session on producing HLS with HEVC (co-produced with David Hassoun and Jun Heider from Realeyes Media), we compared the CPU required for H.264 and HEVC playback on two phones and three computers, using files prepared and hosted for us by Bitmovin. It's not a lot of data, but we found it both interesting and useful.
To start, let's have a look at the compatibility matrix for iOS 11, which is what you'll need to playback HEVC and HLS on iOS devices like iPhones and iPads.
Figure 1. iOS 11 compatibility matrix from Apple
As Figure 1 shows, iOS 11 works on phones as old as the iPhone SE/5S, but the oldest phone we tested was an iPhone 6, the first with HEVC hardware decoding. Specifically, we used the System Status App to capture the processor utilization charts shown in Figure 2 for about two minutes of video playback. On the left is H.264, on the right HEVC.
As you can see, H.264 averaged about 50% utilization during H.264 playback, while HEVC pulled slightly over 60%. As you may remember, Apple added HEVC playback to the iPhone 6 for FaceTime, and it seems like it’s being used for HEVC playback with HLS as well. Still, based on this experiment, you would expect some minor decrease in playback time with HEVC as opposed to H.264, though that’s difficult to estimate based solely upon CPU utilization.
Figure 2. CPU utilization on the iPhone 6
If you’ve upgraded to the iPhone 7, however, you’ll find the CPU requirements very similar, as you can see in Figure 3. If you’ve upgraded to the iPhone 8 or X, which use more advanced processors than the iPhone 7, you’d expect little, if any, battery life hit due to HEVC.
Figure 3. CPU consumed by H.264 and HEVC on a iPhone 7
What About iPads?
How would this relate to current iPad models? Since we had no units to test, we can only speculate. Note that the iPhone 6 uses Apple A8 system on a chip (SoC), the same as the iPad mini 4, so you'd expect a similar increase in CPU utilization on the mini 4. The standard iPad uses the A9 SoC, which should be slightly more efficient than the A8, but we have no relevant data. The 10.5" and 12.9" iPad Pro uses the A10X, which is more advanced than the CPU in the iPhone 7, so both these devices should be at least as efficient as the iPhone 7, meaning little or no battery hit due to HEVC.
Regarding iPhones, the iPhone 5 used the Apple A6 SoC, two generations behind the A8 used in the iPhone 6, and has no HEVC acceleration. I would expect CPU utilization to be much higher than H.264, but you never know until you test (and if you have tested, please let us know in a comment). When you consider that the iPhone 5 shipped more than five years ago in September 2012, this model is likely not a concern for most streaming producers.
Overall, though our iPhone tests are limited, it appears that deploying HEVC within HLS will have minimal impact on most devices, and absolutely none on iPhone 7 and newer phones, and likely iPads using the same SoCs.
What About Computers?
2010 iMac and High Sierra
Figure 4 shows the compatibility matrix for High Sierra, which you’ll need to upgrade to for HEVC support in HLS.
Figure 4. You’ll need one of these computers to play HEVC in HLS.
On a 2010 iMac that barely snuck into the compatibility grid, CPU utilization was very consistent between HEVC and H.264, as you can see in Figure 5. This could be because this computer was equipped with an ATI Radeon HD 5670 graphics engine, though this card pre-dated HEVC so obviously doesn't support HEVC decoding. Of course, battery life doesn't matter on a device that only works when plugged in, but it was good to see that the video played at or close to full frame rate.
Figure 5. CPU load for the two formats on a 2010 iMac.
MacBook Pro and High Sierra
No pretty pictures for the MacBook Pro, where we had to rely up CPU-related results from Activity Monitor to distinguish the playback load between the two formats. Our mid-2014 notebook had a four-core Core i5 CPU with Intel Iris graphics. On this notebook, overall CPU utilization was around 44% for H.264 and 50% for HEVC, which should translate to some decrease in battery life, but probably not a dramatic one.
Looking at the computers shown in Figure 4, the oldest generation is a concern, since they're equipped with intel Core 2 Duo CPUs that lack the hyperthreading and other architectural advancements of the newer Intel Core i3 and i5 CPUs. However, in 2011, the MacBook and MacBook Air notebooks got a refresh to a Core i5 CPU with more than sufficient power for either H.264 or HEVC.
For example, the last computer we tested was a 2011 MacBook Air with a 1.6 GHz Intel Core i5, and you can see the CPU required for playback of the two formats in Figure 6. Again, the difference was around 7% of overall CPU, but it’s clear that even this six-year-old computer had plenty of power to spare.
Figure 6. Playback CPU on a 2011 MacBook Air with a 1.6 GHz Intel Core i5 CPU
Overall, while you may have issues with the oldest generation of supported mobile devices and computers, the next generation in all cases show only a slight increase in CPU utilization for HEVC playback, while new iPhones at least show relative parity. Publishers considering deploying HEVC should do so without concerns that the higher-end format will create significant battery life issues for most potential viewers.
Many who heard that Apple is adding support for HEVC playback in HTTP Live Streaming were left with more questions than answers. Here's what developers need to know.
Adding HEVC to your HLS streams is looking like a no brainer, but if you decide to do so, you may not want to take Apple's HEVC encoding recommendations verbatim. You'll deliver noticeably higher quality video if you follow the advice detailed below.
We're looking to find out how publishers are using HEVC with their HLS streams. Take our survey to help us find out—and get a copy of the report when it's published.
The latest macOS includes support for HEVC, but not for older Macs. In the rush to support higher resolutions and lower files sizes, are too many people being left out?
Hardware acceleration and field programmable gate arrays may be the answer to the rising costs of encoding for multiple codecs including H.264, H.265, VP9, and soon AV1
At its Worldwide Developer Conference this week, Apple announced it would support HEVC/H.265 in High Sierra and iOS 11 in a combination of hardware and software decoding, depending on the device. Here are the details of how Apple will implement it.
H.265 is coming to the Android operating system, as well, but it's nowhere to be seen on desktop or notebook computers.
Companies and Suppliers Mentioned