HEVC: What it Is, What You Need to Know

HEVC is the next-gen compression technology lauded as the enabler for a host of new services and capabilities. I discuss the historical development of HEVC and its technical underpinnings in my article "What Is HEVC (H.265)?" In this article, I'll briefly review those findings and focus on how quickly HEVC will make a mark in streaming media and OTT markets.

What We Know About HEVC

According to multiple studies, HEVC should deliver up to 50% better compression than H.264 in video on demand (VOD) applications, which means similar quality at half the bitrate. In live encoding, which obviously has to be performed in real time, the initial expectations are a 35% bandwidth reduction at similar quality levels. Alternatively, HEVC can also enable larger resolution movies, whether 2K or 4K.

Essentially, these are the two benefits of HEVC in the streaming space. The first relates to encoding existing SD and HD content with HEVC rather than H.264, enabling cost savings and/or the ability to stream higher quality video to lower bitrate connections. The second relates to opening up new markets for ultra-high-definition (UHD) videos.

On the playback side, there are multiple data points, but no real clear picture. Several companies have announced software decoders, but it's unclear how much horsepower is necessary to drive them. The original targets for HEVC were 10x encoding complexity and 2x-3x decoding complexity as compared to H.264, and most sources have confirmed the 10x figure for encoding. Perusing various comments from various sources, decoding complexity has ranged from "same as H.264" to the 2x-3x figure.

Remember, however, that H.264 playback is accelerated in hardware on most playback platforms, including GPU-accelerated playback on computers. According to my source at graphics vendor NVIDIA, "There currently isn't any dedicated [hardware] support for HEVC in our current GPUs. I'm pretty limited to what I can say about future products. But I'll just say that our goal is for our GPUs to support all current video standards. Now, that said, it is quite possible for third parties to write HEVC encoders and decoders using CUDA to use the processing capability of current GPUs."

Though GPU acceleration may be coming, it isn't here yet, so H.264 and HEVC aren't on a level playing field when it comes to accessible playback hardware. Still, according to a source at Qualcomm, Inc., "[W]e are able to get 1080p, 30fps HEVC Main profile video with just a little bit over 50% CPU utilization on a quad-core architecture."

On the mobile side, my source reported, "At CES we showcased Ittiam's ARM based decoder and played back 1080p HEVC and 1080p H.264 videos side by side on recently announced Snapdragon 800 platform. Ittiam's decoder is still in development and will get further enhancement." You can see a video of HEVC playing back on the Qualcomm Snapdragon platform from the Mobile World Congress 2012.

According to a report titled "HEVC Decoding in Consumer Devices," senior analyst Michelle Abraham from Multimedia Research Group, Inc. estimated that the number of consumer devices that shipped in 2011 and 2012 that would be capable of HEVC playback with a software upgrade totaled around 1.4 billion, with more than a billion more expected to be sold in 2013. According to Abraham, in compiling these statistics she assumed that all PCs shipped in each year would be HEVC-capable.

Encoding with HEVC

I spoke with several encoding companies; many were bullish on HEVC and have either made HEVC-related product announcements (Elemental Technologies, Inc.) or will at NAB. Another made the very cogent comment that the encoding side was always ahead of the game and that the path to actual producer adoption is widespread playback availability.

Speaking of playback, none of the major players -- Adobe, Apple, Google, or Microsoft -- have announced HEVC playback support in their respective players, browsers, or mobile or desktop operating systems. One reason why -- and a potential monkey wrench in at least the short-term HEVC adoption cycle -- is that no one knows what it will cost to use HEVC.

HEVC Royalty Issues

What's clear at this point is that multiple companies have patents relating to HEVC technology, and they plan to ask for royalties from those who use their technology. This was the case with H.264 as well, and though many in the streaming industry grumbled about the royalties, this disgruntlement certainly didn't limit H.264's success.

Two things are different with HEVC. First, where H.264 involved a single group of patent holders administered by MPEG LA, it appears that some HEVC patent holders want to pursue royalties outside of a patent group, which will make it more challenging for HEVC users to license the technologies. According to "Patent Snafus Could Delay New Video Codec," Mediatek and Qualcomm do not want to join the HEVC group formulated by MPEG LA, and Samsung hasn't decided either way.

One chipmaker executive, speaking anonymously for the EE Times article, commented, "HEVC has so many patent holders and some of them say they will not be part of the pool but want to collect royalties themselves. If say 20 people all want to collect royalties it will kill the standard -- we need a fixed cost, it cannot be variable," he added.

Beyond this uncertainty, HEVC is coming to the streaming media market much faster than H.264, where royalty policies were in place well before any significant market adaption. To recount, the H.264 spec was approved in March 2003, and MPEG LA announced licensing terms in November 2003. Obviously, when Apple announced support for H.264 in QuickTime 7 in April 2005, royalty policies were firmly in place. Ditto for when Adobe announced that it would include H.264 in Flash in March 2008, and when Microsoft added H.264 to Silverlight in July 2009.

Our contact at MPEG LA reported that while the HEVC group had met three times as of February 2013, there was still no guarantee that a group would be formed or that all patent holders would join the group. So it appears that HEVC early adopters will have to decide to implement the technology without knowing the cost.

For large companies such as Adobe, Apple, Google, and Microsoft, that might be tenable; the H.264 license was capped, and it's reasonable to assume that the HEVC license will also be capped. All four companies can amortize that cost over millions of product units shipped, and I think it's highly likely that one or more of these companies will announce HEVC integration by NAB.

Even the encoding companies that I spoke with commented that they might incorporate HEVC technologies into their encoding tools without knowing the cost, because, as one exec said, "[S]upporting new formats is the race that we run." The exec also noted, however, that this was the first time that they were ever forced to consider embracing a codec without having an idea about the licensing structure.

However, let's get back to the two potential benefits that actual publishers seek from HEVC: cost savings and opening up new products and services. In both cases, it seems unlikely that any producer would use HEVC-encoded video without a known cost structure. Sure, H.264 usage for free internet video is free, but that decision was made under a completely different set of circumstances, and it's doubtful if HEVC usage will be similarly unencumbered.

How H.264 Became Free

A short history lesson will explain why H.264 became free. When the terms of the initial H.264 license were announced, there was a royalty on H.264-encoded video deployed in a pay-per-view or subscription operation. The royalty was not on free internet video, at least through the initial term of the license, which ended Dec. 31, 2010.

The licensing terms attributed this waiver to the fact that the internet streaming market was "still developing," though this is likely disingenuous. The fact of the matter was that the H.264 implementations of that time offered only a slight quality improvement over VP6, the predominant Flash codec, and required more CPU horsepower for playback. There was also no mobile platform such as iOS or Android that wasn't compatible with VP6 that could force producers to use H.264. So 99% of producers were satisfied with VP6 and wouldn't have experimented with H.264 if there was a royalty involved.

In February 2010, MPEG LA extended the royalty moratorium for free internet video through December 2016. In an interview with Streaming Media, MPEG LA president and CEO Larry Horn attributed this decision to the fact that "[t]hough some companies are doing well with advertising supported video, overall the models are still in flux, and the patent group didn't want to plug a royalty into a business model that's still unsettled."

In May 2010, Google announced WebM, an open source alternative to H.264 that offered very similar quality and playback performance. In August 2010, MPEG LA announced that there would be no royalties on free internet usage of H.264 in perpetuity. Though MPEG LA never publicly admitted that the availability of a free, open source solution contributed to this decision, the timing would suggest it did.

Why HEVC Probably Won't Be Free

Fast-forward to 2013. HEVC is ready, and at this point, it has no real competition. Sure, you can point to VP9, but considering how poorly Google executed taking VP8/WebM to market, it's unlikely that any producers -- or patent groups -- will take it seriously. In 2012, online video advertising jumped to $2.9 billion, so it's tough to say that this market is still in flux.

For these reasons, it seems unlikely that if HEVC enables large publishers to cut their bandwidth costs by 50%, HEVC patent holders wouldn't want their share. It seems equally unlikely that publishers seeking to reduce bandwidth costs via HEVC would start using the technology until the cost structure was known. So HEVC implementation seeking to harvest this benefit is likely on hold pending the announcement.

Now let's look at the potential for HEVC to jump-start new markets for UHD video.

HEVC and UHD Video

Again, I'm focusing my analysis on the streaming and OTT markets, since those are the ones I know best. Even without considering the royalty cost uncertainties, it seems unlikely that HEVC will spawn many new UHD-related products and services over the next 2 or 3 years for three reasons: bandwidth, the lack of 4K displays, and the lack of content.

Let's start with bandwidth. In his blog post "H.265/HEVC Ratification and 4K Video Streaming," compressionist Alex Zambelli, previously with Microsoft and now with iStreamPlanet, estimated that if HEVC produced a 40% bandwidth saving over H.264, a 4K movie would require bitrates of between 12Mbps and 15Mbps. Assuming that HEVC actually produced the 50% target, these numbers would drop to 10Mbps and 12Mbps.

According to the latest Akamai Technologies "State of the Internet" report, for 3Q 2012, the average connection speed in the U.S. was about 7.2Mbps, up from about 6Mbps the previous year. While users connecting on some premium services could handle HEVC's 10Mbps-12Mbps in the short term, it's unclear when a significant portion of the U.S. population will be able to support 4K HEVC movies.

Though the U.S. is far from the performance leader, only South Korea, with an average connection speed of 14.7Mbps, could sustain a 4K movie today, with Japan next at 10.5Mbps. Again, while dedicated satellite or cable networks could certainly carry this load, it's unlikely that the type of shared internet connection used for OTT would be able to in the short term.

Download and Watch

One market that seems potentially ripe for larger-than-1080p viewing is the traditional download to view, particularly given the multiple viewing options. For example, TimeScapes is a film by Tom Lowe featuring slow-motion and time-lapse cinematography of the landscape, people, and wildlife of the American Southwest. Shot in 4K, the movie is available in 10 versions, ranging from SD DVD and 1080p Blu-ray to 4K, including custom versions for 30" 2560x1440 displays and the MacBook Retina display (2880x1620). It's an innovative strategy that could portend the optimal strategy for UHD movie distribution (see Figure 1).

Figure 1: This 4K movie is available in 10 different versions.

I asked Lowe about how his sales were distributed among the available offerings. He responded, "Actually the ‘30 inch' 2560x1440 version is selling like hotcakes. Sales have exceeded my expectations, in terms of percentage sold vs Blu-ray, 1080p download, etc. I would say for every one [in] ten 1080p HD downloads we sell, we sell about four 2560 copies. So many people have the Dell and Apple 2560 displays, but have never, ever had any video to play at that resolution."

Interestingly, watching video on tablets and computers seems like the optimal use of 4K screens, as viewers are actually close enough to the device to see the difference. Though it will likely never see the light of day, Panasonic showed an 18.7 x 13.1 tablet computer at CES with a 4K screen -- could true 4K tablets be far behind? And once they're available, wouldn't 4K movie downloads from iTunes seem like the natural next step? Once HEVC playback becomes available, it would cut download times and storage space by 50%.

How likely is true 4K viewing on computers? I asked Lowe for his thoughts about computers and tablets as a potential viewing platform. He replied, "I have always believed that 4K monitors are where 4K will really catch on. On a projector, or large TV, you don't notice it as much. On a monitor only a couple feet away, the difference between 1080p and 4K is very striking. I think 4K monitors will catch on fast once they come out, among gamers, Photoshop enthusiasts, and people watching or making high-res video."

Conclusions

What's this add up to? For producers seeking to distribute SD and HD content encoded with HEVC, the lack of a known royalty structure is a major buzzkill. At this point, no one knows whether a single royalty structure will be in place or whether multiple IP properties will pursue royalties independently, or the timing of either of these efforts.

HEVC encoding should be generally available by the end of 2013, if not sooner, and the player-related picture should also be clearer. Though it's impossible to predict what Apple will do, I would be surprised if there wasn't an HEVC decoder for the iPad 3 and MacBook Retina line announced before the end of 2013. Ditto for Adobe announcing HEVC playback in Flash.

Those attempting to leverage HEVC to create new opportunities for the distribution of UHD video have the royalty mountain to climb, as well as a paucity of content and viewing platforms and the lack of bandwidth to deliver the streams. Given the data rates involved, it feels like downloading for viewing is a better short-term model, providing custom resolutions for specific resolution displays.

The increased efficiency that HEVC provides will ultimately make streaming video more affordable and deliverable. However, until the royalty picture clears, it's hard to get excited about any projected cost savings from HEVC. On the UHD side, HEVC is only one piece of the puzzle that must come together to make 4K viewing a reality, and not a very important piece of the puzzle at that.

This article appears in the April/May 2013 issue of Streaming Media magazine as "The Future of HEVC."

Jan Ozer's article first appeared on OnlineVideo.net