Beyond Standards: Moving Past MPEG to Create a Video-Friendly Web
Codecs, caching, and transport are three of the major building blocks required for video compression and delivery.
Each has a unique role. Codecs are at the core of compression and decompression. They’re used to shrink the size of a video-on-demand (VOD) file or lower the bandwidth required to deliver a live stream across the web while still maintaining visual quality. The last two building blocks—transport and caching, the act of storing permanent or semipermanent content closer to the end user— have historically been considered outside the expertise of the streaming industry and more in line with routing and internet architectures.
But that is changing, thanks in no small part to the “Netflix-ification” of the web, as over-the-top (OTT) content continues to rise in popularity and puts stress on both the infrastructure and the delivery methods that the streaming industry has relied on for decades.
Alongside this need to rebuild the web to be more video-centric comes a parallel desire, at least by some— whether they need a working solution now or think they can capitalize on that very need—to escape the confines of standards-based video.
From streaming media’s early days—when codecs such as MPEG-1 faced off with Intel’s Indeo, Microsoft’s Windows Media, and RealNetworks’ RealVideo— there has always been an underlying tug-of-war between standards and advances, between open source and licensing, between one-size-fits-all and customization on a per-use basis.
The flag in the middle of the tug-of-war rope, dangling precariously over the mud pit that awaits the losers, is innovation. If the standards bodies win, with their decision-by-committee mentality that slows or even stops the pace of innovation, then the industry faces periods of time when inefficient technologies fail to keep pace with video consumption growth.
This article will explore some of the tensions around that tug-of-war scenario as well as a few of the players that stand to gain from the move away from standards.
The Culprit: VOD
Consumption of VOD content, by far the vast majority of media delivered online today via online video platforms (OVPs) and OTT service providers, has risen at such a rapid pace that new technologies are desperately needed to keep up with the growth.
For VOD content, the codec and its encapsulating encoder make up a key battleground. Encoders are used to compress files either faster or slower than real time, depending on the quality desired for the final encoded video and the need for bandwidth savings.
Standards-based video compression often comes from two groups: the International Telecommunication Union (ITU) and the Motion Picture Experts Group (MPEG).
The ITU is a 150-year-old organization based in Geneva that started in 1865 as an arbiter of telegraph standards. As a part of the United Nations, the ITU is tasked with making sure that telecommunications—from radio spectrum to telephone switches to more recent internet standards—interoperate with one another across the globe. The ITU has a Video Coding Experts Group (VCEG) that ratified the ITU-T H.264 standard for use in telecommunications, namely in videoconferencing.
MPEG hasn’t been around for 150 years, but it has an outsized influence due to the fact that motion pictures are now being shot and delivered in a digital format. These movies are considered premium content, due to Netflix’s and other OTT providers’ desire to charge subscriptions for premium content.
MPEG took the H.264 standard, reworked it slightly to emphasize its use for high-quality VOD content, and named it MPEG-4 Part 10. At some point, the ITU and MPEG worked together to form a joint working group—now called the Joint Collaborative Team on Video Coding (JCT-VC)—and pushed the Advanced Video Coding (AVC) name as part of the joint standard. That’s why we often see terminology such as AVC/H.264 when referring to the same compression-decompression technology.
The differentiator between encoder companies that adhere to video standards such as AVC is centered on the quality of the encoding solution, since the decoders themselves are at least theoretically implemented the same way on a universal scale. Encoders can encode in either constant bitrate (CBR) or variable bitrate (VBR). Common wisdom states that CBR content can be encoded faster (normally a single-pass encoding) while VBR requires multiple passes across the content to yield higher-quality video files.
Some innovations, though, turn that on its head. For example, EuclidIQ initially took an approach to its perceptual quality optimization that ignored standards-based encoding— along with the “macroblock training wheels” that Richard Wingard, the company’s co-founder and CEO, was fond of dismissing—in a quest to get significant bandwidth reductions while maintaining visual quality. (Full disclosure: I worked for EuclidIQ from October 2015 until May 2016.)
The problem with this approach is that customers must download a special decoder. This problem faced older innovators such as On2 (with its VP7/8 codec, later sold to Google), and it still affects current industry innovators such as V-Nova (with its PERSEUS codec). As Jan Ozer writes in his overview of V-Nova’s approach, the need to have a specialized decoder means that use cases are limited to closed environments. This is true for set-top boxes, appliances with the specialized decoder flashed to silicon, or even end users who are forced to download a decoder to watch a single video enhanced with an innovated codec.
EuclidIQ faced that issue with EuclidVision. It ultimately moved back toward the macroblock and H.264 to offer its standards-compliant IQ264 perceptual quality optimization as a plug-in for a leading open source H.264 encoder.
Licensing: One Company, Two Patent Pools, or None at All?
Beyond the technical issues, though, a bigger issue seems to arise every time the committees innovate: licensing.
Streaming Media has covered the labyrinthine licensing issues over the years, beginning with the H.264 licensing pool started by MPEG-LA as a way to monetize the innovation of AVC for patent pool holders.
Innovation seems to flourish during times when the industry is subjected to licensing uncertainties around encoders, transcoders, and key delivery tools. Maybe “flourish” is too hopeful a word, but innovation seems to be brought to the forefront as a unique selling proposition during these times.
As an example, let’s ignore On2 Technologies’ history during the early H.264 licensing snafu days and consider the more recent uncertainty around the HEVC/H.265 codec. The licensing issues surrounding this joint MPEG-ITU standard have driven even major corporations to take desperate—or opportunistic, if you’re so inclined—steps to fill the void.
A blog post from Cisco’s Jonathan Rosenberg, championing the idea of a royalty-free codec, sums up the issue nicely. The 2015 post, “World, Meet Thor—a Project to Hammer Out a Royalty Free Video Codec,” first notes that Cisco has worked to make H.264 the foundation of its real-time communications tools, such as WebEx.
Rosenberg then states the case for patent licensing uncertainty for H.264’s successor, known interchangeably as H.265 and HEVC. “The next generation codecs are just beginning to emerge,” he writes. “There are two of note—Google’s proprietary VP9 codec, and the industry standard H.265 (HEVC) codec, which is the successor to H.264 (AVC).”
H.264 had one licensing pool, Rosenberg notes, but the HEVC licensing scenario involves two competing license pools. “Two distinct patent licensing pools have formed so far, and many license holders are not represented in either,” he writes. “The total costs to license H.265 from these two pools is up to sixteen times more expensive than H.264, per unit. H.264 had an upper bound on yearly licensing costs, whereas H.265 has no such upper limit.”
The end result, according to Rosenberg, is a limitation on using “H.265 in any kind of open source or freely distributed software application, such as web browsers. They also preclude its usage in freemium products—like WebEx or Cisco Spark—which have versions that users can use for free.” He explains, “[H.265] is not something that can serve as a universal video codec across hardware and software. Thus, we believe the industry needs a high quality, next-generation codec that can be used everywhere.”
In other words, if you don’t like the terms of a committee of large companies, you can use a codec created by another large company in its place.
According to Rosenberg, Cisco contributed Thor “as an input to the Internet Engineering Task Force (IETF), which has begun a standards activity to develop a next-gen royalty free video codec” in late 2015. He also notes that Mozilla is working to create a similar royalty-free codec called Daala.
Related Articles
OTT companies innovate to differentiate their products from their competitors' products, but for the industry to reach its potential, they'll have to cooperate.
02 Oct 2017
Enough with the codec confusion. The online video space won't really thrive until industry standards are put in place and enforced by the government.
08 Jun 2016
The industry is turning away from plug-ins and embracing HTML5 everywhere. Here's how the vendor-independent streaming standard is gaining momentum.
29 Mar 2016
Just when Media Source Extensions and Encrypted Media Extensions are making HTML5-based video playback a reality, DASH royalties threaten to derail it.
27 Jul 2015
Companies and Suppliers Mentioned