Codec Licensing and Web Video Streaming
The web video streaming market has experienced exponential growth, with market size predicted to increase by more than USD 300 billion in the coming years.[i],[ii] The rise of online video platforms has disrupted the traditional linear television industry, with many consumers opting for online video streaming services.
Yet participants in this market disruption have struggled to find a profitable business. Commercial content providers must not only compete with each other for attention, time, and revenues but also with social media, user-generated content, and video games.
Though superficially unchanged, web video streaming is a fundamentally new proposition. Because of this fact, companies whose codec standard essential patents (SEPs) have contributed to this web video revolution are faced with a multifaceted problem.
- Recent efforts to license SEPs have resulted in complex licensing schemes involving multiple patent pools. Can a codec licensing scheme like that employed for linear television and optical discs be made to work for streaming video?
- Past attempts to derive substantial codec licensing fees from video streaming content providers met with strong resistance and were subsequently abandoned. Is there a substantial role for content-based codec licensing, for example, pay-per-view or subscription video codec licensing?
To find an answer to these questions it is useful to examine the role of codecs in the shifting landscape of commercial web video streaming, and the history of content-based codec licensing.
The role of codecs in web video streaming
Commercial web video streaming is made possible by four fundamental, component standards.
Two components for both free Internet and commercial streaming clients are high-quality, low-bitrate audio and video encoding standards. The dominant audio and video codecs today are AAC and AVC, respectively.
- Introduced in 1999 by the ISO/IEC JTC 1 Moving Pictures Expert Group (MPEG), AAC (ISO/IEC 14496-3)[iii], also referred to as MPEG-4 Part 3 and MPEG-4 Audio, is a widely available high-quality, low-bitrate audio encoding standard.
- Introduced in 2003 by the ITU-T Video Coding Expert Group (VCEG) in collaboration with MPEG, AVC (ISO/IEC 14496-10)[iv], also referred to as H.264 or MPEG-4 Part 10 is a widely available high-quality, low-bitrate video encoding standard.
AAC and AVC are not sufficient to enable commercial web media streaming. Two additional components are required – a standard for interoperable digital rights management (DRM) and a standard for adaptive bitrate delivery packaging.
- Introduced in 2012 by MPEG, Common Encryption or CENC (ISO/IEC 23001-7)[v] is the DRM-interoperability standard supported by the principal digital rights management systems – including Microsoft PlayReady, Google Widevine and Apple FairPlay.
- Introduced in 2018 by MPEG, the Common Media Application Format or CMAF (ISO/IEC 23000-19, 2018)[vi] is the audio-video encapsulation standard supported by the two most commonly used adaptive bitrate streaming protocols – Dynamic Adaptive Streaming over HTTP or DASH (ISO/IEC 23009-1)[vii] and HTTP Live Streaming or HLS (IETF RFC 8216)[viii].
Although non-commercial video streaming is possible with AVC and AAC alone, the present-day exponential growth of the commercial web media industry would not have been possible without the global availability of CENC and CMAF.
Component Licensing
No license is required to stream or distribute content using the AAC audio encoding format. However, a license and payments are required for all manufacturers or developers of AAC decoders. The terms use per-unit pricing, and there are no royalties charged on content, subscribers, or attributed revenue.[ix]
The commercial use of AVC video codec technologies necessitates licenses from and payment to the VIA Licensing Alliance (formerly MPEG LA), the private company which administers the AVC patent pool for most (but not all) of the AVC patent holders.[x]
To ensure that AVC remained attractive and viable to free online streaming services such as YouTube, the patent holders committed to not charge any kind of content-based royalty for free services, while placing a cap on paid services, so, for example, a paid streaming service like Netflix would today pay no more than $100,000 a year for its AVC streaming license, independent of the number of serviced subscribers, resolution of the streamed video or the stream duration.
The companies who proposed CMAF and CENC did so mindful of the consumer value of a global web media client and the consumer-focused business opportunities such a client would present. Anticipating that a royalty bearing license would hamper web-centric business opportunities, they were insistent that CMAF and CENC be published license-free.
Though not license-free, the royalty model employed for AAC and AVC clearly hit the sweet spot for large scale adoption; but these codecs have their technical limitations. AVC, for example, is not especially suitable for high resolution or high framerate content.
Introduced by the VCEG in collaboration with MPEG, HEVC (ISO/IEC 23008-2)[xi], also referred to as H.265 or MPEG-H Part 2, is an advanced codec that can achieve up to 50% bitrate reduction compared to AVC while maintaining or improving the quality. It is especially suitable for high-resolution and high framerate content.
HEVC is without question a better video codec standard than AVC, yet its adoption has been painfully slow. The licensing model for a technology can have a profound impact on its adoption and the growth of the market. Just what went wrong?
The shifting landscape of web video streaming
From one perspective the story of the last two decades of codec development has been dominated by an effort to improve compression ratios. For example, video codec H.265/HEVC (2013) offers about 25%-50% better data compression when compared to H.264/AVC (2003) at the same level of video quality, or substantially improved video quality at the same bit rate.[xii]
But the improvement of compression ratios is only part of the web video streaming story. The effort to improve codec compression ratios took place while two revolutionary changes swept the Internet, changes that brought into question important, unexamined codec licensing assumptions.
This shifting landscape was the emergence of mobile and smart TV audio-video applications, and the availability of royalty-free codecs.
Mobile and Smart TV audio-video applications
Apple announced the iPhone in 2007[xiii] and the first Android phone went on the market in 2008.[xiv] That same year Smart TVs appeared in the market, allowing users to download and install audio-video applications on their TV directly from the Internet. By the time HEVC was released in 2013 there were more than 930 million mobile phones and 84 million smart TVs in the market, all supporting Web audio-video applications.[xv],[xvi]
With linear broadcast, support of a codec which is optional for the broadcaster is mandatory for the set top box, leading to stiff competition over which codecs will be required by a linear television standard to be supported by device manufacturers. On the other hand, modern web media applications support codec capability discovery, so the web streaming service can discover what codec the device supports and optionally provide content in that format, making the codec support decision market-driven for both the device manufacturer and the web streaming service.
Why does this matter? Market-driven codec selection combined with device codec capability discovery places downward pressure on codec SEP royalties when compared to what could be demanded for linear television devices.
Royalty-Free Codes
Google released the open and royalty-free codec VP8 in 2010 and VP9 in 2013, the year that HEVC was released. In 2018 the Alliance for Open Media released the AV1 codec, a standards-based royalty-free video codec.
Why does this matter? The availability of royalty-free codecs provides web streaming services an alternate codec should a codec SEP holder decide to charge subscription service content fees.
Content-based codec licensing
In 2014 MPEG LA (now VIA Licensing Alliance) announced that they would charge 20 cents per product to encode or decode video using HEVC, though the first 100,000 units would be free. This amounted to parity with the licensing of the HEVC predecessor AVC. MPEG LA also announced that, while reserving the right to do so, they would not charge royalties when a HEVC encoded video is streamed over the Internet or sold on a Blu-ray disc.[xvii]
Dissatisfied with the anticipated SEP licensing revenue from MPEG LA, some codec SEP holders moved to a new patent pool – HEVC Advance (now Access Advance), where they were promised higher revenue including revenue from content providers. Indeed, in 2015 HEVC Advance announced that it wanted 0.5 percent of revenue taken by any paid streaming video service delivered with HEVC.[xviii] But by 2018, under pressure from content providers, HEVC Advance withdrew all royalty fees for Internet streaming.[xix]
A third patent pool for HEVC was administered by Velos Media[xx], a company operated by the consulting firm Marconi. As of 2023, although they have their own HEVC patents, Velos Media no longer maintains an HEVC patent pool, having returned all third party patents to their respective owners.[xxi]
In 2021 Micky Minhas, senior vice president at Marconi, wrote an article arguing for a new approach for Codec licensing. He suggested that the new approach
"... needs to reflect how codecs are used now, rather than how they were used twenty years ago... It should license at multiple points in the video encoding, decoding and transcoding ecosystem that are realising value from video coding standards, including streaming and cloud-based services, and not just end user devices."[xxii]
To understand this approach, it is important to note that SEPs pertain to the normative language of a standard’s specification. Codec standards provide normative language about two things – the bitstream format itself and the decoder actions when processing that bitstream. By design the codec standards leave how an encoder produces the bitstream unspecified, allowing innovation to flourish on the encoding side, relying upon the bitstream and decoder specification to ensure interoperability.
Patent pools have managed to set a license requirement on encoders by reference to the bitstream specification, but this doesn’t work for a streaming service. Streaming services use formats like MPEG’s Dynamic Adaptive Streaming over HTTP (MPEG-DASH) or the Apple designed HTTP Live Streaming (IETF HLS). Neither of these specifications make use of normative language in a codec bitstream specification.
If the web video streaming service neither makes use of the codec specification nor indirectly infringes on a codec SEP, viz., does not enable or encourage a third-party to directly infringe on a codec SEP, how would the SEP holder justify requiring the web video streaming service pay a license fee, as Micky Minhas has proposed?
Participating in the web video revolution
Minhas suggested that the perfect licensing solution “… needs to reflect how codecs are used now, rather than how they were used twenty years ago” in my opinion, with this one comment, he is spot on.
Let’s contrast how codecs were used twenty years ago with how they are used today.
- Twenty years ago, linear services were closed, royalty bearing ecosystems; codec support which was optional for the broadcaster was mandatory for the device manufacturers; no royalty-free alternative codecs existed; and consumers would use a set top box supporting a single linear service application.
- Today the web itself is dominated by royalty-free standards, including royalty-free standards essential for commercial web video streaming; codec capability discovery makes codec support a market-driven decision; royalty-free video codecs are available and in use; and the consumer watches video on a device capable of supporting an unlimited number of applications.
All of these changes from how codecs were used twenty years ago have put downward pressure on codec SEP licensing revenue. I would argue that the era of MPEG-2 licensing for linear television, DVD and Blu-ray devices created an unrealistic SEP revenue expectation, resulting in today’s complex licensing schemes with multiple patent pools.
And since web video streaming does not refer to codec normative specification language, any attempt to charge streaming services a content-based royalty fee exposes a fundamental misunderstanding of what is involved in modern day web video streaming.
That said, one fundamental difference between linear and web media services suggests an enhanced revenue path for codec SEP holders, a path already embraced by modern web media companies.
When one service owns the device, as is the case for linear services, value can only be derived by or through that one service. In that case the codec SEP holder can only recoup their research and development costs by receiving license fees for their patents.
However, the modern web media device is a platform with a value chain “wide at the top”. The codec SEP holder has more creative ways to recoup the cost of developing standards essential patents – such as providing content and more importantly providing and/or supporting services which utilize features of their SEPs.
This fact about web services is one reason why the world wide web evolved into a royalty free ecosystem. It also represents the best way that codec SEP holders can more profitably participate in the web video revolution.
___
[i] Video Streaming Market - North America, Europe, EMEA, APAC : US, Canada, China, Germany, UK - Forecast 2023-2027, Technavio, December 2022, https://www.technavio.com/report/video-streaming-market-industry-analysis
[ii] Video Streaming Market - Global Industry Analysis, Size, Share, Growth, Trends, Regional Outlook, and Forecast 2022-2030, Precedence Research, https://www.precedenceresearch.com/video-streaming-market
[iii] ISO/IEC 14496-3:2019, Information technology — Coding of audio-visual objects — Part 3: Audio, https://www.iso.org/standard/76383.html
[iv] ISO/IEC 14496-10:2022, Information technology — Coding of audio-visual objects — Part 10: Advanced video coding, https://www.iso.org/standard/83529.html
[v] ISO/IEC 23001-7:2023, Information technology — MPEG systems technologies — Part 7: Common encryption in ISO base media file format files, https://www.iso.org/standard/84637.html
[vi] ISO/IEC 23000-19:2020, Information technology — Multimedia application format (MPEG-A) — Part 19: Common media application format (CMAF) for segmented media, https://www.iso.org/standard/79106.html
[vii] ISO/IEC 23009-1:2019, Information technology — Dynamic adaptive streaming over HTTP (DASH) — Part 1: Media presentation description and segment formats, https://www.iso.org/standard/75485.html
[viii] IETF RFC 8216, HTTP Live Streaming, August 2017, https://www.rfc-editor.org/rfc/rfc8216
[ix]AAC License Fees, VIA Licensing Alliance, https://www.via-la.com/licensing/aac/license-fees/
[x] AVC/H.264, VIA Licensing Alliance, https://www.via-la.com/licensing-2/avc-h-264/
[xi] ISO/IEC 23008-2:2013, Information technology — High efficiency coding and media delivery in heterogeneous environments — Part 2: High efficiency video coding, https://www.iso.org/standard/35424.html
[xii] High Efficiency Video Coding, https://en.wikipedia.org/wiki/
High_Efficiency_Video_Coding
[xiii] Apple Reinvents the Phone with iPhone, Apple press release, January 9, 2007, https://www.apple.com/newsroom/2007/01/09Apple-Reinvents-the-Phone-with-iPhone/
[xiv] Official: First Android Phone to Debut on September 23, Charlie Sorrel, September 17, 2008, https://www.wired.com/2008/09/official-first/
[xv] Smartphones – statistics & facts, https://www.statista.com/topics/840/smartphones/
[xvi] Smart and connected TVs – statistics & facts, https://www.statista.com/topics/4761/smart-and-connected-tvs/
[xvii] Patent holders cut streaming fees for next-gen video tech, CNET, Stephen Shankland, Sept 30, 2014, https://www.cnet.com/tech/services-and-software/patent-holders-cut-streaming-fees-for-next-gen-video-tech
[xviii] New fees cast shadow on next-gen high-quality streaming video, CNET, Stephen Shankland, July 22, 2015, https://www.cnet.com/tech/services-and-software/new-fees-cast-shadow-on-next-gen-high-quality-streaming-video/
[xix] HEVC Advance Eliminates Content Distribution Royalty Fees and Reduces Certain Royalty Rates and Caps, March 13, 2018, https://accessadvance.com/2018/03/13/hevc-advance-eliminates-content-distribution-royalty-fees-and-reduces-certain-royalty-rates-and-caps/
[xx] https://velosmedia.com/
[xxi] Velos ends its patent pool for HEVC, simplifying video codec licensing picture, Alex Davies, January 31, 2023, https://rethinkresearch.biz/articles/velos-ends-its-patent-pool-for-hevc-simplifying-video-codec-licensing-picture/
[xxii] IAM: It’s time for a new approach to codec licensing, Micky Minhas, September 18, 2021, https://marconi.com/iam-its-time-for-a-new-approach-to-codec-licensing/
Biographical Information
John C. Simmons, Principal Engineer at John Simmons Consulting, LLC. Established in 2020, streaming media clients have included Apple, Google, and Microsoft.
John worked at Microsoft for 16 years, where he led the effort to standardize DRM-interoperable encoding and adaptive bitrate streaming, to collaborate with Google, Comcast, Netflix and the W3C to define HTML5 Media Extensions, and to partner with Apple to develop the Common Media Application Format.
John’s standards evangelism across the media and entertainment industry led to the creation of the Consumer Technology Association (CTA) Web Application Video Ecosystem (WAVE) project, a 60-company collaboration to adopt ISO/IEC and W3C standards for streaming media on a global scale. John is chair of the CTA WAVE Content Specification Task Force.
John received the CTA 2017 Technology Leadership Award for the creation of the WAVE Project and the National Academy of Television Arts Sciences (NATAS) 2019 Emmy Award for the Microsoft, Google, Comcast, Netflix and W3C “Standardization of HTML5, Encrypted Media Extensions (EME) and Media Source Extensions (MSE) for a Full TV Experience”.
[Editor's note: This is a contributed article from John Simmons Consulting, LLC. Streaming Media accepts vendor bylines based solely on their value to our readers.]
Related Articles
Traditional, controlled broadcast environments are being replaced by dynamic, complex ecosystems where multiple applications and services must coexist harmoniously. The challenge is to maintain the quality of a provider's main service, like linear TV with dynamically inserted ads, while accommodating the diverse requirements for and potential interference from third-party applications. Yoann Hinard of Witbe discusses the challenges faced throughout this process, including remote monitoring, dynamic ad insertion, and QC testing across devices.
21 Mar 2024
What are the key factors that drive and impede new and emerging codec deployment on premium OTT streaming platforms like Netflix, Disney, and Apple TV+? Streaming Learning Center's Jan Ozer provides a much-needed codec deployment reality check in this clip from his presentation at Streaming Media East 2023.
25 Aug 2023
Amazon considers six key factors when choosing and adopting streaming codecs. AWS solutions architect Kevin Yao breaks them down in this clip from Streaming Media Connect 2023.
03 Apr 2023
Which streaming codecs do Netflix and Facebook most commonly use? Jan Ozer of Streaming Learning Center asks Andrey Norkin of Netflix and David Ronca of Facebook to discuss in-depth the specific codecs they employ for their organizations.
22 Nov 2022
What are the current innovations in reducing the carbon emissions and the carbon footprint of streaming? Jan Ozer of the Streaming Learning Center asks Kevin Yao of Amazon Web Services and David Ronca of Facebook about the innovations and new technologies their organizations are using to lessen the environmental impact of streaming.
30 Sep 2022
What are the key advantages of proprietary versus open-source codecs? Many factors come into play based on finding the right balance of customer needs in various use cases
14 Sep 2022
Advances in video codec technology driven by the explosion in video over IP are creating heated competition and a fragmented market. Will one winner emerge or is this the new normal?
04 Apr 2019