How to Think About VVC
On July 6, multiple companies announced the finalization of Versatile Video Coding (VVC) or H.266, the latest MPEG-based codec. Fraunhofer's announcement boasts "the same visual quality at 50 percent" the bitrate of HEVC, while Qualcomm claims a "40% reduction in file size." Congratulations to all involved; all standards are the culmination of years of effort by many companies.
What's not in the releases, of course, are many details that will determine VVC's success in both the short and long term. Two of these were expressed in David Ronca's response to Fraunhofer's VVC announcement on LinkedIn. By way of background, Ronca is currently the Director of Video Encoding at Facebook and was previously Director of Encoding Technologies at Netflix. Here's what he said about VVC.
"It is of no use that VVC can deliver 50% if it takes 3 weeks to encode 5 minutes of video. What matters is the practical capability of the encoder. That and the lack of IP uncertainty. A codec without clear and unambiguous licensing also has no value, and VVC carries the same uncertainties as HEVC, and thus I'm neither impressed with, nor excited about VVC."
On the first point, when I last wrote about VVC in Streaming Media magazine, VVC was projected to have a 10X complexity increase over HEVC. This doesn't translate to 3 weeks to encode 5 minutes of video, but it does mean that playback in software on mobile devices, TVs, and set-top boxes is probably out of the question, calling into question Qualcomm's statement that "commercial deployments [are] expected in 2021."
That's because at this complexity level, hardware playback is required, and it takes two years to develop the chip-level encoders/decoders and the consumer products that deploy them. As an example, the Alliance for Open Media (AOM) announced the code freeze for AV1 around April 2018 and the first TVs with AV1 decode starting shipping in April and May of 2020.
So, best case, VVC players will become available around mid-to-late 2022. However, as with any technology standard, there's at least some risk that it may not succeed at all. This relates to the second issue raised by Ronca, IP uncertainty. The largest mainstream markets for streaming video are computers, mobile devices, set-top boxes, and smart TVs. Manufacturers in these segments adopted HEVC without knowing the royalty structure because they assumed that it would be reasonable, and it was anything but. Once burned, twice shy, and you'd expect these manufacturers to delay committing to VVC until the royalty picture is clear.
In this regard, the Fraunhofer press release states that "A uniform and transparent licensing model based on the FRAND principle (i.e., fair, reasonable, and non-discriminatory) is planned to be established for the use of standard-essential patents related to H.266/VVC. For this purpose, the Media Coding Industry Forum (MC-IF) was founded."
Patent Pool Fostering
On a very positive note, the MC-IF recently announced that they were "fostering" a patent pool for VVC. What is fostering? Here's the definition from Fostering by Standards Bodies of the Formation of Patent Pools by American University's Carter Eltzroth:
[Fostering] is the process of encouraging patent holders to consider the formation of a pool covering a technology, for example, a standard. The activities of fostering can include raising awareness of the attractiveness of pooling to launch a market for the covered technology, initial meetings among potential rights holders, agreement on methodology for the choice of a facilitator, establishment of criteria for the choice of patents to be included in the pool, etc.
In the FAQ shipped with the announcement, the MC-IF further explained:
"Pool fostering" is an initial, precommercial step leading ideally to the formation of a single, voluntary patent pool. Participants in MC-IF's pool fostering effort will seek to select a pool facilitator that will complete pool formation and administer the pool, including licensing, royalty collection, and distribution. MC-IF will not itself complete the pool or administer it.
Jud Cary, who is VP and Deputy General Counsel at Cablelabs and Chair at MC-IF, announced the effort, which will start with a virtual meeting to be held on September 1, 2020. According to the press release, 17 companies have indicated that they will participate in this fostering effort. Cary also indicated that Eltzroth, the author mentioned above, will assist MC-IF in this effort as co-convener. This is significant because Eztzroth has participated in patent pool fostering in his work for European standard-setting organization DVB (Digital Video Broadcasting).
The major issue at this point is timing and the lack of certainty of success. The MC-IF FAQ states that "it is anticipated that the fostering work will have concluded, and a pool administrator selected, by late 2020 or early 2021." From there, forming a pool, vetting all the patents, and setting royalty rates, can easily take another 12-24 months. So we may not know royalty rates until early 2022 or later.
And there is no guarantee that the fostering effort will result in a single pool. As the FAQ states,
However, there can be no assurance that MC-IF's fostering efforts will result in consensus around a single pool facilitator. Those participating in MC-IF fostering are not obligated to join the facilitation effort or, if they do, they may choose to abandon that process at any time. As a result, the chosen pool facilitator may not be able to complete a pool with a critical mass of VVC patents. And notwithstanding the selection made by participants in the MC-IF fostering effort, other commercial facilitators may launch competing facilitation efforts.
The classic image that represented the dysfunctional HEVC licensing structure was created by Jonatan Samuelsson of Divideon; three patent pools and many very large companies not in any pool (this was the original image produced by Samuelsson in 2017; the picture has changed since then). We're at least five months away from knowing what a similar picture will look like for VVC, and a full year or so, at best, away from knowing the royalty rate and royalty structure.
The HEVC licensing landscape, circa 2017, according to Jonatan Samuelsson
We should note that, outside of lack of support in browsers owned by AOM members Google, Microsoft, and Mozilla, HEVC support is now near-universal on computers, mobile devices, smart TVs, and the latest generation of OTT devices. So, this dysfunctional royalty schema didn't sink HEVC, though it certainly slowed adoption.
That said, there's no guarantee that VVC can achieve the same success if the patent owners don't formulate a cohesive and commercially reasonable royalty structure. Companies that decided to deploy HEVC before the royalty picture was finalized may decline to take the same dive with VVC. And a similarly muddled picture could completely sink VVC.
This is particularly true given that all computers, mobile devices, smart TVs, and mobile devices will almost certainly support AV1 within the next 12 months if they don't already. Though AV1 may not be royalty-free—Sisvel announced an AV1 patent pool while AOM is "confident that AV1 overcomes these challenges"—even if there is royalty, it's a fraction of the HEVC royalty and companies may decide that AV1 support makes VVC unnecessary (the author consults with Sisvel regarding the AV1 and VP9 pools). It seems unlikely that Google, Mozilla, and Microsoft will support VVC in their respective browsers when it becomes available, so AV1 should give all publishers a much larger distribution footprint.
It's also worth noting that VVC has competition from inside MPEG, including Essential Video Coding (EVC) and Low Complexity Enhancement Video Coding (LCEVC). While neither offers the encoding performance that VVC does, both should have a clearer IP picture, and LCEVC can be implemented completely in software so it doesn't have the same implicit two-year delay (for more on these codecs and VVC, see "Inside MPEG's Ambitious Plan to Launch 3 Video Codecs in 2020").
The Net/Net
What's this all mean? Fraunhofer, Qualcomm, and other VVC contributors deserve their victory lap, and the MC-IF fostering the VVC pool is a great first step. Bringing in an experienced professional like Carter Eltzroth as convener adds great credibility to the effort and increases the probability of success.
But the computer, mobile, and consumer product developers who were burned by the HEVC licensing mess may refuse to buy into VVC without knowing the royalty structure, which may not become available for 12 to 24 months. Even if they decide to go forward immediately, the first products with VVC playback won't be widely available until mid-to-late 2022, at best. If they decide to wait for a clear royalty picture, it could be one to two years later.
So, if you're a streaming producer, the right way to think about VVC is to put off thinking about it at all until actual decoder deployments are announced on platforms that matter to you.
Related Articles
Tony Jones of MediaKind discusses the steady adoption rate of Versatile Video Coding (VVC) and its numerous benefits, such as significant cost savings and reduced energy consumption for streaming.
31 Jul 2023
A study from Unified Patents suggests that VVC isn't likely to ever gain widespread adoption due to both royalty issues and technology changes
30 Sep 2021
Access Advance and MPEG LA will both administer patent pools for VVC, but it's still unclear what will be covered under the patents and what the likely royalty costs will be.
28 Jan 2021
VVC today can be both useful and usable; let's hope that VVC IP owners can formulate a royalty policy that delivers the same.
07 Dec 2020
Advance testing shows the codec producing strong bitrate savings over its competitors, but a tiered licensing model could prevent all users from enjoying the best performance.
14 Nov 2019
The old realities that used to dictate codec adoption no longer apply. Opening up new markets now matters more than reducing operating expenses. How are HEVC, AV1, and VVC positioned for the future?
16 Aug 2019
Bitmovin Codec Engineer Christian Feldman unravels the relevant licensing issues surrounding the emerging codecs AV1 and VVC in this clip from his Video Engineering Summit presentation at Streaming Media East 2019.
24 Jun 2019
Bitmovin Codec Engineer Christian Feldman provides a snapshot of the current state of the AV1 and VVC codecs in this clip from his Video Engineering Summit presentation at Streaming Media East 2019.
19 Jun 2019