How LCEVC Works
Learn more about emerging codecs and standards at Streaming Media West 2020.
Read the complete transcript of this clip:
Guido Meardi: Very often, when talking about LCEVC, I hear words like "too good to be true", or "how is it possible," or "it cannot be possible." So, my objective would be to make sure that you understand how it works and the fact that it's not magic. It's actually pretty simple also; that's why it's "low complexity."
Essentially, the codec works by using another codec at the lower resolution, of course, to make sure that the combined entity of leverage codec plus enhancement takes less power and less bandwidth One requirement is using the leverage codec at the lower resolution so that it can focus on fewer pixels. So, less processing, and it can be more efficient than cranking those pixels. And then, the high frequency, so the additional data is encoded using a special encoder, which is LCEVC, that is aimed specifically at coding enhancement data.
I will show in a bit what enhancement data is, but essentially, what LCEVC does is it codes the lower resolution and then the delta between the lower resolution and the top resolution, top quality, gets encoded with this special enhancement codec.
At the decoding side, the video stream is a master stream containing two elementary streams, a normal stream of the base codec, and an enhancement stream which can be packaged as a separate PID or SICI metadata, or even sent as an independent stream--there are several options.
But at the decoding side, there is a standard decoder, even hardware that decodes the base codec, and then, the enhancement is decoded separately and overlaid and merged with the base in order to obtain the final output. If you want to see it in a more visual way, this is kind of like how it works at a decoding site, where you would get a sort of thumbnail video or a lower-resolution video encoded with the traditional codec, whether H.264, HEVC, AV1, whatever. There's a first layer of residual correction before an upsampling pass.
There can be also an upsampling pass here--it depends; it's parametric. There's a correction layer that corrects before the most important upsampling pass, that brings to full resolution, and then, there's an additional sublayer of residuals that adds the details at the end. Also, leveraging a temporal buffer.
Importantly, as you can see, these sublayers of one LCVEC and codes are not pictures, they are residuals, and that's actually quite important, because they are something like this: When you talk about details, this is an example of what LCEVC would encode of a 1080p of, in this case, it's pretty evidently a Formula One video. And you can see that there are a lot of details, a lot of dots, a lot of edges, a lot of text detail. This stuff, which would be high frequencies, is very costly for traditional codecs.
If you are using a DCT with big blocks, think about HEVC 64x64 coding units with DCT, when you have a single dot, it will be tens of coefficients to encode it. LCEVC is actually meant to encode this stuff, so it has very small transforms, aimed at high frequencies that are extremely efficient at compressing high detail.
Unlike other cutting-edge codecs coming to market, LCEVC will have an immediate impact on the streaming landscape. Here's how to use it.
V-Nova CEO & Co-Founder Guido Meardi discusses current issues with encoding complexity and cost and how LCEVC--Low Complexity Enhancement Video Coding--addresses those issues in this clip from his presentation at Video Engineering Summit.