How to Balance Performance and QoE Metrics for Livestreams
When it comes to monitoring live streams, teams often divide metrics into technical categories related to streaming performance like bitrate and latency and buffering, and more subjective metrics related to quality of experience for end users. But according to Warner Bros.’ Subhrendu Sarkar and NBCUniversal’s Seb Emin, the two are really inseparable, as delivering satisfying experiences to viewers whether they’re watching on 65" smart TVs or on a “mobile device on a train going through a tunnel” is always the end goal, and they explain how to strike that critical balance while always keeping QoE front-of-mind in this clip from Streaming Media Connect 2025.
Quality drives engagement
Dillon Media Ventures' Rob Dillon frames the topic with a pointed question: "When it comes to streaming performance, how do your teams balance the data between technical aspects like bitrate and latency and user-centric metrics like engagement and satisfaction?"
Emin counters that the two categories of quality metrics can't really be separated, as they're all pointed toward the same purpose; "quality ultimately drives engagement," he contends. "If you have a stable, feature-rich platform, users are going to stay and they're going to keep watching. If they experience constant dropouts and rebuffering, they are much, much more likely to go and do something else because it's going to be very frustrating. So we don't separate technical from engagement because one drives the other."
The real trick, he says, is to deliver a consistent experience across all the devices on which your audience is likely to be watching. His team, he says, puts "a lot of time and effort into looking at the data and then optimising on a device by device level." Successful streaming is not a "one size fits all" proposition. "We optimise for a given platform so that we can improve the metrics specifically on that platform. That's how we've ended up where we are: through using that data to inform decisions on what we build and what we work on to drive that experience."
Sarkar agrees that engaging experiences are always driven by multiple factors. "Historically, and even today, it's very challenging to look at user engagement and say, 'This bitrate increase actually led to better engagement.' Because there are always a lot of other variables in the mix, things like, 'What is the content and how popular is that content by itself?' So it can be difficult to correlate," he concedes.
"But what is encouraging for me is that better quality metrics are becoming table stakes for the industry." Whereas at one time users didn't necessarily expect 1080p streams, he says, today 4K and HDR are becoming standard. "So I think we are just pushing the boundaries up and up and making the user experience better and better. This drives the overall engagement in the platform and how consumers consume the content."
Tunnel vision
When it comes to quality expectations on a technical level, Emin allows that broadcast sets the standard at the top end, and streaming must meet that bar. "From a user perspective, between broadcast and streaming, users generally have the same expectations in terms of quality, and I think that's completely fair," he says. "Why shouldn't they?"
But he also acknowledges that "streaming has far more variables and complexities. When you're working in a broadcast environment, you can make some very safe assumptions about the sort of environment that user is watching in and the conditions under which they're watching streaming."
With streaming, by contrast, "You have all sorts of different bandwidths, you have all sorts of different devices, and I think people take that for granted a lot. As a service, we need to still provide that livingroom experience that you would get on broadcast on a 65" TV in someone's livingroom. But similarly, we also have to provide a great experience to someone that's watching on their phone, on a train, going through a tunnel, and that's where the real complexity comes in and I think that's overlooked a lot. But I think that's where these metrics marrying together becomes really important, especially with the uptick in usage in streaming over broadcast."
Related Articles
When it comes to live stream monitoring and analytics, the problem is usually having too much data rather than not enough, along with identifying the key data points to pinpoint problems and optimize streams. NBCUniversal's Seb Emin and Warner Bros. Discovery's Subhrendu Sarkar explain how two major content providers keep analysis paralysis at bay in this discussion with Dillon Media Ventures' Rob Dillon at May's Streaming Media Connect.
27 Jun 2025
Streaming Media presented its 17th Connect virtual conference May 20-22, featuring speakers from Paramount, Meta, Google, NBA, Roku, Warner Bros. Discovery, AWS, Akamai, Fubo, DAZN, Fremantle, IAB, Philo, Estrella, Eluvio, nanocosmos, Cerberus, and more, and session topics ranging from and sports on FAST to biddable advertising to SGAI to AI in live streaming to MoQ and more.
27 May 2025
How do you define Quality of Experience (QoE) and Quality of Service (QoS) in live and on-demand streaming, and what are the key metrics for measuring them? IMAX's Abdul Rehman, YouTube's Sean McCarthy, SVTA's Jason Thibeault, and PADEM Media Group's Allan McLennan discuss these topics in this clip from Streaming Media Connect 2024.
10 Dec 2024
Companies and Suppliers Mentioned