Measuring Success: The Multiplatform Analytics Challenge
The team at Amplitude, which focuses on web and mobile analytics, gives an example of event-based analytics as one form of measurement that can be properly calibrated across multiple platforms.
Amplitude tends to intentionally focus on app measurement, since a number of the company’s clients offer online games that may spike in popularity in very short periods of time. Like streaming, which has its peak demand periods—from the launch date of a new season of a popular episodic program to the occurrence of a major sporting event—the gaming industry faces rapid spikes in demand and equally rapid falloffs of interest in any given game.
Amplitude’s work with the gaming industry, especially its mobile gaming customers, is applicable to what we do. And what Amplitude seems to do well is help companies decide how to make their products available on as many platforms as is profitable.
In our infancy, the streaming industry had the mentality of making content and players available on as many platforms as possible, sometimes to the detriment of profitability. But as the gaming industry shows us, the use of event-based analytics helps strike the balance between user actions and the need for a balanced number of apps spanning multiple platforms and devices.
Event-based analytics, though, require careful thought as to which steps (events) in a typical customer workflow are important to measure.
“Sending meaningful event data—data that tracks with customer retention and value—to your analytics platform is the single most important step toward understanding how your users are engaging with your product,” writes Archana Madhavan in a sneak peek of Amplitude’s upcoming Retention Playbook, a set of best practices in how to retain customers in a mobile-first environment.
In other words, not all measurements are created equal. Correlating goals and events is key to properly measuring and auditing delivery effectiveness and, ultimately, profitability.
Measuring Users
One area where the effectiveness of event-based analytics can be sidetracked is a lack of correctly defining an active user. In classic web analytics, an active user measurement is defined by the number of unique visitors to a site at distinct increments.
Inaccurate measurement of active users can lead to overinflated viewer numbers, based on redundant counts or inclusion of those who might not actually be performing steps deemed important to event-based analytics.
“To get the most value out of your active user counts and retention metrics,” writes Madhavan, “make sure to measure active users based on events that actually matter to you.”
If a user is just passively browsing content but not actively viewing clips, episodes, or full-length premium content, should they be counted as active users?
Google Analytics defines the increments in four durations: 1, 7, 14, and 30 days. Google allows these increments to be defined by total active users, as well as by direct or organic traffic.
Visits Versus Viewers
The difference between visits and active users is often a murky area. A good example of this can be illustrated by the massively popular drudgereport.com. The Drudge Report website may look like a 1997-era HTML class project that hasn’t kept up with the times, but the site proudly displays numbers that would make even social media mavens jealous. In essence, the site is a series of well-crafted headlines, linking to other sites such as the Associated Press, The New York Times, and even U.K.-based newspapers.
The day this article was written, drudgereport.com had been visited 33,383,997 times in a 24-hour period and had racked up almost 10 billion (yes, billion) visits in the last year: 9,371,312,497 to be exact, based on the counter at the bottom of the page.
Despite the impressive statistics, though, the number of active users is significantly lower. Quantcast reports that the number of unique visitors over a 30-day period, or what we’d call active users, is 20.4 million with 19.5 million of those coming from the United States. The average person visits the site eight times in a month, and actively clicks through a headline four times per visit.
What’s fascinating, when viewing the Quantcast data about this heavily viewed site, is that cross-platform content viewing shows a consistent pattern between mobile and desktop viewing. Desktop viewing rises to almost 1.5 million active viewers at the beginning of the workweek but falls off precipitously on Friday afternoon to well below 1 million active viewers on Saturday and Sunday.
Mobile viewing, by contrast, is an almost constant 1 million active viewers, regardless of the day of the week. This steady viewership means that weekend visits to the site are more likely to occur on mobile devices, a fact that should have significant impact on both the site’s mobile layout and its advertising formats.
Build for Peaks or Valleys?
So how does the active user data for one of the most widely read news aggregation sites compare with active users for Netflix, the world’s most popular OTT content aggregator?
While the Quantcast data is a bit slim, it appears that Netflix averages 53 million unique active users on its website. Yet that average number only tells a small portion of the Netflix story, since the majority of Netflix content is consumed on apps across multiple devices—Amazon Fire, Apple TV, Android TV, and Roku are just a few of the dozens of devices—and multiple operating systems.
We also know as an industry that the Netflix average active userbase tends to fluctuate based on the release of premium content. In fact, the Quantcast data reveals just how wide the swing can be for the world’s premier OTT provider: from an average of 53 million in September 2015, Netflix traffic dipped to 30 million unique active users in November 2015.
Just a few months later, though, in February 2016, active users spiked to a 110 million peak. By the end of May 2016, the numbers seesawed back to just under 9 million unique active users per month, then returned to about 56 million active users in July 2016.
Quantcast points out that Netflix, which it ranks as the 18th-highest trafficked website in the world, has not been accurately quantified, or measured. Knowing how to accurately build for the peaks and valleys of OTT viewership, then, would require more than just the Quantcast data.
On the Netflix investor relations page, the company defines its established userbase as, “over 83 million members in over 190 countries.” These numbers are obviously higher than the 53–56 million unique viewers shown in the Quantcast data, but they’re much lower than the 110 million peak hit earlier in the year.
So how do we rationalize these two distinct sets of numbers? Fortunately, as most industry veterans know, Netflix membership allows for multiple simultaneous active users. Some members choose a higher tier of service, which not only allows them to watch higher-quality streams but also allows up to five users to simultaneously view content on different screens (e.g., devices or platforms).
Conclusion
We’ve barely scratched the surface of this complex topic, but let’s wrap up this discussion of multiplatform delivery and the corresponding analytics with a key thought from earlier in the article: Reach and profitability may be diametrically opposed to one another.
Perhaps distributing content on multiple platforms is not the most profitable approach for most content owners. But perhaps it is. Your conclusion will be based on multiple facets. Fortunately, each of those facets, if based on established goals and analytics calibrated across multiple platforms, can be measured. And those measurements—based on a multipronged approach to analytics that goes far beyond total viewers or minutes viewed—are what will help online video content owners decide how best to make their products available on as many platforms as is profitable.
[This article appears in the October 2016 issue of Streaming Media magazine as "The Multiplatform Analytics Challenge."]
Related Articles
Experts from Akamai, Comcast, Conviva, and Yahoo! discuss key analytics, metrics, and assessment strategies that will help you deliver more effective and profitable live streams.
28 Sep 2016
For two years an error in how Facebook calculates average view time led to dramatically higher averages. The problem shows the need for third-party verification.
23 Sep 2016
Objective quality benchmarks provide exceptionally useful data for any producer wanting to substitute fact for untested opinions.
01 Apr 2016
At long last, broadcasters, content creators, and advertisers will be able to compare a program's views across all platforms.
23 Oct 2015
Also, Adobe announces TBS as a Primetime customer; will drive live linear and VOD content for TBS apps and sites.
24 Sep 2013