Video: Pros and Cons of Supervised Machine Learning for Content Personalization

Learn more about real-world video metrics at Streaming Media's next event.

Watch the complete video of this panel, AI104A: Personalization Using Real-Time End-User Feedback, in the Streaming Media Conference Video Portal.

Read the complete transcript of this clip:

Rafah Hosn: Let's talk a little bit about interactive learning for personalization. Let's take a content scenario. This is Paul. Paul likes to browse news before he makes his weekly fantasy picks. Susan, on the other hand, she likes to browse business news right before she calls her clients.

The question is, why should these two people with very different interests see the same content? We should be able to personalize for each. The second question, which comes up very often is, if you're personalizing content that changes constantly underneath your feet, how do you manage that? A good personalization service is one that is continuously adapting to both content and context changes.

How do people do this in machine learning-land? The most common way among folks that do machine learning for personalization is to use a supervised learning paradigm. In a supervised learning paradigm, you start with a training example. Then you spend X-millions of dollars labeling those things. You shove them into one of multiple varieties of supervised learners. Out comes a classifier, which you can give a number in real time if you've trained it on numbers. For a certain amount of accuracy, it will give you that number.

Supervised learning works well. It's the underlying technology for Speech-reco, Siri, Alexa, and Google Home. All of these thing came about from supervised learning: Image Net, Image Reco, Face APIs. All of these are the results of a good supervised learning algorithm.

How about news? What if you wanted to personalize any type of content, such as news, video feeds, playlists, and you're trying to maximize a business outcome? Suppose, for the case of this scenario, you're interested about click-through rates. To build a supervised learner, you collect some user article-click information. You build a feature set, and you learn how to predict a click given a user and an article. At most companies, you deploy this in an A/B test. Then it fails. The question, is why is my A/B test failing?

So, what goes wrong? The first question: Is Canada interesting to Rafah? If you've never offered me Canadian news on my favorite website, you have no historical data on me ever clicking on a news article pertaining to Canada. Your supervised learner has never seen that example, so if you're training on historical data, the answer is no, but it turns out I'm Canadian and I follow Canadian news, so you have to have the right signal to get the right answer. It's very important, especially when you're training over historical data.

Another thing that goes wrong is when you're creating supervised learning models that are based on historical data. This is real data, by the way, from a partner of ours. A model that's trained day one loses half its value by day two, especially when content is constantly changing underneath your feet because, again, you're training on historical data. Unless you're continuously relabeling and retraining, those models lose their value over time.

Streaming Covers
Free
for qualified subscribers
Subscribe Now Current Issue Past Issues
Related Articles

Video: Demo: How to Make Your Video More Searchable

SeaChange International SVP, Strategy Mark Tubinis demonstrates how to use enriched metadata in OTT personalization in this clip from Streaming Media West 2018.

Video: The Challenge of OTT Personalization

SeaChange International SVP, Strategy Mark Tubinis discusses the difficulty of managing the data necessary to create personalized OTT experiences in this clip from Streaming Media West 2018.

Video: How to Use Data to Create Dynamic User Experiences

Ownzones CTO Aaron Sloman discusses how Ownzones provides value to its enterprise video clients through big data analytics in this clip from his panel at OTT Leadership Summit at Streaming Media West 2018.

Video: 5 Questions to Ask in UX Personalization

contentwise VP, Technical Services Ronato Bonomini discusses OTT personalization strategies in this clip from OTT Leadership Summit at Streaming Media West 2018.

Video: Best Practices for Training Your AI

Limelight's Jason Hofmann, Citrix' Josh Gray, and REELY's Cullen Gallagher discuss best practices for training AI systems at Streaming Media East 2018.

Video: How to Know When AI Isn't the Solution to Your Problem

Google's Matthieu Lorrain cautions of the risks of doing AI for its own sake in this clip from Streaming Media West 2018.

Video: Key Considerations When Choosing a Video AI Platform

RealEyes Director of Technology Jun Heider discusses the importance of internal self-assessment and which use-case elements to consider when choosing a platform for video AI in this clip from Streaming Media East 2018.

Video: Who Are the Key Players in Video AI?

RealEyes Media Director of Technology Jun Heider identifies the key players in the AI platform space in this clip from Streaming Media East 2018.

Video: Tips for Getting Started with Video AI Platforms

RealEyes Director of Technology Jun Heider outlines the first steps in choosing an AI platform in this clip from his presentation at Streaming Media East 2018.

Video: How Reinforcement Learning Enables Personalized Viewing Experiences

Microsoft Principal Product Manager Rafah Hosn makes the case for reinforcement learning as a machine learning paradigm for content personalization in this clip from Streaming Media East 2018.

Video: How Microsoft's Custom Decision Service Improves Content and Ad Engagement for Brands

Microsoft Principal Product Manager Rafah Hosn explains how Microsoft's machine learning-driven decision services helps brands target viewers and increase engagement in this clip from Streaming Media East 2018.

Video: How Do We Define Quality of Experience for Streaming Video?

Comcast Technical Solutions Architect Ribal Najjar defines video QoE both in terms of subjective experience and qualitative measurement in this clip from Streaming Media East 2018.

Video: How IRIS.TV Implements Machine Learning in Production Environments

IRIS.TV CEO & Co-Founder breaks down discusses IRIS.TV's approach to helping traditional media companies capture and leverage audience data and machine learning in this clip from Streaming Media East 2018.

Video: How USA Today Leveraged Video AI at the 2018 Winter Olympics

Gannett Senior Director Kara Chiles discusses how USA Today leveraged IRIS.TV and data to localize and personalize their Winter Olympics 2018 coverage in this clip from Streaming Media East 2018.

Video: How to Use Machine Learning to Create Personalized TV Experiences

ZoneTV's Tom Sauer describes how machine learning can be used to overhaul the TV world and deliver more individualized experiences in this clip from Streaming Media East 2018.

Video: How AI Can Open Up New Markets for Your Video

REELY CEO Cullen Gallagher makes the business-growth case for content owners developing an AI strategy in this clip from Streaming Media East 2018.

Video: How IBM is Using Video AI

IBM Watson Media's David Clevinger discusses how media entities are currently using video AI in this clip from Streaming Media East 2018.

Video: How Video AI Helps Businesses Interpret Experience Metrics

Citrix Principal Architect Josh Gray explains how video enables higher-acuity metrics analysis in this clip from Streaming Media East 2018.

Video: How Video AI Improves Content Delivery Efficiency

Limelight VP of Architecture Jason Hofmann discusses how AI impacts content delivery optimization in this clip from Streaming Media East 2018.

Video: Best Practices for Developing Machine Learning Algorithms for Video

Citrix' Josh Gray provides tips on AI model development and Reality Software's Nadine Krefetz and IBM's David Clevinger speculate on the possibilities of metadata-as-a-service in this clip from Streaming Media East 2018.

Video: How Will Machine Learning Impact the Media Supply Chain?

Google's Leonidas Kantothanassis explores the vast range of applications for machine learning in the media workflow and supply change in this clip from his Content Delivery Summit keynote.