▼ Scroll to Site ▼

Video: How Reinforcement Learning Enables Personalized Viewing Experiences

Learn more about personalizations at Streaming Media's next event.

Watch the complete video of this presentations, AI104A: Personalization Using Real-Time End-User Feedback, in the Streaming Media Conference Video Portal.

Read the complete transcript of this clip:

Rafah Hosn: Here's another solution to consider. This is a new paradigm in machine learning called reinforcement learning. And here we use real, end-user feedback to train our model. So, how does that work? Let's take our news example again.

We have a user. They come to their favorite news website. There's an online learner on the background. Whenever a user comes in, a user has context. That means they have some features set, the geolocation they are in, or the type of device they're using. We call this the context X. Then, inside that red box, there is a policy, a model, that's choosing the best list of new stories for this user based on that context X. We call this an action.

Now the key in reinforcement learning is that we don't stop there. The learner proposes an action, and waits for the user feedback. That's why it's called reinforcement learning, because every time the user clicks, that's a positive reinforcement feedback for the online learner. It's exactly like teaching a puppy a trick. Every time the puppy does something, you give him a treat. That's a positive feedback. Every time the puppy does something wrong, you say, "Bad puppy." It doesn't learn from that. It's exactly the same principle. That's kind of the paradigm that we use for personalization.

Now, the key in reinforcement learning is a concept called exploration. For the gentleman that asked about the cats and dogs, at least in the type of reinforcement learning we do, suppose that you love space. So in most, 80% of the time we're going to choose for you space articles because that what we learn that you like. But add some random .2%, .1%, 2%, and we're going to choose a different article. So, we'll show you cats and at some configurable number, we say, "You know what? We're going to explore this space and see if this gentleman actually likes a little bit of dogs." So, we'll show the dog. Then we'll observe your reaction to the dog. If you give us a positive feedback, we say, "Oh, well. Okay. Maybe it's not just cats. Maybe he does like dogs a little bit."

This exploration is very, very powerful. This is not complete random exploration. It's exploration over the set of your feasible actions. So in the context of news, your editorial comes with 12 lists of stories, trending stories that should be shown on the page. So, 80% of the time we will show the news that we think you like, and 20% we will randomize over this list of 12 articles and propose a different type of article and see if the user reacts positively to it. That's our positive reinforcement feedback for this algorithm.

It turns out that exploration is so powerful because it allows you to now label your data set automatically. You don't need to go and spend money labeling your data set because every positive reinforcement is a label. And any time you're exploring, you're actually increasing your data set. So, this gives you a very rich data set that you can learn from.

Streaming Covers
Free
for qualified subscribers
Subscribe Now Current Issue Past Issues
Related Articles

Video: Demo: How to Make Your Video More Searchable

SeaChange International SVP, Strategy Mark Tubinis demonstrates how to use enriched metadata in OTT personalization in this clip from Streaming Media West 2018.

Video: The Challenge of OTT Personalization

SeaChange International SVP, Strategy Mark Tubinis discusses the difficulty of managing the data necessary to create personalized OTT experiences in this clip from Streaming Media West 2018.

Video: Best Practices for Training Your AI

Limelight's Jason Hofmann, Citrix' Josh Gray, and REELY's Cullen Gallagher discuss best practices for training AI systems at Streaming Media East 2018.

Video: How to Know When AI Isn't the Solution to Your Problem

Google's Matthieu Lorrain cautions of the risks of doing AI for its own sake in this clip from Streaming Media West 2018.

Video: Key Considerations When Choosing a Video AI Platform

RealEyes Director of Technology Jun Heider discusses the importance of internal self-assessment and which use-case elements to consider when choosing a platform for video AI in this clip from Streaming Media East 2018.

Video: Who Are the Key Players in Video AI?

RealEyes Media Director of Technology Jun Heider identifies the key players in the AI platform space in this clip from Streaming Media East 2018.

Video: Tips for Getting Started with Video AI Platforms

RealEyes Director of Technology Jun Heider outlines the first steps in choosing an AI platform in this clip from his presentation at Streaming Media East 2018.

Video: Pros and Cons of Supervised Machine Learning for Content Personalization

Microsoft Principal Product Manager Rafah Hosn discusses the benefits and limitations of a content personalization strategy based on supervised machine learning in this clip from Streaming Media East 2018.

Video: How Microsoft's Custom Decision Service Improves Content and Ad Engagement for Brands

Microsoft Principal Product Manager Rafah Hosn explains how Microsoft's machine learning-driven decision services helps brands target viewers and increase engagement in this clip from Streaming Media East 2018.

Video: How Do We Define Quality of Experience for Streaming Video?

Comcast Technical Solutions Architect Ribal Najjar defines video QoE both in terms of subjective experience and qualitative measurement in this clip from Streaming Media East 2018.

Video: How IRIS.TV Implements Machine Learning in Production Environments

IRIS.TV CEO & Co-Founder breaks down discusses IRIS.TV's approach to helping traditional media companies capture and leverage audience data and machine learning in this clip from Streaming Media East 2018.

Video: How USA Today Leveraged Video AI at the 2018 Winter Olympics

Gannett Senior Director Kara Chiles discusses how USA Today leveraged IRIS.TV and data to localize and personalize their Winter Olympics 2018 coverage in this clip from Streaming Media East 2018.

Video: How to Use Machine Learning to Create Personalized TV Experiences

ZoneTV's Tom Sauer describes how machine learning can be used to overhaul the TV world and deliver more individualized experiences in this clip from Streaming Media East 2018.

Video: How AI Can Open Up New Markets for Your Video

REELY CEO Cullen Gallagher makes the business-growth case for content owners developing an AI strategy in this clip from Streaming Media East 2018.

Video: How IBM is Using Video AI

IBM Watson Media's David Clevinger discusses how media entities are currently using video AI in this clip from Streaming Media East 2018.

Video: How Video AI Helps Businesses Interpret Experience Metrics

Citrix Principal Architect Josh Gray explains how video enables higher-acuity metrics analysis in this clip from Streaming Media East 2018.

Video: How Video AI Improves Content Delivery Efficiency

Limelight VP of Architecture Jason Hofmann discusses how AI impacts content delivery optimization in this clip from Streaming Media East 2018.

Video: Best Practices for Developing Machine Learning Algorithms for Video

Citrix' Josh Gray provides tips on AI model development and Reality Software's Nadine Krefetz and IBM's David Clevinger speculate on the possibilities of metadata-as-a-service in this clip from Streaming Media East 2018.

Video: How Will Machine Learning Impact the Media Supply Chain?

Google's Leonidas Kantothanassis explores the vast range of applications for machine learning in the media workflow and supply change in this clip from his Content Delivery Summit keynote.