Video: How to Make MLaaS Work for You

Learn more about machine learning and AI at Streaming Media's next event.

Watch Jun Heider's complete presentation, VES103. Enhancing Media with Machine Learning in 2019, in the Streaming Media Conference Video Portal.

Read the complete transcript of this clip:

Jun Heider: Let's talk about how to make MLaaS work for you. I think the biggest thing to do is ask yourself some questions first. What kind of content do you have? What is your use case, what are you trying to do with your content? How accurate do you need the results?

Somebody that I was talking to in a previous session told me they work for a firm that deals with content that is government legislation and things like that. When you're talking about government legislation, especially in Europe, you want to make sure that that is a really accurate transcription--otherwise people may get mad. So that's a case where you want high, high accuracy.

Do you have any developers on your team? A lot of the services expose an API, and often you're going to want to use their API to actually build some useful applications around what you're doing with these machine learning services.

Finally, your content is your content, and it's not my content; it's specific to you. Some of these services are going to work better with certain types of content, because they haven't had the ability to label every single video that ever existed in human history, and run it through their model. So, some of these are going to be tuned to sports, some of these are going to be more tuned to nature videos that have a lot of animals in them. So you're going to want to take your content and leverage these trials they make available, to see which ones are the best ones for you.

What is your content profile? Movies, user-generated content, nature clips--play with them. If you have a developer on your team, or you are a developer, or you have a team of developers, definitely take a look at the API.

The nice thing about Video Indexer is they actually let you play with their services in their API documentation, so that's a really cool feature. And then those spreadsheets that I shared with you, this is kinda what they look like. We went from Microsoft Video Indexer, Valossa, Google Video, Intelligence, and AWS Recognition, and we pulled all the tags out. We checked to see how accurate they were, and if they weren't, what are our thoughts on why they weren't?

For instance, the Video Indexer recently said that there was a gun in the video, but then when I went back to the frame where it supposedly saw a gun, it was actually an old-style camera. So the old-style, with the flash bulb at the top, and somebody was side-cocking the camera, which I could think is why it may have thought it was a gun. Another example is, I ran a very old black-and-white cartoon and there was a villain that was kind of doing this, and it thought it was a bat, because it was black and white and he had a big black cloak on.

Feel free to take a look at this, see where the services return similar tags, different tags, where they get tags wrong. This will get you started, but the thing to note is, we did this in 2018. These models are constantly learning, they're constantly training these models, so I can guarantee you, if we went back and did this again, it would be different. But this will at least give you a feel for what kind of stuff to get out of these services.

Streaming Covers
Free
for qualified subscribers
Subscribe Now Current Issue Past Issues
Related Articles

Video: Audio Analysis and Machine Learning for Video

RealEyes Media CTO Jun Heider discusses audio-specific machine learning services, from speech-to-text to sentiment analysis in this clip from his Video Engineering Summit at Streaming Media East 2019.

Video: What's the Difference Between Machine Learning and AI?

Microsoft's Andy Beach and IBM/Watson Media's Ethan Dreilinger break down the differences between machine learning and AI in this clip from their panel at Streaming Media West 2018.

Video: Key Considerations When Choosing a Video AI Platform

RealEyes Director of Technology Jun Heider discusses the importance of internal self-assessment and which use-case elements to consider when choosing a platform for video AI in this clip from Streaming Media East 2018.

Video: Tips for Getting Started with Video AI Platforms

RealEyes Director of Technology Jun Heider outlines the first steps in choosing an AI platform in this clip from his presentation at Streaming Media East 2018.

Companies and Suppliers Mentioned