Streaming Media West: Apple Creates Advanced Experiences With HLS
When looking for HLS advice, go to the source. Delivering a second day kickoff address at Streaming Media 2018, Roger Pantos, media streaming engineer at Apple and one of the architects of HTTP Live Streaming (HLS), offered detailed advice on creating experiences that stand out.
"If you don't do a good job of content delivery, your user experience is going to suffer," Pantos said.
Standing out means making the best use of a few new and key technologies. First is HTTP/2, the successor for HTTP. It's been ratified for a few years now and is in wide use. This version offers a number of improvements for delivering media experiences, such as improved privacy with the ability to encrypt user requests.
"At Apple, we believe user privacy is paramount, so this is something we think is a real winner," Pantos said.
HTTP/2 also allows the HTTP connection to be reused for faster delivery, and gives clients more control over request ordering and bandwidth allocation among competing streams.
As for the media being delivered, Pantos said the goal is to create, package, and deliver only once, something that's possible with HLS. The key is using CMAF (Common Media Application Format). "With CMAF, you can encode once and deliver everywhere," Pantos said. Of course, there's a catch. When using DRM, providers want to use the strongest tools for each platform, but that means having multiple versions of content. The solution is that all major DRM platforms are standardizing on Common Encryption cbsc for CMAF, which allows a single version to work with all clients and platforms.
Content providers need to be careful how they respond to failures in delivery. On the server side, the most important thing to do is signal failures with an HTTP request that accurately reflects what's going on. Use the EXT-X-GAP tag to mark missing segments in linear streams. On the client side, collect anonymized analytics on performance and error rates.
Pantos explained how to package content and why it matters. Today's device population includes older devices that have limited abilities and newer ones that can handle more efficient codecs and provide better experiences with HDR or richer audio. Content owners want to deliver the best experience for each device without flooding older devices with advanced data they can't use. Complicating things, the playback experience can radically change in the middle of a viewing session, such as when the viewer connects an older device to an HDR television or an HDR device enters low-power mode. Providers need to prepare for those possibilities.
HLS helps with this situation by combining multiple encoding ladders in one master playlist. In a typical situation where the provider is offering AVC, HEVC, and HDR encodes, the HLS syntax lets all three ladders live in the same playlist, so the client can switch ladders when needed. For more details on this, Pantos said to view WWDC 2018 session 502, available online.
When providing interstitial content, such as adding bumpers or trailers, or inserting ads in a live linear stream, providers need to use the discontinuity tag, which lets the playlist know that a discontinuity is coming. But keep in mind, Pantos warned, that the tag needs to be in the same place in each media playlist. Live streams also need to use a discontinuity sequence tag, which prevents the player from getting lost when switching playlists. The sequence tag must be in sync across all playlists.
"HTTP Live Streaming is an ongoing effort," Pantos said, developed 10 years ago and modified over time to meet industry needs. If content creators need modifications to create new kind of experiences, Pantos said he welcomes feedback. Address comments to email@example.com.
Apple's Roger Pantos addressing Streaming Media West in Huntington Beach, Calif.
The strong got stronger in 2017, with HLS and H.264 growing while VP9 took a tumble. Despite all the noise around 4K, it still isn't used much.
It's not easy, but it can be done, as long as you have the right tool and know the tricks. Here are eight steps to success using Softvelum Nimble Streamer.
Many who heard that Apple is adding support for HEVC playback in HTTP Live Streaming were left with more questions than answers. Here's what developers need to know.