How to Deploy GPAC for FFmpeg Packaging and ABR Distribution

As much as we love FFmpeg for transcoding op­erations, it can get frustrating when packag­ing your content for ABR delivery. By packag­ing, I mean formatting and segmenting your media files, creating manifest files for HLS and DASH, for­matting for CMAF, and managing multiple audio and subtitle streams. Fortunately, there are easier-to-use solutions that are equally open source and equal­ly free. In this article, I’ll focus on GPAC, which is a great packaging alternative.

As you can read on its website, gpac.io, GPAC is an open source multimedia framework used in many media production chains. It’s an adjunct to FFmpeg, and it can use FFmpeg for multiple processes. In this article, which focuses on VOD, I’ll use FFmpeg and MP4Box, one of the GPAC executables, separately.

In terms of credentials, GPAC is used by many sig­nificant companies, including Netflix, which is pack­aging some of its live-streaming offerings via GPAC. You can read about some other GPAC users here.

Getting Started

Let’s start with some preliminaries. GPAC is avail­able for Windows 32-bit (Vista+), Windows 64-bit (Vis­ta+), Mac OS X, Linux 32 bits (Ubuntu 18.04*), Linux 64 bits (Ubuntu 20.04*), iOS (needs re-signing for your device), and Android. Once you install, there are three sets of tools:

• MP4Box, the multimedia packager that I will use in this article
• gpac, a generic media pipeline orchestrator not used in this article
• Bindings for Python and Node.js, which are also not used in this article

You’ll need Python and FFmpeg on your system to run MP4Box. I used Python 3.10.7 and FFmpeg 6.

I’ll work with a common set of files through all scenarios, and you can download these, the batch commands, and all output files here. As you can see in Figure 1, there are three files in an encoding lad­der, as well as English and French audio and subtitle tracks. The subtitles are courtesy of Romain Bouqueau, one of GPAC’s prin­cipal architects since 2007 and my technical contact throughout this process. The French audio translation is computer-gener­ated and not particularly syn­chronized, but it is French, so it probably won’t care.

Figure 1. Source files for all projects

I’ll create four sets of packaged files:

• HLS—One file per ladder rung (MPEG-2 transport stream format)
• DASH—One file per ladder rung (fragmented MP4 format; fMP4)
• CMAF—One file per ladder rung (fMP4)
• CMAF—One file per ladder rung with English/ French audio and subtitles (fMP4)

Example 1: HLS—One File per Ladder Rung (MPEG-2 Transport Stream)

The initial project is to create HLS output in segment­ed transport stream files, the most common use case for HLS. Here’s the command string and explanation:

MP4Box -dash 2000 -profile onDemand jan_1080p.mp4 jan_720p.mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out hls.m3u8 --muxtype=ts

• MP4Box—Calls MP4Box
• –dash 2000—Creates 2,000 ms segments
• –profile onDemand—Creates a single .ts file for each media asset. Use -profile live for separate .ts files.
• A list of all media files in the ladder (including captions)
• -out hls.m3u8 --muxtype=ts—Names the HLS manifest files and formats them as .ts files, not fMP4

Note that the subtitles didn’t appear in the VLC media player until I upgraded to version 3.0.20. If you have any issues at all with any of these processes, be sure to upgrade to the most recent copy of GPAC and the most recent version of VLC (go2sm.com/vlc).

The outputs are (see Figure 2):

• hls.m3u8—Master manifest
• hls_n.m3u8—Manifest file for each media asset
• .ts/vtt—Media files for each media asset

Figure 2. Media and manifest files created by MP4Box

Copy all of these into the same folder on your website, and link to the master, and you’ve got an HLS asset ready to play.

Note that the media files have the word “dash” in­cluded, because architecturally, HLS is generated by the GPAC module creating the DASH output. You can remove the dash designation via commands docu­mented in the help file, but I decided to keep it sim­ple for this tutorial.

Moving on, Figure 3 shows the master manifest file, which you would link to or click to play the HLS presentation. As you can see, the master contains file details that allow the player to choose the correct file at the start and during play­back. Typically, playback begins with the first file in the master but can switch to any other vid­eo based on available bandwidth or compatibility. So, if a very old iPhone clicks on the link, it might have to skip to the 540p file if it can’t play the 1080p file.

Figure 3. The master manifest file identifies the location and relevant details of the media files.

If you open any of the other .m3u8 files, you’ll see that they call 2-second byte range re­quests in each media file to play the presentation.

Example 2: DASH—One File per Ladder Rung (fMP4)

Now, let’s create DASH output from the same source files. Here’s the command string:

MP4Box -dash 2000 jan_1080p.mp4 jan_720p. mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out jan_DASH.mpd

• MP4Box—Calls program
• –dash 2000—DASH output, 2-second byte range requests
• A list of all media files (including captions)
• -out jan_DASH.mpd—Labels the MPD file jan_DASH

MP4Box creates the files shown in Figure 4. You would link to the MPD file, which contains byte range requests for all of the 2-second segments in the media files.

Figure 4. DASH output

Figure 5 shows the subtitle track and subtitles play­ing over the video. Since I’ve only included a single subtitle track, that’s all you see.

Figure 5. Here’s the DASH presentation playing in VLC.

Example 3: CMAF—One File per Ladder Rung (fMP4)

A CMAF presentation contains a single set of audio, video, and caption files that play on HLS and DASH and manifest files for HLS and DASH. Note that the output format must be fragmented MPEG-4 files for both stan­dards to work and can’t be transport streams.

Here’s the GPAC command:

MP4Box -dash 2000 jan_1080p.mp4 jan_720p.mp4 jan_540p.mp4 jan_audio.mp4 jan.vtt -out jan_CMF.mpd:dual

The new command does the following:

jan_CMAF.mpd:dual—Forces CMAF output with HLS and names manifest files JAN_CMF

Figure 6 contains all of the output files. The shared media files are all in fMP4 format, which won’t play on the already tiny and rapidly shrinking range of legacy HLS devices. The master HLS mani­fest is jan_CMF.m3u8, which links to the byte range requests in the other media manifest files. The sin­gle MPD file contains all DASH links.

Figure 6. CMAF output with a single set of content files and DASH and HLS manifest files

Again, you can create individual segmented fMP4 files for the content using the profile live switch. If you play either the m3u8 file or the MPD, you can load the captions file by clicking Subtitle > Sub Track and choosing the only available subtitle track.

Example 4: CMAF—One File per Ladder Rung With English/French Audio and Subtitles (fMP4)

Our last project is a valedictory exercise, just like the previous exercise, but I’ll add the French audio and subtitle track and will label both sets of subtitles and audio tracks:

MP4Box -dash 2000 jan_540p.mp4 jan_1080p.mp4 jan_720p.mp4 jan_audio.mp4:#udta_name= “English”:#HLSMExt=DEFAULT=YES Jan_audio_fr .mp4:#udta_name=“French” jan.vtt:#Language= en:#Representation=“English” jan_fr.vtt: #Language=fr:#Representation=“French” -out JanCMAF_ML.mpd:dual

Here are the new commands:

• jan_audio.mp4:#udta_name=“English”:#HLSM Ext=DEFAULT=YES—Identifies the audio file as English and makes it the default audio track
• Jan_audio_fr.mp4:#udta_name=“French”— Identifies the audio files as French
• jan.vtt:#Language=en:#Representation= “English”—Identifies and labels the subtitles as English
• jan_fr.vtt:#Language=fr:#Representation= “French”—Identifies and labels the subtitles as French

This produces the same files as shown in Figure 7, with one additional audio and subtitle track. Playing the HLS file in VLC enabled both audio tracks, as shown in Figure 7. The French track should be a rel­atively accurate translation, although I didn’t take the time to synchronize the spoken word to the video.

Figure 7. Properly labeled audio tracks

You can see the subtitle tracks in Figure 8, with French appearing over the video. This is playing the DASH MPD file; you will see a very similar, but not identical, presentation in HLS.

Figure 8. Rocking English and French subtitle tracks in VLC

Summing Up

I’ve found that packaging media for ABR distribution in FFmpeg can be challenging, frustrating, and some­times downright impossible. GPAC simplifies most common use cases into command strings that even a total novice could master in less than a few minutes.

How to Produce VVC With FFmpeg

Anytime you start working with a new codec, there are some fundamental tests that you should run to achieve optimal performance/quality optimization. In this article, I'll take you through those tests while encoding VVC using a version of FFmpeg that includes the Fraunhofer VVC codec.

21 Mar 2023

How to Script for FFmpeg Using PowerShell and BASH

FFmpeg was designed as a cross-platform solution for video and audio recording, conversion, and streaming using simple static command lines. Using variables and "for loops" in a command string simplifies the reuse of existing scripts and helps automate their operation. While you can't use these scripts in the Windows Command window, you can use Microsoft PowerShell in Windows and Bash on Linux and the Mac. In this tutorial, you'll learn how to create and run such scripts with PowerShell and Bash.

14 Mar 2023

How to Encode with FFmpeg 5.0

Rather than focusing on random tasks, this tutorial will walk you through the fundamentals of encoding with the latest version of FFmpeg.

27 Mar 2022

How to Produce CMAF Output and Testing Playback

CMAF has reached maturity. Here's one way to go about using it, with AWS Elemental MediaConvert

20 Mar 2020

What Is Adaptive Streaming?

A look at what adaptive streaming is, the primary technology providers, and the factors you should consider when choosing an adaptive streaming technology

28 Apr 2011