360° VR Live-Streaming Workflow
To help VR content creators contend with the deluge of information and product announcements, this article will focus on VR streaming workflows and toolkits that have worked well together in various VR/360 productions I've done.
Although streaming VR from your encoder to your CDN over RTMP, MPEG-DASH, or HLS is identical to that of standard live-streaming workflows, VR video—especially at 4K—requires large amounts of bandwidth, especially when streaming at higher frame rates at around 20+Mbps with the H.264 codec. Once HEVC/H.265 is widely adopted with increased hardware support on mobile devices and desktops, it will bring much more efficiency with lower overhead, thereby allowing for better-quality streams.
You can optimize your upstream by using HEVC encoding as a mezzanine encode. HEVC is roughly twice as efficient as H.264. Generally, a 4K stream in H.264 might need 12–15Mbps of bandwidth. With HEVC, you can stream video with the same quality at around 6–8Mbps.
Something else to consider is using Open Broadcaster Software. OBS is a powerful, versatile, and open source switching, encoding, and recording solution for live streaming for Windows, Mac, or Linux. I recommend using OBS in conjunction with a high-end PC as an alternative to “Big Iron” encoders, since the software is free and feature-rich. It has served us well on our productions, and can be a critical component for VR streaming events. If OBS isn’t your primary encoder, I suggest having it as a backup encoder.
Case 1: Single-Rig Live VR Streaming With Vahana VR or Teradek Sphere
Figure 6 (below) maps out a single-camera VR workflow I assembled through trial and error to use in relatively simple VR productions.
Figure 6. A single-rig live VR streaming setup using Vahana VR or Teradek Sphere
At the heart of this setup is a VR camera rig with HDMI/HD-SDI outputs and a workstation PC for live stitching and encoding.
If you’re planning on building your VR workflow around Vahana VR, here are a few things to note:
• For Vahana VR, software stitching is sufficient, but for better stitching results, consider importing a PTGui template.
• Vahana VR can be buggy at times, so I highly recommend using a separate encoder.
• As of August 2017, Vahana VR is the only software available that offers live-view with Oculus headsets.
Case 2: Z Cam S1-Based Complete IP Workflow
The setup diagrammed in Case 2 (Figure 7, below) shows a complete IP workflow achieved with a Z Cam S1. To assemble this kit, you need only one Ethernet cable between the Z Cam S1 and a WonderLive installed machine. The Z Cam controller, WonderStitch, and WonderLive used in the Case 1B setup, are sold as bundled software. The caveat is that one unique license is required per camera.
Figure 7. A Z Cam S1-based complete IP workflow setup for VR/360 streaming
As in Case 1, software stitching is sufficient for baseline use, but for better stitching results, import a PTGui template.
Streaming Platforms
For 360° livestreaming to the general public, I recommend encoding directly to platforms such as Facebook or YouTube. You can also stream to other CDNs or services such as Bitmovin, Wowza, or Visbit when you need more robust or customized solutions. Overall, using Facebook or YouTube is great for standard 4K, since they are both easy to configure and provide comprehensive dashboards to work with.
YouTube has continuously refined its live platform and is very straightforward in its approach to panoramic live streaming. In our projects, it has proven the most consistent and stable platform while at the same time delivering good, high-resolution 4K streams.
YouTube supports VR streaming at up to 4K @ 60 fps, but depending on your bandwidth, I suggest using more conservative specs, such as 2560x1440 at 30 fps, uploaded at 6–10Mbps; or for 3840x2160 at 30 fps, with an upload bitrate of 13–20Mbps.
Facebook, with its Live 360 video platform, is making major investments in its ecosystem and community to bring together its content creators and consumers. Facebook now features live-streamed 360° video up to 4K resolution. The streams are viewable in 4K/360° on the Samsung Gear VR. Facebook intends to offer 4K/360° live video for Oculus Rift in near the future.
You can find upload specs and recommendations for Facebook Live 360 here.
Live Preview
Being able to see a live preview on-set allows the director, clients, or production crew members to get an instant visualization of what is being captured from the stitched camera view. The Teradek Sphere allows for live preview and live stitching of panoramic video view while on-set. Having this feature enhances overall production value and allows you to troubleshoot possible stitching errors that diminish the quality of the stream.
Vahana VR software also provides a live preview in real time. It has an equirectangular preview mode for viewing the full panoramic video on a connected display, as well as interactive mode for first-person view in a VR headset.
Developing Your Own VR/360 Streaming Workflow
As mentioned previously, there are many choices in VR production, and every company or individual producer will have their own methodologies in accomplishing a goal. What I have presented are general workflows that have served as a basis for my company in 360° streaming live events including fashion shows, music concerts, corporate presentations, and product launches. Hopefully this will serve as a template that you can build on and expand based on your own or your client’s specific needs.
360° live streaming is still in its infancy and has yet to reach its full potential. When content improves and captures the hearts and imaginations of its viewers, only then will it be an easier sell to a broader swath of consumers and make inroads into the mainstream. But until then, it will remain a niche market, albeit one with great potential.
Related Articles
Scott Mayerowitz, Digital Storytelling Editor at the Associated Press, breaks down the differences between "VR/Virtual Reality" and "360," two terms often used interchangeably.
AP Digital Storytelling Editor Seth Mayerowitz discusses the pros and cons of different 360 VR camera solutions in terms of features and price points.
VR remains a niche market at this writing, but it's a growing one with huge potential. Here are the latest developments and what it means to adoption in the live production and streaming world.
The cameras have poor controls, the headsets are bulky, and the results sometimes make people sick. Still, don't write off virtual reality just yet.