Video: How to Reduce Latency for Mobile VR Streaming
Watch Satender Saroha's full presentation from Streaming Media East, Challenges & Opportunities in Delivering 360 VR/3D Experiences at Yahoo, on the Streaming Media Conference Video Portal.
Read the complete transcript of this clip:
Satender Saroha: Building the mobile VR experience, the biggest challenge is to do true motion-to-photon latency on mobile. the latency between the physical movement of a user's head and the time it takes for the updated photons from your head-mounted display to reach your eyes is called as motion-to-photon latency. ideally, this should be less than 30 milliseconds. True devices like Oculus Rift, Vive, they already provided in less than 35 milliseconds, but all the mobile devices, especially the RED VR, have a latency of approximately 90 milliseconds.
How do we solve this? What is the real challenge there? Traditionally, on mobile devices, rendering is double-buffered, which essentially means two buffers, stored in GPU memory. One is currently being scanned out, so there are two steps, rendering and scanning out. What you see on the display is what is getting scanned out. That scanned-out display is called “front buffer” and one that is being rendered to is the “back buffer.”
The GPU will never render the same buffer that’s being scanned out, so this has the advantage to prevent the artifacts, but at the same time in VR, because we are trying to show as soon as the user moves his head, the side effect is this leads to increase latency. An alternative approach is to, what we call a single rendering is to render directly to the front buffer, but time things out so carefully that you have rendered each line of the image just shortly before the scan out is going out of the display. So here we use a technique which provides scan-line racing. Essentially, scan line racing is a process of knowing where the scan line is in the screen and just rendering before that not overshooting that to avoid the artifacts.
That solves the problem of motion-to-photon latency by going for single rendering on top of scan line racing. And we have used or using the data in VRPAPK for the VR experience.
RealEyes' David Hassoun discusses what low latency is and what it isn't, and sets reasonable expectations for the current content delivery climate.
Limelight's Charlie Kraus discusses three emerging strategies for delivering low-latency live streaming in the post-Flash era.
Streaming Video Alliance's Jason Thibeault and Limelight's Charley Thomas address the question of whether WebRTC provides a viable solution for network latency issues in this panel from Live Streaming Summit.
Nokia's Devon Copley discusses the challenges throughout the pipeline for streaming 6K or 8K VR/360, and how viewport-adaptive streaming provides an interim solution.
StackPath's Nathan Moore explains the protocols, latency, and bandwidth challenges inherent to delivering video content to iOS devices and how content providers can stream to these devices more effectively.
Wowza Senior Product Manager Jamie Sherry discusses key latency considerations and ways to address them at every stage in the content creation and delivery workflow.
Wowza's Mike Talvensaari confronts the myth that low latency for large-scale streaming is always worth the expense, and discusses various applications and use cases along a continuum of latency requirements for effective delivery.
Reel Solver's Tim Siglin, Rainbow Broadband's Russ Ham, and Verizon's Daniel Sanders discuss how attacks on Net Neutrality would impact video delivery in general and latency in particular.
Ooyala's Paula Minardi and Level 3's Jon Alexander discuss the key issues facing live streaming and VOD providers regarding latency times, buffering, and meeting evolving viewer expectations.