5G Doesn't Matter for VR

News publications from Forbes to VentureBeat have boasted that 5G is going to be the future of VR. But a lot of people in the industry, including former Oculus CTO John Carmack, have some serious concerns about the impact 5G will actually have on VR. What these publications are boasting is definitely the dream for VR – to use cheaper, slower, less battery-intensive hardware in standalone headsets and handle all the processing for the headsets on a remote server. We could have higher-quality graphics, way better battery life, and a much more immersive experience for a cheaper price. Who wouldn’t want that! But is 5G really the technology that is going to change this for VR?

Speed

The biggest selling point of 5G for VR is the theoretical speed being around 10Gbps – making it about 100x faster than 4G/LTE^. 5G marketing materials promote the “100x faster” argument but conveniently ignore many of the realities of the difference between 5G and 4G. 

In particular, it ignores the more recent 4G specification of LTE-A, which has a theoretical speed of 1Gbps^. 5G only being 10x faster than 4G really doesn’t have the same ring to it, does it? Also, this is the speed for only mmWave 5G. Mid-band 5G has a theoretical speed of 5Gbps, and low-band 5G has a theoretical speed as low as 300Mbps.

Of course, there is also the difference between theoretical speed, hardware-supported speed, and real-world speed. LTE-A has a theoretical speed of 1Gbps, a current hardware-supported speed of 300Mbps, and a typical real-world speed of only 42Mbps^. While this specification has existed for 8 years, it is nowhere remotely close to reaching its theoretical speed.

Assuming this is a perfect world and we could hit even 1/10th of the theoretical speed of 5G and have a 1Gbps connection, we then have to talk about the bandwidth requirements. Some countries, such as Canada where I live, have incredibly low data limits. “Unlimited” plans only became available in the last few years and even those have a limited speed after 5GB of data usage. If we really want to use intensive VR hardware over 5G, these data caps must be removed. Home internet providers faced this issue when video streaming first took off. And even today, it continues to impact those with rural internet connections powered by satellite connections. Without a change in how data caps are applied to the mobile internet, it’s going to be very hard for VR over 5G to truly take off.

Latency

The latency of what you’re viewing in VR is one of the most important factors in an enjoyable VR experience. With too high of a latency, you suffer some serious motion sickness, which absolutely ruins any amount of immersion in your experience.

Currently, the launch hardware for 5G sees a latency of 8 to 12ms just from headset to tower^. Verizon reports their early deployments have a latency of 30ms^. This doesn’t include a roundtrip connection to wherever the processing server for the headset is located or the time it takes that server to render whatever you’re going to view. 

Of course, other things can be done to compensate for this latency. Google’s Project Stadia is attempting to do some incredible things with AI to combat latency with video game streaming. The way this AI works is that it’s trying to predict inputs with a keyboard and mouse or gamepad before they actually happen. Making AI predictions with a video game keyboard, which only has a few keys, and a mouse, which only has an X-axis, Y-axis, and velocity to predict VR input, is an entirely different story. Not only do you need to predict the controller inputs, but you also must predict each part of a human body moving in 3D space.

Until the latency for 5G is reduced to be much closer to the targeted 2ms of the 5G specifications, it’s going to be a nausea-inducing experience to use a VR headset over current 5G connections.

Headset manufacturers have been working very hard to reduce these latencies to as low as possible without even factoring in network latency. The general term used for VR is “Motion to Photon Latency” – the latency between a movement in VR and when that can be displayed. This has involved working on reducing the display’s pixel switching time, meaning how fast a display can switch what is showing for a pixel. With the amount of work that has gone into making local hardware work well for VR without even factoring in a network connection, it really shows how far-fetched 5G is for VR.

Coverage

Part of what makes mmWave 5G special is the higher frequencies it uses, but this comes at a huge cost. A 5G base station has so much less range than a 4G base station that it’s estimated it will require three times as many base stations^ to cover the same area. Now, of course, this type of reduced coverage means that it’s never really going to be available in rural areas – a fact that T-Mobile’s CTO confirmed^. Can you really picture a world where a VR headset is going to be sold that can only be used if you have reliable 5G access? This would cut out a huge percentage of the world and result in many upset people who purchase one of these headsets only to set it up at home and find out it won’t work.

Now, this may make it seem like it’s going to work great for those who live in a major city. But the other side effect of requiring higher frequencies is that its ability to penetrate materials decreases substantially^. This means that if you live in a large concrete apartment building – like most modern apartment buildings – chances are that you won’t have much or any coverage inside your own apartment except for right by your window.

One of the ways that 5G gets such high theoretical speeds is through a technology called multiplexing. At its most basic, this simply means sending data over multiple concurrent connections rather than just one. This is the same technology used by Wifi 802.11n, newer Wifi specifications, and current wireless headsets like the HTC Vive Wireless Adapter. One of the most important parts of the design of multiplexing is the ability for it to dynamically scale the number of concurrent connections depending on how stable the connection is between you and the access point. This is why the HTC Vive Wireless Adapter requires line of sight. The moment you put a wall in the way, it degrades the quality of the connection, and you can no longer maintain such a high connection speed. 5G is going to be affected by this exact issue – but do we really want our experiences in VR dynamically degrading as our connections get worse? Any change in the quality of a VR experience while you are inside of a headset is going to cause a lot of discomfort.

But how could it work?

Ultimately, I believe 5G will contribute to VR but we will have to wait and see what that contribution looks like. At the 2019 VRX Conference in San Francisco, I heard about an implementation where edge servers were onsite and used 5G for a backpack-less free-roam experience. Definitely an effective use of 5G for VR, but also something that could be handled just as well with Wifi 6.

Additional research is the leading contender in making 5G work well for VR. It’s what has taken VR this far and made the headsets we have today possible. Ultraleap has spent years developing hand-tracking technology, which has helped pave the way for the Oculus Quest to offer hand tracking in a standalone headset. If you’ve ever played a game at SandboxVR and experienced their absolutely atrocious hand tracking, despite having hardware Velcro-ed to your hands and powerful servers onsite to process it, then you understand how impressive this feat is.

Using 5G for just part of a VR experience is also a possibility. For example, keep the main effort of VR interaction local to the headset, but let an edge server handle partial rendering of a scene or specific complex models in a scene. This may not revolutionize VR in the home, but it will improve the quality of experiences on future standalone headsets.

What else is 5G good for?

I definitely don’t buy into all the hype surrounding 5G. I haven’t read a single use case that I actually believe will be an industry changer. But there is still lots to look forward to with 5G!

It’s nice to be getting faster mobile internet! Faster internet to our mobile devices will continue to increase our quality of life, especially in major cities. I’m writing this article on an iPad with built-in LTE right now – I hope we see a future where 5G chips are more readily available on laptops for working while traveling. 

Expect your devices to switch from 5G to 4G connections when you move from outdoors to indoors. It’s definitely an improvement over LTE-A, but it’s by no means the giant leap forward all the marketing materials claim. It’s definitely not going to cause what Intel claims^ – “a new dawn of VR-driven experiences will emerge as early as 2025.”

What will the future of VR really be? There’s no doubt in my mind it involves getting rid of all the wires. Every time I use my Oculus Quest or do a free-roam experience, it makes it harder for me to go back to using a tethered headset. With Qualcomm’s new XR2 specification, we’ll see standalone headsets slightly more powerful than the Oculus Quest, but don’t expect a giant difference, just a marginal increase. With desktop headsets, I’m hoping we’ll see new wireless headsets based on the Wifi 6 specification, and I’d love to have a wireless version of the Valve Index or HP Reverb.

I’m excited to see what creative applications people will come up with for 5G in 2020 and beyond!