Requiem for the HMD

Four hard things need to happen before the HMD can replace mobile and desktop screens.

Current generation HMDs (The Oculus Quest, Rift, and HTC Vive) will at best enjoy modest consumer success as niche gaming devices, for specialized industrial training, and visualization applications that can match their high cost and low utility. Oh, and they will also be remembered in computer museums all over the world. But as a VR platform developer, that and $3.95 gets you a Caramel Frappuccino at Starbucks. 

Here is what needs to happen for them to be the revolution we are all waiting for:

Wearable All Day 

Photo Credit: All About Vision (left)

Most people can’t keep an HMD on for more than 15 minutes, that is if they can even handle wearing them at all. To succeed, next generation HMDs need to be superior to the screens we use today—TVs, computer monitors, mobile phones—that most people stare at for nearly half their day. To get there, the weight needs to drop to the same as ski goggles — about 300 grams. We know those can be worn all day. And with no ridiculous head strap. Can we build a social virtual world only for people with strong necks and short attention spans? No, we can’t. 

See-through Display

The Oculus is a social psychology test: It measures if you are willing to blindfold yourself and then walk around your room, while running into things and being watched by other people. Most people won’t do that, and most of the rest are only willing to do it once. 

You need to be able to see the world around you while you are using these things. You can’t trade your personal security for a new experience, no matter how compelling.  That means either a see-through/AR display, or (perhaps more likely) a high-resolution, low-latency pass-through camera. Varjo has allegedly built a good pass-through camera, but I haven’t tried it yet. The resolution of the pass-through camera will need to be at least 2K per eye, to make it possible to read a keyboard for typing (see below). Varjo’s camera is said to be 4K per eye. So we’re making progress here, but this is just one piece of the puzzle.

You Can Type at a Normal Speed 

And don’t give me any of that crap about how we are going to just use our voice. Try working all day just using voice commands. Do you think I would have written this article with my voice? Scotty was wrong. On a standard computer keyboard, the modern-day typist cranks out 52 words per minute (wpm). People who use both thumbs on their mobile phone keyboard type 38 wpm, and younger kids can type even faster than that. To get utility out of a head-mounted display, we also need to be able to use a mouse (or some kind of point and click interaction) and a keyboard. Hand controllers and VR keyboards are a far cry from being adequate. And frankly that couldn’t make me happier. If I never get to grab the thumb of a frustrated VR newbie  to help them find the X-button on the Touch Controllers—I’ll still be able to sleep at night. 

Think how the world could have been different if Palmer and Brendan had never touched a D-Pad. 

To be able to type at a normal speed, we will need both a high-resolution pass-through camera, as well as accurate finger tracking. This was demonstrated as a prototype at Oculus OC6, but I tried it and it isn’t good enough yet for typing or pretty much anything other than using your index finger with your palms facing away from you and your hands separated. Oh yeah, and as people start to pay attention to VR in social settings, with multiple users in one physical space unlike the solitary experience of today — that hand tracking will need to account for other people’s hands. Good luck. 

Same Quality as a Desktop Screen

What’s the point of typing if you can’t see what you’re writing? HMDs need to be just as sharp as traditional desktop displays, and you need to be able to read the content at the same distance you do today. Fortunately, the fact that the display is now mounted to your face means that the constantly moving pixels get averaged together into a sharper image by the feature detectors in your visual system — meaning that for most people, 3K per eye will look like a retina display on a laptop. A small startup called Vality recently demonstrated to me a 2K per eye display with non-uniform optics that provides 3K resolution at the center of view — now that’s desktop quality—at least to my eyes. It’s progress, but again, it’s just one necessary component.

When Will Version 2.0 Emerge?

It’s hard to say when these challenges will be solved and packaged into one elegant solution at mobile-phone price points. I’m clearly not the only person who thinks we’re in the thick of winter for consumer-grade VR headsets. In the meantime, I’ll keep rooting for the commercial-grade applications that show the world what’s possible and to funnel enough money into the industry with the hope that the next screen comes out in the not-too-distant future.

What are your thoughts? Leave your comments below.