Skip to main content

Your Eyes Could Be the Driving Force Behind VR's Future

BARCELONA, Spain — Creating a virtual world that exists only inside a headset requires a lot of processing power. And that's why virtual reality headsets like the HTC Vive and Oculus Rift require PCs with steep specs and price tags to match.

But what if every graphical detail didn't have to be rendered in full resolution? That's the point SensoMotoric Instruments (SMI) hopes to make by combining eye-tracking technology with foveated rendering to concentrate rendering resources on where a user's eyes are focusing. The result is a virtual reality experience that requires less processing power.

With foveated rendering, only the areas where you're staring are fully rendered; other parts of the screen appear at lower-resolution. SMI's challenge is to make sure that as your eyes flick around the screen, the fully-rendered areas follow.

MORE: What's Next for VR: More Useful Content, Less Nausea

To illustrate how foveated rendering and eye tracking can work together in VR headsets, SMI rigged up an Oculus Rift with its eye-tracking technology for a demo at Mobile World Congress. Strapping on the headset, I took a virtual tour around a Mediterranean-style villa, darting my eyes around the screen. Two circles accompanied on my journey: a small red circle surrounded by a wider blue one to show the vicinity of where I was looking. The graphics inside the red circle were fully rendered while everything in the blue circle appeared at 60 percent resolution. Anything outside that circle was rendered at 20 percent.

The result? I didn't notice any change in graphics quality with SMI's technology turned on, mostly because the area I was looking remained fully rendered. And SMI's eye tracking technology was able to follow along where I was staring without any lag.

More importantly, a monitor next to me was recording just how much processing power was required when I was using SMI's technology. The GPU performance measured around 65 to 70 percent without foveated rendering in place. Once SMI turned it on, though, the GPU's output fell by 50 percent. In other words, not having to fully render all of the graphics took less of a performance hit on the PC's graphics processor so, in theory, a VR headset that incorporated SMI's technology wouldn't need the hefty hardware requirements that headsets such as the HTC Vive and Oculus Rift require.

SMI offered another demo this week in Barcelona featuring a Gear VR equipped with eye-tracking technology, which I used to play a whack-a-mole-style game and another game that involved flipping boxes into the air and keeping them aloft. I had a handheld controller to bring down a mallet on those unfortunate moles or to sending those boxes flying, but my eyes controlled where the target would be. Again, I was able to easily control both games just with my eyes with no noticeable lag in performance.

SMI has no interest in making a VR headset of its own. Rather, the company's hoping to license its technology for the next generation of headsets. How much it would cost to build SMI's technology into a headset would depend on how widely it was adopted, SMI spokesman Tim Stott told me. "If our tech ends up in the big-brand [head-mounted displays], then we are talking just a few dollars," he said.