NASA Uses Kinect and Oculus Rift to Control Robot

Kinect sensors allow the user (right) to control the robot hand (left), which he sees in the Oculus Rift goggles.Kinect sensors allow the user (right) to control the robot hand (left), which he sees in the Oculus Rift goggles.We haven't put a person on Mars yet, but we are getting closer to doing it virtually. NASA's Jet Propulsion Laboratory (JPL) has built a rig that allows an operator to use the Oculus Rift virtual reality headset to see through a robot's image sensors and the Kinect 2 from Xbox One to control the robot's arm.

The Kinect tracks the user's movement, specifically how the arm moves, rotates and clenches. The robot arm follows, though with a slight lag due to the interface and the latency involved with operating a robot so far away. (The video shows a lag of a few seconds, but between Earth and Mars the radio delay is several minutes.) 

MORE: What is the Oculus Rift? 

The interface shows how the person's hand is moving and how the robot arm is responding,  keeping the user updated on the lag's effect in real time. The Oculus Rift headset allows the user to virtually see the environment around the robot, in 3D, for better understanding of the situation.

NASA's Jet Propulsion Laboratory website shows how the agency could use such tech? Imagine a new Mars Rover in which the user controls the  robot, its cameras and wheels and arms, through motion control. And with the Rift, the user can actually perceive the environment in 3D. It would be the closest thing to putting someone on Mars without actually making the trip. JPL has also shown that this telepresence technology could be used to better control the Robonaut 2 robot on the International Space Station.

NASA's Robotic Controller with Kinect and Oculus Rift

Though the Kinect 2 for Windows won't be released until summer 2014, JPL has used an Xbox One developer kit to utilize the latest version of the motion-control sensor (a packages costing about $500). Add in the Oculus Rift developer version for $300, and the multibillion-dollar space agency potentially has a telepresence solution for less than a grand. 

This isn't JPL's first experiment with these technologies. The lab has used Oculus Rift to allow a person to explore the Mars terrain around the Mars Rover, compiled via photographs taken by the Rover. JPL has also used the Rift with the Virtuix Omni treadmill to allow a person to walk around a virtual Mars. All of these technologies together could someday lead to a complete telepresence on other planets.

Follow Kevin Ohannessian at @khohannessian and on Google+. Follow us @tomsguide, on Facebook and on Google+.

Create a new thread in the Streaming Video & TVs forum about this subject
This thread is closed for comments
1 comment
Comment from the forums