Kinect sensors allow the user (right) to control the robot hand (left), which he sees in the Oculus Rift goggles.We haven't put a person on Mars yet, but we are getting closer to doing it virtually. NASA's Jet Propulsion Laboratory (JPL) has built a rig that allows an operator to use the Oculus Rift virtual reality headset to see through a robot's image sensors and the Kinect 2 from Xbox One to control the robot's arm.
The Kinect tracks the user's movement, specifically how the arm moves, rotates and clenches. The robot arm follows, though with a slight lag due to the interface and the latency involved with operating a robot so far away. (The video shows a lag of a few seconds, but between Earth and Mars the radio delay is several minutes.)
MORE: What is the Oculus Rift?
The interface shows how the person's hand is moving and how the robot arm is responding, keeping the user updated on the lag's effect in real time. The Oculus Rift headset allows the user to virtually see the environment around the robot, in 3D, for better understanding of the situation.
NASA's Jet Propulsion Laboratory website shows how the agency could use such tech? Imagine a new Mars Rover in which the user controls the robot, its cameras and wheels and arms, through motion control. And with the Rift, the user can actually perceive the environment in 3D. It would be the closest thing to putting someone on Mars without actually making the trip. JPL has also shown that this telepresence technology could be used to better control the Robonaut 2 robot on the International Space Station.
Though the Kinect 2 for Windows won't be released until summer 2014, JPL has used an Xbox One developer kit to utilize the latest version of the motion-control sensor (a packages costing about $500). Add in the Oculus Rift developer version for $300, and the multibillion-dollar space agency potentially has a telepresence solution for less than a grand.
This isn't JPL's first experiment with these technologies. The lab has used Oculus Rift to allow a person to explore the Mars terrain around the Mars Rover, compiled via photographs taken by the Rover. JPL has also used the Rift with the Virtuix Omni treadmill to allow a person to walk around a virtual Mars. All of these technologies together could someday lead to a complete telepresence on other planets.