Beyond the Touch Keyboard
Gestures are going to become increasingly important on phones. On-screen multi-touch gestures are just the beginning. Bill Pinnell, user interface, graphics and mobile gaming principle product manager at phone operating system company Symbian, predicts the advent of “3D spatialized gestures that are less about looking at device and interacting with it and more about using the device in your hand and interacting with the environment.” He talks about combining touch and multi-touch with shaking, rotating or flipping the phone.
When Series 60 gets HDMI support so you can connect your phone to an HD display, he imagines using it to play a game, control the volume or paint. “How you hold the device would control the strokes of ink on the virtual canvas," Pinnell said. "Think about moving photos around or grabbing things by pinching towards the screen — you’ll have items you can flick around, float around or push and prod.” And you’ll also want haptic feedback to know when you’ve selected an item or the phone has detected a gesture.
All of these functions are due in Symbian 9.5, possibly launching next year, which will have an extensible sensor framework that can integrate accelerometers, compasses and other sensors. There are more advanced vibrotactile haptics coming from Immersion and other companies. Phones with multiple actuators could give you directions with feedback dictating which way you should turn your phone for various effects. Isolating the screen and attaching actuators to it would give feedback through the screen itself. Put it all together with the processing power of the GPUs that Pinnell believes will be in nearly all high-end mobile phones in 2009 and 2010, and 3D gestures and haptics could combine to create an immersive user interface that’s far beyond a virtual keyboard.