“Seeing is believing.”
While there may be some truth in the old adage, it forgets a crucial component of how we interact with the world: touch.
Touch makes things real. Imagine reaching out to grab your coffee mug, and instead of feeling the smooth ceramic surface pressing against your fingertips, you get…nothing. It’s jarring.
Virtual reality has a touch problem.
The release of Oculus Rift and Gear VR to consumers earlier this year clearly shows us graphics and sound are no longer the bottleneck in an immersive experience. The issue — a big one — is how to interact with the world we’re seeing on screen, with all the dexterity our fingers allow in the real world.
According to Dexta Robotics, a US-registered startup operating in China, the solution is an exoskeleton for your hands: a sleek, gorgeous mechanical glove that captures the full range of hand motions and provides real-time tactile feedback to let you touch and feel the digital world.
“It’s the difference between gazing at the moon and setting foot on it for the first time,” said Aler Gu, CEO of Dexta Robotics, in a press release. The glove, Dexmo, “is a leap forward that will inspire worlds of new content and new experiences,” he said.
Dexta Robotics is far from the only company offering a solution to the VR interaction problem.
The half-moon shaped Oculus Touch, for example, curls the hand naturally in a grasping motion around the trackable accessory. Vibrations provide tactile feedback, and users can grasp virtual objects by pressing buttons on the controller.
While easy to use, the Touch isn’t exactly an ideal extension of our hands — the feedback doesn’t match up with what we normally expect, and button pressing is a sad substitution for the myriad of ways our fingers can reach out and explore.
“Vibration alone isn’t enough to fool the brain,” Gu told MIT Technology Review. “The moment you detect anomalies in how objects feel, your sense of immersion is broken.”
Another approach is to track the user’s eye or finger movements and use those inputs to directly control the digital space. It makes intuitive sense: after all, we’re always looking at or touching something, and VR headsets are already optimally placed in front of our eyes.
Companies like Eyefluence, Leap Motion and Gest are all betting on this approach.
Despite initial excitement, without tactile feedback this method falls solidly into the uncanny valley of motion control — sure, the interaction looks strikingly real to your eyes, but your body definitively knows you’re grasping at nothing but air.
Dexmo is inherently different.
Shaped like a pterodactyl claw, the wearable tracks the full range of hand motions — up to an impressive eleven degrees of freedom — as you reach, touch, grasp and feel objects in the virtual world. The glove wirelessly sends the data to a VR avatar, prompting the avatar to mimic your movements.
Say you’re reaching for a digital rubber ducky. As soon as the avatar’s fingertips touch the toy, Dexmo’s software computes the direction and amplitude of the force applied to each individual fingertip. This information is passed on to the five custom-built force-feedback units embedded in the glove, which then dynamically apply a counterforce to the user’s hand. This way, you can feel the virtual object “push back” against your fingers, just like a real one would in our physical world.
Dexmo’s variable force feedback system allows each motor to simulate the force coming from different points on the virtual object, thus translating more subtle physical properties. This way, the motors may lightly press against your fingertips when you pick up a soft slice of cake or offer heavy resistance to your palm when you grab a brick. With Dexmo, you can feel the squishiness of a rubber ducky, the solidity of a big rock, or the jagged edges of a table leg, all inside the virtual world.
And the forces are strong with this one: the glove’s powerful resistance prevents your fingers from passing through an object like a phantom, adding to the quality of immersion.
Dexmo also boosts user-friendly specs to round out the experience.
The exoskeleton weighs around 170 grams, just a tad more than an iPhone 6 Plus, but with the weight distributed across each hand. It runs on battery power, and a full charge is estimated to support over four hours of game play. It uses NRF for wireless communication, averaging between 25 and 50ms of overall latency.
“Wifi and Bluetooth [were] too slow for our applications,” explains Gu.
Since conception, Dexmo has already gone through over 20 iterations, and initial user testing with prototypes has been overwhelmingly positive.
“We conducted many studies where test subjects performed tasks using Vive controller or Dexmo. Tasks such as turning a knob, grasping an odd-shaped object, playing piano, pressing buttons, and throwing a ball. As expected, no contest! Users preferred Dexmo, and reportedly enjoyed a much higher level of immersion,” says Gu.
The utility of Dexmo goes far beyond the gamer crowd.
The company envisions trainee surgeons using the glove to learn complex surgical procedures in VR while receiving realistic feedback. The system could also be developed for military clients or bomb disposal experts, giving trainees the opportunity to feel and tinker with the intricate mechanical parts in their hands in a safe setting. It would also cut down on cost by reducing the need for procuring actual parts for training or otherwise.
Dexmo could even be used to control physical robots or robotic arms and allow the user to gain a better sense of presence of the robot’s movement in space.
According to Gu, the company is working with VR firms to develop and deliver the “best immersive experience possible.” Currently, the glove works in any 3D simulated environment and is compatible with the majority of VR headsets, including Oculus Rift, HTC Vive and Hololens, to name a few. The company is also planning to release a devkit for developers.
Dexta has not yet set a release date or price for the futuristic controller, although Gu believes that it’ll be something that “eventually everybody should be able to afford.”
Haptics interaction in VR is only just beginning to take off.
Whether we’ll finally be able to get a satisfying grip in VR with Dexmo remains to be seen. The glove likely represents the first generation of VR controllers that bring real-world tactile sensations into the digital universe.
The end goal is to reach across the digital divide, says Gu.
After all, “there’s no better feeling than feeling.”
Image credit: Dexta Robotics/YouTube
