Robot see, robot do.

2010 may go down in history as the year of gesture recognition. We’ve seen it in TVs, we have it in our video games (thanks, Kinect!), and now we have it in our robots. The Biological Cybernetics Lab at Tsukuba University, headed by Kiyoshi Hoshino, recently demonstrated a robotic arm that can mimic the position and movements of your own. Using two cameras, the system tracks your hand and arm more than 100 times per second and relays the information to the robot so that it can repeat what you do. The system is fast enough that there is relatively little lag time between your gesture and the robot’s copied motion. Hoshino and his students have pushed the system even further and taught it how to recognize 100 distinct hand shapes. This allows the robot arm to not only track the location and movement of your arm, but to reliably perform the same actions like picking up an object or waving hello. The robot arm was demonstrated at the recent 3DExpo in Yokohama, and you can see it in action in the video below. This sort of intuitive interface could let almost anyone control and program a robot. Hooray for the democratization of technology!

The Hoshino Lab should be commended for its cool approach to robotic controls, but I’m not sure how impressed we should be with their camera setup. I think recent work with the Xbox Kinect sensor shows that you can have some amazing 3D capture for just $150, and one or two of those sensors are probably going to be able to give you a much better idea of hand shapes. Considering how much open source code is being generated for the Kinect it seems like it should be easy to get such a system up and running fairly quickly.

Also, while I’m totally in favor of gesture controlled robots, the video above demonstrates that there are some serious limitations to the technology. First, the lag-time, while pretty small, isn’t negligible. The human controller is obviously having to wait and see before he makes his next move in order to properly pick up the stuffed animals. Second, the system has no tactile feedback to the user, and that’s a problem for me. Without some form of haptics anyone controlling a robot through a gesture system is going to have to be absurdly vigilant or they will end up crushing everything they grab. I think there’s a reason why Hoshino uses soft stuffed animals in the demonstration, and it’s not because cuteness sells.

Even with these limitations, I think that the Hoshino Lab shows us that non-traditional controls for robotics could arrive much sooner than we think. They’re going to be needed. Automation is creeping into all levels of manufacturing, and if small businesses want to keep up with larger corporations they are going to need their own kind of industrial robots. These low-level industrial bots won’t have owners that are engineers or experts in computer code. They will require a simple interface that allows them to teach a robot how to perform a complex task. The gesture guided robot arm from Tsukuba University is a good example of that kind of simplified intuitive programming technology. I hear that Rodney Brooks and his new startup Heartland Robotics are pursuing a somewhat similar system. With the work of these teams, and many others around the world, we may develop a personal robot who learns by watching humans. Anything you can do, it will be able to do.

Only better.

[screen capture and video credit: DigInfo News]
[sources: DigInfo News video, Biological Cybernetics Lab]

Hidden magnetic forces let a virtual lizard lick the pen right out of your hand.

Japanese electronics firm NTT DoCoMo unveiled a cool new way to interact with the objects on your portable screen at the CEATEC conference this past week. The unnamed prototype device combines glassless 3D display with a physical feedback system that pushes on a magnetic pen. You hold the stylus pen near the display screen and when virtual objects want to reach out and touch you, you can feel it through forces on the pen. Haptic technology is already making its way into mobile phones with simple vibration responses. This new magnetic approach is another step towards revolutionizing how you control your phone, interact with digital information, or even play video games. Check out the NTT DoCoMo reps demonstrating the technology at CEATEC in the video below. A virtual lizard nearly licks the pen in someone’s hand to make it jump. I love it.

Many of the 3D technologies we’ve seen in the past few years require viewers to wear glasses of some kind to get the 3D visual effect. NTT DoCoMo developed an alternate approach several years ago that uses virtual parallax – showing slightly different images depending on the angle from which you are viewing the screen. This produces a 3D effect without glasses to those who are viewing the display in person. In the video below, however, the display looks a little fuzzy as the camera is not able to pick up a stereoscopic view of the image from different angles:

NTT DoCoMo’s prototype display doesn’t just take an unusual approach to 3D, it also treats haptics in a new way. At the top of the device is a set of stereoscopic cameras that track the pen’s position. Behind the 3D display is an inductive coil that can be pumped with electricity to create a magnetic field. The magnet buried in the stylus pen’s tip is pushed by the coil when virtual objects need to interact with the user. As you can clearly see in the video below, the induction coil produces a pretty solid kick when powered up. It nearly knocks the pen out of the woman’s hand, even after she expects it.

The prototype is strong then, but not terribly precise. That inductive push doesn’t seem to have much (any?) directional control. To compensate, NTT DoCoMo may want to combine their magnetic approach with other haptics. Vibrations for mild touches, solenoids and inductive coils for hard pushes, and heat/cold for those sensations. We’ve already seen others pursue combined haptics for video games. For now, much of what you’ll see on the market is going to be very simple. Most of the previous haptic technologies we’ve seen rely on vibrations, rather than actual pushing forces, to convey a sense of physicality to virtual objects. Vibrating controllers have been part of the video game scene since the late 90s, and vibrating phones are ubiquitous. These not so exciting forms of haptics are where we’re at, and where we’ll likely stay until developers like NTT DoCoMo can come up with useful alternatives that are also cheap  and small enough to install in everything.

This poster at CEATEC explains how the prototype's 3D display and haptic feedback work.

It’s hard to know exactly what near term applications NTT DoCoMo will find for their mobile 3D haptic display. According to IDG News, representatives from the company at CEATEC stated that there were no definitive plans to develop the prototype into a marketable device. That’s probably a good call. It’s impressive that NTT DoCoMo can fit ‘glassless’ 3D display and haptics into a small mobile device, but these technologies probably aren’t really ready for primetime. They should wait until after the 3D craze hits mainstream television, then portable versions will be more in demand.   All in all, the protoype from NTT DoCoMo is very cool looking, but not something that we need at the moment. Give it a few years however, and similar systems will help us all poke and play with virtual creatures on our mobile phones. There’s a new Tamagotchi fad in our future, I can just feel it.

[image credit: EnGadget]
[screen capture and video credits: Network World/IDG News]
[sources: PC World/IDG News]

LEFT: Bad robot, no crushing! RIGHT: That's it, treat the Sunkist with respect.

If Sam Waterson taught me anything it’s that I need to fear the powerful grip of robots. Because robots are made of metal…and they’re strong. Thankfully Silcon Valley robotics startup Willow Garage has worked to relieve my anxiety by teaching their PR2 bot how not to crush things – including humans. This summer UPenn grad student Joe Romano instructed the PR2 how to use tactile feedback (aka haptics) to hold delicate objects. Sensors in the grippers allow the robot to easily pick up and place objects, or hand them off to a human. Other researchers can freely incorporate Romano’s work because, as with all Willow Garage code, it’s made available via the open source Robot Operating System library (ROS). The PR2 already had some safety features on its movements that kept it from banging into people, but the new haptic feedback is a big improvement. Making sure a robot won’t crush things is a necessity if you want to sell it to researchers all over the world, as Willow Garage is aiming to do. Catch the video below to see Romano showcase the PR2′s human friendly grip. Make sure you watch until the end to catch a great series of high five shenanigans. Double fist pound into explosion – Ha! Priceless.

[screen capture and video credit: Willow Garage]
[source: Willow Garage Blog]

The Tactile Gaming Vest uses solenoids and vibrating motors to let you feel bullet impacts or knife slashes. It doesn't sound pleasant, but trust us, gamers will love it.

If enormous flat screen TVs and surround sound aren’t enough to satisfy your hunger for immersive gaming then maybe you need to be shot. Virtually, that is. Saurabh Palan from the Haptics Lab at the University of Pennsylvania has developed the Tactile Gaming Vest (TGV) a device that lets you feel the impact of bullets, and the slashing sensation of knives as you play action packed video games. The TGV is mounted with solenoids that strike you (for bullets) and vibrating motors (for the knife slashing sensation) that respond to cues from the game and let you feel what your character feels on the screen. Palan hopes to give users a “4-D” experience, with tactile feedback providing the extra dimension of immersion. The TGV debuted at the recent 2010 Haptics Symposium to much fanfare and positive reviews. While this device is still in the early prototype phase, haptic vests may one day be adapted to provide real sensations for military training, movies, and yes, amazing video games.

The TGV isn’t the first vest with physical feedback. In fact, you can go online now and order one from TN Games for $140. That system, however, uses compressed air, making its response slower and less precise than the TGV. Most other haptic vests have relied on just one kind of feedback. Palan has incorporated both impacts and vibrations, and plans on including more. The original TGV prototype (which was tested at UPenn in September ’09) also contained heating elements to give wearers the sensation of blood flowing from a wound. That part of the system had to be removed because of its need for high electric current. According to Palan’s website, they also considered adding in features to simulate punches, kicks, ambient temperature, artillery explosions. Sounds like a full military or disaster relief simulator in the making.

Earlier models of the TGV included heating elements to simulate blood flow. Hopefully later versions will contain a wide range of stimuli to provide a complete virtual experience.

For now, the TGV is most notable for how it points the way to a full-body haptic experience. Combine many different modes of feedback and you’ll be able to approximate a much wider range of sensations. We already have the necessary technology to do haptics today. All it takes is solenoids, eccentric motors, peltier (heating) elements, good programming, and a lot of creativity. Palan seems to have each of those in spades. Hopefully he, or one of the many other haptics researchers, will continue to develop feedback suits until they’re good for more than just a great round of Halo.

[image credits: Saurabh Palan]
[source: iRoboticist (Palan's personal website)]

CES is all about the biggest, brightest, flashiest displays. Kudos to Samsung for its towers of LED TVs.

Integration. Dimensionality. Connectivity. Those were the three key concepts that exhibitors wanted to impress upon me as I walked the floors of CES 2010. The gadgets of this year and next were going to combine previous innovations, bring them into our 3D world, and connect us with people everywhere. The Consumer Electronics Show is the forum for getting your cutting edge technology into the hands of buyers for major retailers all over the world. As such, it’s full of remarkable innovations and cool demonstrations as well as hype and delusions of grandeur. Enjoy the following photo safari through the booths of CES 2010.

Sony, Samsung, LG, and Toshiba were all displaying 3D TVs...but it was Panasonic that won best in show. It will give us 3D HDTV (using special glasses) sometime in the second quarter of 2010.

Speaking of TV...Samsung and others were touting systems that would integrate internet information into broadcasts. The right side of the screen demos how play predictions for a football game could be displayed right next to the game itself.

Samsung, and others, demonstrated how multiple linked monitors could lead to a more immersive experience.

Flying Parrot's AR-Drone outside was awesome. Pricing is still unavailable but a rep did say it would be available in the US this year. Can't wait.

LG used Microsoft Surface to demo some of the cool features of the Chocolate Touch phone. No news on when the UK watch phone will make it to the US.

LG and Ncomputing were both pushing versions of "networking monitors". These ethernet connected screens could allow you to run 30+ work stations off one PC. Hence the $70 "PC".

The Peregrine glove from IronWill transforms your fingers into the keys of a keyboard. Conductive wires lace the glove allowing you to form circuits depending on where your thumb and finger connect. A cool bit of hardware for gaming at $150, and it may have applications in other fields.

Microvision had a Dr. Evil look alike pimping its new Pico Projector.

The Date Check mobile App from Intelius lets you use a phone number or email address to perform some pretty invasive checks of people's backgrounds. Free to download, $5-$50 to see if your prospective mate has a criminal history.

Electric cars are making a comeback!

Kia may soon integrate haptic dials into its cars for radio and other non-driving applications. Played with this thing for a while. It works really well.

Rhythm Touch's Acu-therapy was fairly popular among the CES attendees whose dogs were barking after walking all day. I'm all for alternative medicine, but these guys set off my BS detector.

Can't afford an Actroid? Kokoro makes the iFairy to cheer you up or say hi to guests as they enter a building.

I finally got to pet a Paro. Amazingly soft and cuddly. My psyche definitely felt healed.

Goodbye CES...see you in 2011. (Taken via the Pico Projector from Microvision)

[photo credits: Aaron Saenz]

The Robot Pet Sitter could be remotely accessed via the internet so that you can play with your pets while away from home.

Okay, I rarely spend time looking at “concept machines” but this one just made me chuckle enough that I had to share it. In our hectic lives there is one group that constantly suffers from lack of attention: our pets. So what’s the solution – spending more time at home, or giving your pet a backyard, maybe even finding someone to help care for your pet? No, of course, not. That’s not technologically advanced enough. What your pet needs is a robotic sitter, one that you can control from anywhere in the world via WiFi. MintSelect, a technology concept community, has dreamed up their Robot Pet Sitter, a device that will let you speak to, watch, and even play fetch with your animal. Teach your pet to obey its robotic overlord just as it would obey you.

As far as I can tell, the Robot Pet Sitter would function almost exactly as a Rovio. Like that robot, the Pet Sitter would connect to your home WiFi network and allow you to access the device through a web browser. While I can’t speak to whether or not animals will enjoy interacting with robots, I do know that telepresence is a developing trend in technology. Remotely accessing your home, for security or pet soothing, is just the beginning. Once we are able to transmit sensation from a machine to a human (a capability known as haptics) people will feel what is happening to the robot they are operating. In essence, you will be able to interact with distant objects just as if you were there with them. That level of connectivity may make travel obsolete.

Unlike Rovio, the Robot Pet Sitter concept includes two eyestalks with cameras and microphones to provide stereo vision and hearing. While that functionality would be wasted if you were accessing the robot via a traditional desktop computer, a head mounted display could give you a 3D view of the robot’s surroundings, immersing you further into playtime with Rover. A speaker in the center of the bot lets you talk to your lonely pet while recessed wheels let you chase it as it runs away from the scary looking machine. Have a more aggressive housemate? The eyestalks automatically cover the cameras with plastic lids in the case of chewing or collisions.

Robot and ball are controlled via MintPad

Of course, the really innovative addition to the robot is a remote controlled ball. Launched from the bot, it contains a center of gravity style movement system that would allow you to give your pet something to chase or fetch. The robot, or the ball, can be accessed via point and click commands from a hand held writing device, the MintPad (see below).

You’ll notice the gratuitous use of ‘would’ and ‘could’. This robot, despite it’s very pretty graphics, is simply a concept. It’s never been built and may never be built. MintPass is a company that produces streamline, perhaps even gorgeous, communication devices – think Apple but Korean. Their MintPad is a hand held note taking web browsing video and audio player, and their Sapphire software allows you to seamlessly transfer media files from internet to storage. As part of their dreamy tech savvy, MintPass has created MintSelect, a webpage where they can post concepts of possible products and see how they work. The Robot Pet Sitter is just one of more than a hundred “what ifs” that litter MintSelect.

Maybe I find the concept of the Robot Pet Sitter so charming because of those MintSelect neighbors. Alongside a USB powered UV toothbrush sanitizer, or a life-like grenade with a speaker that plays funny sounds, the Pet Sitter seems pretty realistic.

If I’m being a little harsh on MintSelect it’s because I know that the technology they are referencing in the Pet Sitter is no laughing matter. Telepresence is already having a profound effect on business travel, as it becomes cheaper and better it is likely to affect how individuals stay in touch with loved ones or even take vacation. Who knows, maybe a telepresence robot really could comfort a pet while you were at work or traveling. No matter which applications become popular, the ability to remotely access a location via the internet is already here, and will improve in the years ahead. That’s one concept that MintSelect has definitely gotten right.

[photo credits: MintPass]

Did you just call me a Power Glove? Talk to the hand, fool.

For those of you who can’t help but make comparisons to Nintendo’s ill-fated Power Glove, you should know that the acceleglove is a much more sophisticated device….it’s kind of  like a Wiimote. Seriously, the glove uses many small accelerometers (hence the glove’s name) to measure movements and orientations of the fingers. Those accelerometers are sensitive enough, and tiny enough, that a user feels like they are simply moving their hand in a traditional glove. That’s great news for the intended users.

Who’s wearing these things right now? Well Anthrotronix already has a software package setup on their site for American Sign Language users. The acceleglove is easily adapted to serve as an ASL recorder or translator. While the deaf community already communicates with the hearing well without this technology, I can imagine an ASL recorder/translator would work nicely for public speeches and community events. Though there are computers already learning sign language on their own which might work better than the glove.

On the other end of the spectrum, military uses for the acceleglove are promising. The vast array of human-robot interactions that will characterize the next generation of the US army need to be as simple and effective as possible. Already soldiers communicate with their hands to relay information and suggest broader tactical maneuvers. Using the acceleglove, robots could easily receive hand signals from commanders in the field for quick and adaptive responses as combat situations evolve. A support drone could be told to go high and scout for a sniper, or stay low to avoid fire, or bring in support; all using fast hand movements that soldiers already know.

When it comes to the next generation of robotic/computer interfaces, I’m not sure how the acceleglove will stack up. Jack Vice using finger motions to control a robot hand seemed a little contrived. Honestly, without some form of haptics I don’t know how the glove can compete with fully immersive virtual reality controls. As we’ve discussed before, the computer-human interface may start focusing on moving the computer, or interacting with it tactically, in a way that cuts out the need for an input device of any kind.

Still, for what it does, the acceleglove seems to perform really well. Intuitive interfaces will always find a market even if it is a niche. Gamers, I’m sure, will put this thing to good use. Likewise, the ASL and military applications are pretty sound. It will be interesting to see if a third party developer or research scientists finds an interesting way to employ the acceleglove in a way that Anthrotronix hadn’t considered. At $500, a single glove isn’t cheap, but it’s well within the budget of someone hoping to find a new use. If someone could include a way for users to receive sensations while wearing the acceleglove then I think it would be a truly revolutionary device.

On the outside it looks like a regular keyboard, but inside it knows how hard you're typing.

While this keyboard doesn’t provide any feedback to the user’s fingers, it does grant you a whole new way to interact with your software. It’s somewhere, but not quite, on the level of haptics we’ve discussed before. Even with just visual feedback, however, the keyboard would allow a more immersive experience

Where will we see this experience? Well first, most of us make errors when typing, hitting one stray key or double pressing others. By measuring differences in pressure, Microsoft can inform your spell checking software which letters are least likely to be intended by the user. Backspace/delete and arrow keys can have variable meaning depending on pressure. Want to delete a whole row of text? Jam down on the delete key. I can imagine the next generation of MS Office will have a whole series of variable responses based on pressure sensitive keys.

Of course, the much cooler applications will come from the worlds of gaming, where pressure sensitive controls may make the difference between virtual life and death. The simple simulation they show in the video is actually impressive enough that the pressure sensitive keyboard may sell well based on a gaming market alone.

I’m less certain of the applications for online chatting, where keyboard pressure could signal larger font size, or use of bold style. While it may help my dad stop banging on his laptop like a 70s typewriter, it will also probably lead to people unwillingly typing in all caps. I’ve had enough of that as it is.

The keyboard is probably going to be as resilient as the standard ones we use today. After all, most of the components are the same. Instead of simple off or on pegs under the keys, however, the pressure keyboard has carbon surfaces that vary in contact area depending on pressure. More parts to break, sure, but pressure sensitive electrical resistance is a relatively old technology.

The big deal is that unlike other new user-computer interfaces, the Microsoft keyboard at least has the potential to be widely available and soon. The same can’t be said for the cooler siftables, reactables, and other mixed-media interactive devices we’ve discussed before. Of course, Microsoft has a history of sitting on cool gadgets rather than trying to launch them. Hopefully these guys will see that the market is ready for the keyboard now. If MS doesn’t move quickly, another company may create a similar product. It would be better if Microsoft got the third party development rolling for the keyboard on their own terms.

Obviously Microsoft agrees as they’ve sponsored its use in the first annual UIST contest. The contest is geared to students that are interested in receiving a free keyboard, creating software that utilizes it, and possibly winning $2000. If you’ve already enrolled, here’s another demo video to whet your competitive appetite. Good luck, and make sure to send me 10% if you win.

]]> http://singularityhub.com/2009/08/18/microsofts-cool-new-pressure-sensitive-keyboard-plus-a-contest/feed/ 20 http://singularityhub.com/2009/08/07/is-surrogates-movie-getting-closer-to-reality/ http://singularityhub.com/2009/08/07/is-surrogates-movie-getting-closer-to-reality/#comments Fri, 07 Aug 2009 16:16:43 +0000 Aaron Saenz http://singularityhub.com/?p=5761 You know you live in a crazy age when blockbuster movies look like they might come true. Next month marks the debut of the scifi film Surrogates, starring Bruce Willis and adapted from Robert Vendetti’s comic series of the same name. The movie follows a police officer who lives in a future where everyone (including him) is a remote controlled android called a surrogate. You can feel everything that this robot does, but without any worries about danger. Pretty far-fetched, right?

Will the world of Surrogates come true?

Well, a new micro-documentary on Wired.com is questioning just how much fiction there is in this science. With commentary from the director and some key experts in the field, The Science-Fact Behind Surrogates is out to show that the technology we have today is laying the foundation for the reality of the film. Watch the micro-documentary video after the break.

Surrogates is set in the year 2054, when everyone stays at home and sends beautiful android versions of themselves out into the world. It’s the ultimate form of telepresence, and futurist Dr. James Canton believes it could happen in the next decade or so. Canton isn’t the only expert weighing in during the documentary. Anybots, which the hub covered previously, make robots that operate through telepresence and demonstrate that humans can already interact through their artificial minions.

So we have remote controlled robots, but what about the rest of the technology promised in Surrogates? Singularity Hub has already told you about haptics, feeling what your robot feels, and Braingate, the technology of reading your mind in order to control computers and machines. This stuff is here today. Super good looking and super strong robots obeying our every command? Well, Hanson robotics and Cyberdyne technologies have you covered there as well. We’re so close people, so close.

Question is, do we want to be? The Surrogates comics series isn’t a wholesale endorsement of the telepresence lifestyle. Far from it. Vendetti wrote Surrogates after reading about relationships that failed when someone became obsessed with online gaming and social networking. In the series, the writer lampoons the American obsession with idealized looks, surrogates come with mouth-watering physiques, and our willingness to trade safety with freedom. Or rather, our belief that safety is freedom. The whole comic (and movie, I presume) revolves around a plot by a terrorist to destroy all surrogates everywhere. It’s a technophobe versus technophile conflict that futurists have been worried about for a while. It got brought up at the debut of the Transcendent Man documentary, and it reflects the real doubt some have about whether or not humans are losing their humanity in the technological ether.

Forget the heavy stuff for a while, though, and just enjoy the smash-’em bash-’em romp that I’m sure Surrogates will be. The trailer looks fun and the movie website even has a place where you can design your own virtual stand-in. Heck, we can start taking bets on whether or not Surrogates or AVATAR will be the biggest blockbuster starring remote controlled androids this year. The future may hold impressive decisions about the nature of human interaction but for now the biggest choice you have to make is if you want to go out and see the movie. I might just send someone in my stead.

SURROGATES trailer in HD