Membership Signup
Singularity University

Robot: I Now Have Common Sense. Engineer: Great, Go Fetch Me a Sandwich!

"Extra oil and vinegar please!"

Willow Garage’s gifted PR2 robot just got even smarter. Developers from the University of Tokyo and the Technical University of Munchen have collaborated to give it the ability to find an object it can’t see or isn’t even sure is there. In a way similar to humans, the PR2 can now go after an object by reasoning where it is most likely to be.

If you’re at someone else’s house and you need a bottle opener you’d probably begin by looking in the kitchen drawers as opposed to, say, the bathroom (candles, maybe, but not the bottle opener). By using “semantic search” the PR2 calculates the probability of finding the object in a number of places. If it doesn’t find it in the first place, it continues its search in other likely places.

In the following video, University of Tokyo researchers send the PR2 out for a sandwich. It looks in the refrigerator, doesn’t see one, then takes the elevator to the ground floor and orders one at Subway!

This sort of common sense is not so common among robots (and many humans too, but that’s beyond the scope of this article). Key to semantic search is knowing where sandwiches, etc are likely to be in the first place. For this reason, the researchers introduced probability maps that indicated the refrigerator and Subway are high probability sandwich locations. The maps can be updated say, if Subway closes, and like seemingly everything else modifications can be made from an iPad.

Semantic search is the latest creation from TUM’s Intelligent Autonomous Systems. The PR2 robot platform uses Robot Operating System-based software that TUM has already contributed to significantly. Semantic search can be combined with other code packets found in the treasure trove of ROS so the PR2 can cook you breakfast, fetch you a beer, or clean up after you in a smarter way. In a video we showed last year, the PR2 depended on a human to decide which cups and bottles on the table were to be cleared away for washing and which ones should stay. By increasing the specifications of semantic search the robot’s selectivity can be made finer so that, for instance, any cups that are half-full should remain.

If robots are going to work with us and for us they’re going to need the kind of common sense that semantic search gives them. If we have to tell the robot what room, what refrigerator, what shelf and what can of beer it needs to fetch we’re apt to do it ourselves.

And that’s just no fun.

video: sandwich


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>


Singularity Hub Newsletter