"Extra oil and vinegar please!"

Willow Garage’s gifted PR2 robot just got even smarter. Developers from the University of Tokyo and the Technical University of Munchen have collaborated to give it the ability to find an object it can’t see or isn’t even sure is there. In a way similar to humans, the PR2 can now go after an object by reasoning where it is most likely to be.

If you’re at someone else’s house and you need a bottle opener you’d probably begin by looking in the kitchen drawers as opposed to, say, the bathroom (candles, maybe, but not the bottle opener). By using “semantic search” the PR2 calculates the probability of finding the object in a number of places. If it doesn’t find it in the first place, it continues its search in other likely places.

In the following video, University of Tokyo researchers send the PR2 out for a sandwich. It looks in the refrigerator, doesn’t see one, then takes the elevator to the ground floor and orders one at Subway!

This sort of common sense is not so common among robots (and many humans too, but that’s beyond the scope of this article). Key to semantic search is knowing where sandwiches, etc are likely to be in the first place. For this reason, the researchers introduced probability maps that indicated the refrigerator and Subway are high probability sandwich locations. The maps can be updated say, if Subway closes, and like seemingly everything else modifications can be made from an iPad.

Semantic search is the latest creation from TUM’s Intelligent Autonomous Systems. The PR2 robot platform uses Robot Operating System-based software that TUM has already contributed to significantly. Semantic search can be combined with other code packets found in the treasure trove of ROS so the PR2 can cook you breakfast, fetch you a beer, or clean up after you in a smarter way. In a video we showed last year, the PR2 depended on a human to decide which cups and bottles on the table were to be cleared away for washing and which ones should stay. By increasing the specifications of semantic search the robot’s selectivity can be made finer so that, for instance, any cups that are half-full should remain.

If robots are going to work with us and for us they’re going to need the kind of common sense that semantic search gives them. If we have to tell the robot what room, what refrigerator, what shelf and what can of beer it needs to fetch we’re apt to do it ourselves.

And that’s just no fun.

video: sandwich

Peter Murray was born in Boston in 1973. He earned a PhD in neuroscience at the University of Maryland, Baltimore studying gene expression in the neocortex. Following his dissertation work he spent three years as a post-doctoral fellow at the same university studying brain mechanisms of pain and motor control. He completed a collection of short stories in 2010 and has been writing for Singularity Hub since March 2011.