"Extra oil and vinegar please!"

Willow Garage’s gifted PR2 robot just got even smarter. Developers from the University of Tokyo and the Technical University of Munchen have collaborated to give it the ability to find an object it can’t see or isn’t even sure is there. In a way similar to humans, the PR2 can now go after an object by reasoning where it is most likely to be.

If you’re at someone else’s house and you need a bottle opener you’d probably begin by looking in the kitchen drawers as opposed to, say, the bathroom (candles, maybe, but not the bottle opener). By using “semantic search” the PR2 calculates the probability of finding the object in a number of places. If it doesn’t find it in the first place, it continues its search in other likely places.

In the following video, University of Tokyo researchers send the PR2 out for a sandwich. It looks in the refrigerator, doesn’t see one, then takes the elevator to the ground floor and orders one at Subway!

This sort of common sense is not so common among robots (and many humans too, but that’s beyond the scope of this article). Key to semantic search is knowing where sandwiches, etc are likely to be in the first place. For this reason, the researchers introduced probability maps that indicated the refrigerator and Subway are high probability sandwich locations. The maps can be updated say, if Subway closes, and like seemingly everything else modifications can be made from an iPad.

Semantic search is the latest creation from TUM’s Intelligent Autonomous Systems. The PR2 robot platform uses Robot Operating System-based software that TUM has already contributed to significantly. Semantic search can be combined with other code packets found in the treasure trove of ROS so the PR2 can cook you breakfast, fetch you a beer, or clean up after you in a smarter way. In a video we showed last year, the PR2 depended on a human to decide which cups and bottles on the table were to be cleared away for washing and which ones should stay. By increasing the specifications of semantic search the robot’s selectivity can be made finer so that, for instance, any cups that are half-full should remain.

If robots are going to work with us and for us they’re going to need the kind of common sense that semantic search gives them. If we have to tell the robot what room, what refrigerator, what shelf and what can of beer it needs to fetch we’re apt to do it ourselves.

And that’s just no fun.

[video credits: GerbilGod7 via YouTube]
video: sandwich

You can shake my hand for two hundred grand. I'm the PR2 SE.

Willow Garage is running a half off sale on their PR2 robot. Take half off the price and half off the robot. The PR2 is a two-armed world class research platform for personal robotics that uses open source software to power its growing arsenal of capabilities. Its full retail price is $400,000. Now, however, Willow Garage has announced the creation of the PR2 SE, a single armed version that will retail for $285,000 but be eligible for a 30% discount to open source developers, bringing the price tag down to $199,500. That means that more labs around the world will be able to get their hands on one of these research robots. With the second arm able to be installed at a later date, the PR2 SE is a gateway drug for engineers. As always, Willow Garage is hoping to hook scientists on their platforms and their approach to open source development in order to guide the fledgling personal robotics industry towards a brighter and more profitable future.

Over the past few years we’ve covered the PR2 quite often, but that’s because it’s done so much. I could write an entire article just listing its accomplishments, but instead here are a few favorite examples of its prowess: The PR2 can plug itself in to recharge, going on a 140 km marathon run just to prove how robust and self-reliant it was. It has enough sensor input and coordination to shoot pool. And it can even fetch you a beer. The PR2 is undeniably a great robot, even if it’s terribly expensive.

Part of the long term benefit of researching robots using the PR2 is the platform’s association with open source software. The PR2 uses code packets found in the Robot Operating System (ROS) library, and virtually every line of software is publicly shared, edited, and reused. Since Willow Garage gave away 11 of its PR2 robots to research institutions around the world, the amount of code on ROS has continued to grow exponentially. With so much of the basic code already figured out and shared via ROS, new labs can skip right to the good part – experimenting with the robot to further science. As they develop new code, PR2 owners will in turn share it with the community, allowing other researchers to build off their work. It’s like a giant game of leap frog that ends in a world full of awesome personal robots.

“From a standing start (robot off, never logged into a PR2 before), two of my graduate students were able to learn a map, get the navigational stack working, and drive the robot autonomously to the chair’s office on the other side of the building in less than two hours…I’ve been in robotics for about 20 years now, and this is the first time I’ve seen something work this well out of the box.”
—Bill Smart, Associate Professor of Computer Science and Engineering at Washington University in St. Louis, and recent PR2 owner.

Yet Willow Garage has sold or given away less than twenty of the robots, putting the total number of PR2s out in the world around the mid thirties (5-10 are still at Willow Garage headquarters in Silicon Valley, last time I checked).

As early PR2 prototypes showed, it only takes one arm to plug yourself in. The PR2 SE should have much the same capabilities of its sibling, but at a price that many more researchers should be able to afford.

That’s why the PR2 SE makes so much sense. Those labs with a history of open source development can get the next best thing to a PR2 but for less than $200k. That’s going to open a lot of doors. Willow Garage’s price for the PR2 SE puts it squarely in the target range for grants from the National Science Foundation and they know it. In their press release Willow Garage mentions the NSF National Robotics Initiative, whose goal is to accelerate the field of personal robotics. If you look at the grants already awarded for the National Robotics Initiative, you’ll see that the new PR2 SE will fit in perfectly – both in terms of price and research goals. Not only is Willow Garage making their platform more affordable, their aiming it at a growing bracket inside the robotics research community. Smart move.

Another smart part of the PR2 SE is that it’s upgradeable to a full PR2. The missing arm can simply be added in later. So researchers can buy the PR2 SE with their current grant, quickly produce some new work, apply for the next grant, and then upgrade their robot and do it again. Of course, a surprising amount of development in robots only uses one manipulator, so it may be a while before labs even need to upgrade in order to continue their work.

Willow Garage’s development of the PR2 (and now the PR2 SE) shows how acceleration-minded they are. Their 30% discount practically ensures that every buyer will participate in the open source community. (A similar discount with the PR2 was so appealing that only 1 or so of the 12+ labs that bought the PR2 did so at its full $400k price tag). Willow Garage hosts regular bi-monthly meetings for its developer community, fostering the sense of mutual cooperation. Meanwhile, the labs that buy the Willow Garage robot produce results faster, which in science is the surest path to financial success. Now, with the PR2 SE, there are even multiple entry points into this process so that more research lab can participate; which will only increase the rate of development in the community as a whole.

Building that community is essential because, by design, the PR2 simply isn’t a commercial robot. Willow Garage bots are research platforms, and even with the lower price tag of the PR2 SE I can’t imagine the potential market for research bots is orders of magnitude larger than the current level of sales suggest. There may only be a few hundred labs around the world (presently) that really are capable and willing to take on a PR2-class device. In my opinion, Willow Garage has to sell the community they build as much as the robot itself, because growing that community is the only way to lay the foundation for the construction of a personal robotics industry that Willow Garage is ultimately aiming to compete in.

I know I’m always cheerleading the PR2 and ROS, but that’s just because they make so much sense to me – both for Willow Garage and the robotics industry as a whole . Now is not the time for every robot researcher to toil on their own. A shared standard (not to mention a shared platform) benefits everyone, and will more quickly get us to a point where splitting off and working on proprietary projects is actually feasible. We’ve seen many exciting products in development that could be called “personal robots” and I’m sure that at least some of those will have a big impact on our world. But the really human-like robots – the ones you see in movies and would want in your home folding laundry and cooking dinner – those are going to take massive amounts of further research to create. Willow Garage is one of the companies helping us get there, and the PR2 SE is a great step in the right direction. Kudos.

[image credit: Willow Garage, Engadget]
[source: Willow Garage]

TUM-Rosie is ready to make the best wurst you've ever tasted.

It’s official: robots make great roommates. Sure, they may hog all the electric outlets and WiFi bandwidth, but they’ll also cook you breakfast. The Technical University of Munchen (TUM) recently demonstrated the culinary skills of two of their robots by having them whip up a traditional Bavarian breakfast of sausage and bread. TUM-Rosie is a unique bot made of Kuka parts that knows how to boil a wurst in the best way. TUM-James is a PR2 from Willow Garage that takes care of the shopping and bread slicing. Together they make one helluva good meal. And this wasn’t even their first time in the kitchen – the duo made pancakes last year! Check out their culinary experiments in the videos below. While getting a free breakfast from a robot is amazing, it’s just a small step towards the ultimate goal: designing, building, and programming a bot that can live and work in human environments.

Here’s the clip of TUM-Rosie and TUM-James working together to build a Bavarian breakfast. TUM-Rosie places the sausages in boiling water, keeping track of how long they’ve been inside to cook them thoroughly. TUM-James picks up a few things at the ‘store’ and then slices the bread with an ordinary electric slicer. The results look delicious:

Also available are full length (non-accelerated) videos of the TUM-Rosie and TUM-James efforts.

Last year TUM-Rosie and TUM-James pulled off making pancakes, with TUM-Rosie in charge of the actual cooking again. Unfortunately this video hasn’t been edited or sped up, so it gets a little tedious. But hey, it’s hard to complain when you’re eating pancakes:

Making a meal is a complex endeavor, and TUM’s Intelligent Autonomous Systems Group has rightfully received a lot of praise (and press) for these demonstrations. Less discussed, but perhaps more important however, is TUM’s considerable contributions to the Robot Operating System. ROS is an open source library for robotics code, and both TUM-Rosie and TUM-James are running on this software. TUM’s description of the code packets used and created for these cooking experiments highlights how interconnected (and thus, mutually supporting) the ROS community has become. There’s software to manage the Kinect sensors, object recognition, control of the cooking utensils, etc. Breakfast is one thing, but what TUM is really demonstrating is how powerful open source robotics can be.

Don't worry Rosie, real world robotic help is on the way!

I fully believe that in the years ahead, small projects like these will snowball into greater efforts. Already, the PR2 (and TUM-Rosie) are safe to move around in human environments. They won’t break your arm if you get in their way, and they won’t crush the objects they pick up. TUM, and other ROS contributors, are building up the library of programming packets that allow these robots to perform various tasks. We’ve already seen how the PR2 can fetch you a beer, fold towels, or clean up after your party, and developers will continue to add on to these capabilities in the near future. Eventually, the compounding nature of shared code on ROS should give us a host of robots capable of performing a huge range of domestic tasks. From there, it’s really just about pricing, availability, and business models. Big hurdles, sure, but the robotics part will be understood.

In the last few years, robot cooks have proven to have at least a novel appeal. Eventually, I think they’ll be invaluable resources in our homes. Cooking, cleaning, laundry – at first they’ll only be able help with these chores, but one day they’ll be able to take over completely. Robots have already revolutionized manufacturing, they’re poised to move into healthcare, and homes are the next frontier after that. It may be a little early to salivate, but make no mistake, robot breakfast is coming.

…and it just so happens that pancakes and sausage is my favorite morning meal. Awesome.

[image and video credits: Technische Universität München]
[source: TUM, Willow Garage Blog]

Scratching a ten year itch with the help of the PR2 and Robots for Humanity.

For ten years, Henry Evans couldn’t even scratch his nose. Now he has a $400,000 robot do it for him. A mute quadriplegic since his stroke at the early age of 40, Evans saw Georgia Tech’s Professor Charlie Kemp demonstrating a new robot on TV. It was the PR2 from Silicon Valley’s Willow Garage, and Evans decided he just had to get in on the action. Collaborating with the Healthcare Robotics Lab at Georgia Tech and Willow Garage, Evans has slowly gained a mastery of commanding the PR2. No easy feat considering he can only move his head and a single finger. With head tracking and simple controls Evan is able to command the robot to pick up items, drop them off, shave his face, and, yes, scratch his itches. Check out the amazing video below. Part of a new initiative called Robots for Humanity, the work done with Henry Evans hints at the awesome potential that the PR2, and personal robots in general, have to revolutionize the healthcare industry.

I’ve seen some pretty incredible projects in robotics, but Robots for Humanity is something very special. The tasks the robot performs may not seem extraordinary, but its the interaction with Henry Evans that really sets this program apart. Evans must be able to command the sophisticated PR2 with minimal input and somehow still achieve exactly the right outcome all while maintaining a work environment that is perfectly safe for both he and all the other humans in it. It’s a high wire walking act performed remotely using only head motions and a single finger, but the PR2 programming turns it into a cake walk. I can’t get enough of the smiles on the faces of Henry and his wife, Jane. Perhaps they’re catching a little glimpse of what their future could be.

As you see in the video, part of what makes the PR2 such a useful tool to Evans is its extensive library of autonomous behaviors. While he can move the robot’s grasping hand independently, the more impressive tasks (such as moving an object from one room to another) are done by initiating autonomous routines for navigation, object recognition, and grasping. The software for those self-guided actions is what makes healthcare robotics different (and potentially more practical in some cases) than robotic prosthetics. Some patients will be able to replace lost limbs with robotic appendages they can feel and command with their thoughts. Others will be able to augment their broken bodies with robotic exoskeletons. But for some patients, especially quadriplegics like Evans, a remotely controlled partially autonomous robot is the only solution.

What a solution it is. Clearly Robots for Humanity is in its very early stages, and we shouldn’t expect game-changing developments from them soon. Still, this project shows how a robot with the PR2′s level of sophistication goes a long way to providing patients with a device that can handle many of the tasks that normally would require a human assistant. In some ways, using a robot like this is tantamount to giving a patient a remote body to command. The sensory feedback and controls are both relatively primitive compared to what nature provides, but it’s still astounding. Give brain interface technology enough time, and maybe we could even wire these helper robots directly into our heads. Such possibilities are still years away, but they are getting nearer.

The possible benefits for programs like Robots for Humanity extend beyond quadriplegic patients like Evans. There are huge emerging opportunities in longterm elder-care that will likely dwarf the more specialized (and demanding) market for completely disabled users. The Healthcare Robotics Lab at Georgia Tech is also working on their Assistive Mobile Manipulation for Older Adults at Home project. Using the PR2 as a research platform, the lab is looking to develop the software and structure to allow elders to live on their own with the assistance of a robot. We’ve seen similar initiatives from Panasonic and Toyota, though all are in fairly exploratory phases. Imagine a time when personal robotics are so advanced, and so universal, that every senior citizen or disabled person can have their own mechanical helper that helps them live happily and independently. Projects like Robots for Humanity make that day seem much closer than ever before.

Best wishes to Henry Evans, Jane Evans, Charlie Kemp and everyone involved with Robots for Humanity. Personal robotics has never looked so important.

[screen capture, image, and video credits: Willow Garage]
[source: Willow Garage blog, Healthcare Robotics Lab at Georgia Tech]

The PR2 caught the philosophical mood of the informal press dinner at Willow Garage.

When a robot asks you out on a date, say yes. Wednesday night, Willow Garage hosted a small press dinner at their offices in Silicon Valley. The young company looking to jumpstart the personal robotics industry wasn’t making formal announcements, but they gave myself and the half dozen or so other attending journalists a rare glimpse into their visions for the future. There was also really good food. What do the next few years hold for Willow Garage? Spin-off companies, mid-priced platforms, and tons more open source robotics. Read below to see some of the many cool concepts that CEO Steve Cousins and Open Source Development Director Brian Gerkey batted about over dinner.

Willow Garage set out to accelerate the field of personal robotics. Four years ago, they thought the elbow of the exponential curve of development was 10-15 years away. Now, Cousins is willing to guess it could be as little as 5 years away. In other words, Willow Garage thinks they are making a positive impact on the industry and could see the enormous fruits of their labor relatively soon.

A team of developers working on Project Texai, a telepresence robot with a simple design and a huge display screen, has officially spun off from Willow Garage. The new company, Suitable Technologies, is headed by Willow Garage founder Scott Hassan, and adopted “quite a few engineers” from its parent firm to get started. We’ll hear more about this group as they bring Project Texai to market, but expect big things from this first proprietary and distinctly profit-minded venture from Willow Garage.

Anticipating integration with Android, this TurtleBot got a hacked holder for a tablet.

Remember that amazing towel folding routine performed by the PR2 robot living in UC Berkeley? Well, Gerkey says that they’ve successfully been able to port that program into a general demonstration Willow Garage performed on the road at ICRA 2011. Gerkey pointed out that while open source robotics has promoted the easy exchange of code, reproducing experiments is still a problem in the scientific/engineering community. Getting the towel folding parameters up and running was a significant achievement, and one that will hopefully be repeated across many more robot experiments in the future.

The Robot Operating System continues to impress. Back in November we reported that the library of open source robotics code had been growing exponentially over the course of its three year history. Gerkey says that trajectory is still on course. There are now around 3000 packets of code on ROS, nearly double what there was in November! There are roughly 100 depositories on ROS (of which Willow Garage is hosting about 12). Gerkey estimates there are about 30 ROS developers directly associated with Willow Garage, with about 1000 people on the user’s list. Clearly ROS is a thriving community.

Willow Garage has been working closely with the Google Cloud Robotics team, which has a few of the new WG TurtleBots to play with. Gerkey and Cousins are excited about the integration of Android and ROS, and we even saw a hacked mount to put a tablet on a TurtleBot. While TurtleBot won’t be sold out of the US, the robot is officially open source hardware, meaning anyone can build it. Already, Gerkey says there are enthusiasts looking to establish an EU version of the robot.

There are 30+ PR2 robots in the world, about 9 at Willow Garage, 11 given away for free. Of those that were sold, only one paid the full $400,000 price tag. The rest took advantage of the 30% discount afforded to those groups which follow strict open source guidelines.

Could there be room in the Willow Garage family for a mid-range robot?

Looking to the future, Cousins casually mentioned that Willow Garage will eventually find a middle ground between their research platform juggernaut, the PR2, and their enthusiast level platform, the TurtleBot. You can do some really amazing things with a $400,000 PR2, and some pretty neat stuff with a $1200 TurtleBot. But there’s a huge range to explore in there. Hopefully Willow Garage will find a sweet spot where they can create a new tier of robot that will capture the interest of a whole new market, while simultaneously continuing to attract developers to open source robotics.

As always, visiting Willow Garage was as fun as it was interesting. There’s a lot of vitality coming out of their Silicon Valley office, especially considering how many of their residents aren’t human. Good luck, guys, and try not to wait too long before you give us some really juicy announcements.

[image credits: Aaron Saenz/Singularity Hub]

Germany's pool shooting robot has some serious skills.

Not every masters thesis will earn you street cred in a pool hall, but this one will. Thomas Nierhoff, from the Institute of Automatic Control Engineering at the Technical University of Munchen in Germany, programmed a two armed robot to shoot billiards…and to do it well. The bot, with seven degrees of freedom in each arm, was able to sink easier shots with up to 80% success, which isn’t quite world class level, but definitely beats my skills. Watch Nierhoff’s pool protege sink five balls in a row in the video below. Presented at last month’s ICRA conference in Shanghai, Nierhoff’s work demonstrates how advanced research robots are getting better at performing complex cognitive tasks. More than ever it looks like it’s a question of when, not if, robots will match humans in all kinds of physical feats. The answer: sooner than you think.

The Munchen robot starts its five ball run at 1:32 in the video below. Make sure to watch until the end to catch more examples of the bot’s successes (and failures) at shooting pool.

Nierhoff (in cooperation with Omiros Kourakos and Sandra Hirche) tested the robot in 400 separate shooting situations. While able to sink nearly 80% of the easiest shots, the robot did progressively worse as shots became more difficult. That’s to be expected, but its closer to 30% success for the hardest situations would make me think twice before betting any money on the robot’s skills in a real pool game.

Overall, the Munchen robot was pretty impressive, but it’s not the first pool shooting bot we’ve seen. Willow Garage’s PR2 wowed us with it’s snooker skills almost exactly a year ago. While the Munchen robot relies on an overhead camera, the PR2 did all its visual analysis using onboard sensors. The latter is more impressive to me, though I appreciate the ability of humans to piggy back on the Nierhoff system to improve their game (see the video above at 1:21). On the other hand, the PR2 required a special adapter tool to hold the cue and get a smooth stroke on its shots, while the Munchen robot used nothing but its own hands. The circumstances surrounding each project are perhaps equally matched as well: Munchen’s was a master’s thesis, not a mainline endeavor, and Willow Garage put together their demo in a single week.

If the two robots went head to head in a match, it’s hard to imagine which would win in a contest. Luckily we don’t need our imaginations. The Technical University of Munchen received a PR2 in Willow Garage’s give away last year. The PR2′s hosts at TUM, the Intelligent Autonomous Systems group, should bring their bot over to the Institute of Automatic Control Engineering lab and see how the two robots do in a real contest. I mean, the software for the PR2 is openly available on ROS, how hard could it be to get a competition going?

The real match up of the 21st Century, however, will be robot versus humans. While robot soccer teams are hoping to compete with World Cup caliber humans by 2050, robot pool could be ready decades earlier. In situations where they don’t need bipedal walking, balancing, or difficult dynamic motions, robots are going to be our equals in physical performance fairly soon. In many ways, they’ve already surpassed us – just look at manufacturing plants the world over. Nierhoff may have been producing a cool demonstration of his programming skills for his thesis, but he was also showing us how easily robots could be adapted to even the most human of activities: having fun. Good luck to the new engineer and his colleagues in Munchen, I’m sure their next project will be even more amazing.

…especially if it’s a pool competition with the PR2.

Let's make this happen, people!

[screen capture and video credits: Thomas Nierhoff via IEEE Spectrum (GerbilGod7)]
[source: IEEE Spectrum, Technical University of Munchen]

GRASP taught the PR2 to read...I taught it to use photoshop to look awesome.

The University of Pennsylvania has been elevating students’ understanding of literature since 1740…and robots’ since 2011. UPenn’s GRASP Lab has taught one of Willow Garage’s PR2 robots how to read out loud – the man-sized research platform is able to locate text in its environment and convert it to speech. In the video below, graduate student Menglong Zhu walks the PR2 through an impressive variation of fonts in real world settings to show off how well the new literacy code works. Posters, emergency signs, even handwriting – the GRASP PR2 reads them all and at strange angles and orientations as well. It’s pretty damn impressive to see a full sized robot rolling around and reading what it sees. Makes you wonder when it will be invited to storytime over at the local elementary school.

Zhu does a great job demonstrating all the different real world text the PR2 can handle in the following video…but it does get a bit repetitive. Feel free to skip around.

The PR2 is far from the first literate robot, there have been automated machines with optical character recognition (OCR) for decades. More recently, 2009 saw a bot in Japan that could read books, and last year a different machine in the UK did the whole “roaming and reading” routine. There are two things that set the work at GRASP Lab apart, however. First, the PR2 itself. This isn’t a dedicated reading robot, or a small cart on wheels with a camera and OCR- it’s a life-sized humanoid that can accomplish a whole range of different activities. Instead of building a robot to read books, GRASP took an existing robot and taught it to be literate.

Secondly, and perhaps more importantly, the results produced by Zhu (with Post Doc Kosta Derpanis and Professor Kostas Daniilidis, by the way) are open source. As with all the Willow Garage PR2 robots that have been given to research institutions like UPenn, the work here is going to be freely shared through the Robot Operating System (ROS) library as open source code. It’s not just the PR2 then that benefits from Zhu’s work. Ostensibly, one day ROS enabled machines everywhere could use this code to make themselves literate – it’s like Hooked on Phonics for robots.

I’m really interested to see how this work gets shoe-horned into other projects. We’ve seen how a single innovation -the adaptation of the Kinect 3D sensor, has lead to an explosion of ideas for ROS. Might a reliable literacy program for the PR2 produce similar results? Too soon to tell, but the possibility alone highlights how powerful and accelerating the concept of open source robotics can be. Today we’re teaching robots how to read. Tomorrow maybe they put what they read to good use. …That or they get distracted by romance novels. You never know with robots.

[screen capture and video credit: DreamDragon1988 (Menglong Zhu)]

[source: ROS]

Open source robotics will conquer the world... but first it has to conquer fads from the 80s.

“Welcome to your new home, now please solve this Rubik’s Cube.” – when you’re a robot, people greet you in some funny ways. No sooner had Willow Garage’s PR2 shown up to the University of Ulster than they set it to work…er, play. Chris Burbridge and Lorenzo Riano of the Cognitivie Robotics Group taught their newly arrived PR2 to solve a Rubik’s Cube completely autonomously. Place the cube in the robot’s hand and it scans, solves, and shifts the puzzle until it is finished. Watch the PR2 master this icon of the 80s in the video below. Little demos like this are just cute introductions to a much more serious endeavor. As one of the growing number of research institutions using the PR2 as an open source development platform, the Cognitive Robotics Group at the University of Ulster is helping create the next generation of personal robots, one piece of code at a time.

I’m not going to lie to you, I’ve seen robots solve Rubik’s Cube way faster than the PR2…and they were made out of Lego. In fact, the PR2 is slow enough that in order to keep this video a reasonable length (<2 minutes or so) they gave it a really easy cube to solve. Yet the following demonstration is remarkable when you consider everything the robot is doing. It has to rotate the cube and scan its surface, find a (potentially sub-optimal) solution, and then translate that solution into physical movements for its hands. That’s a lot of work for a measly 2 minute demo, but it highlights just how complete the PR2 is: manual manipulation, sensing, calculation, and translation of digital models into physical actions. All autonomously. Not bad for an opening act – either for the PR2 or Burbridge and Riano.

Willow Garage’s PR2 is one of dozens of bots using the Robot Operating System (ROS). This open source library is the digital soul to the PR2′s metallic body. As the University of Ulster, and the more than 15 other institutions using the PR2, continue to innovate its code on ROS, they’ll be opening up many opportunities for their work to earn compound interest. Code developed by one team can be cannibalized, optimized, and reused by any other team. The same goes, in a slightly more limited way, for the hundreds of developers using ROS on platforms other than the PR2. The arrival of the newest metal member of the Cognitive Robotics Group speaks to the possibility that personal robots could be undergoing an accelerated growth in the near future fueled by the open source community. That’s pretty exciting.

Speaking of excitement, I’ll leave you with this video the Cognitive Robotics Group made of the PR2′s arrival last month. For anyone who has unwrapped a long awaited gift, the reaction will seem very familiar. Keep up the enthusiasm guys, and push yourselves to create even better demos. Next time I want to see the PR2 throwing Pop Rocks into a soda can, dancing to Thriller, or following the flashing lights of Simon. I’m just going with the 80s theme here…

[screen capture and video credits: University of Ulster]
[source: University of Ulster, Willow Garage Blog]

In a test of its fortitude, the PR2 has gone the distance.

What do you do when your robotic child is all grown up? I guess you make it run a marathon. Over the past few years, Willow Garage has taught its PR2 robot to be autonomous. The Silicon Valley startup showed the bot how to plug itself in. They taught it to navigate on its own. They even taught it how to call for help in case of emergency. On December 8th it was time to see how well those lessons were learned. Willow Garage told the PR2 to start rolling and not stop. It traveled over 70 km in 7 days…then it kept going for six days more! On December 21st the bot ended its marathon having gone a total of 138.9 km (~ 86 miles). The PR2′s epic journey probably doesn’t look like much from the outside, after all the robot was simply wandering around the Willow Garage office, but it represents a major accomplishment for the company and for open source robotics. In the future, robots like the PR2 will be able to perform a wide variety of jobs, and without human supervision.

Willow Garage employees gave the PR2 a little flair for its epic journey.

This marathon run is important because it pushed the limits of the PR2′s autonomous operation, not because it could gauge the physical stamina of the machine’s parts. I don’t just care if a wheel breaks, I want to know how long I can leave a robot alone and not have to worry about whether it is stuck or fallen over. To that end, this in-office demonstration was a great example of how the robot could function on its own. When the PR2 encountered an obstacle, it moved around it. When its batteries were low, it found an outlet and plugged itself in. Most importantly, when it got desperately stuck it sent a text message to an engineer who could log on to a web portal and remotely get the robot rolling again. According to Willow Garage, that scenario only happened twice: “During the run, there were only two interventions: one to help the robot maneuver around a chair, and another to tell the robot where it was (“re-localization”). In both cases, the robot noticed there was an issue and sent a message for help, and the issue was resolved over the web.” Two bugs in 139 km? Not bad.

What finally did the robot in? Its plug got stuck somewhere the robot couldn’t reach. That simple. If this robot was in your house, doing chores, you could have solved that problem in 5 seconds. This 139 km marathon run shows that personal robots will not only be able to operate autonomously, but that their most common failures might be very easy to fix.

Certainly engineers have created robots that have gone farther than the PR2. NASA’s Mars Rover? That has Willow Garage beat thousands of times over. Yet the PR2 is both more practical and more accessible. The techniques used to keep the PR2 running autonomously are available to all robot developers through the open source ROS software library. There are now 16 different groups around the world developing the PR2, and many more using ROS. PR2 developers can write code for the robot that takes days, or even weeks, to finish and remain confident that the bot can finish the task. The PR2′s marathon session is just a tiny step in the grand history of robotics, but with the connectivity of the open source community it will be a step multiplied many thousands of times.

[image credit: Willow Garage]
[source: Willow Garage Blog]

Willow Garage announces it's first sales for their PR2 robots. Get dressed boys, your new parents are ready to take you home!

More institutions are buying into Willow Garage‘s vision for accelerating the field of robotics. The Silicon Valley startup announced that it recently sold its PR2 robots to four new research groups. Willow Garage has already shipped new bots to Samsung’s resarch center in Suwon, South Korea, and the University of Washington in Seattle. Soon others will arrive at George Washington University in DC and the CNRS Laboratory of Analysis and Architecture of Systems (LAAS-CNRS) in Toulouse, France. Quite the global spread. At $400,000 each these robots aren’t cheap, but they come backed by an ever expanding collection of professional quality open source software via the Robotic Operating System (ROS) library – a free resource that can be utilized by many different robots, including the PR2. With code found on ROS, the PR2 robot has been programmed to accomplish an incredible variety of tasks. Willow Garage highlights these successes in their newest video montage shown below. Four more world-class research institutions adopting the PR2 platform and ROS mean that the global community of mutually supportive robotics engineers are receiving a valuable boost.

When Willow Garage announced it would begin sales of the PR2 back in September it also described a sharp discount (down to $280,000) for institutions with a history of open source contributions. It’s no surprise then, that some of the four new buyers (Willow Garage will not comment on which) will be receiving their new robots on the cheap (relatively speaking). Why would Willow Garage voluntarily cut into its profit margins? Community building is more important. Earlier this year, the company gave away 11 of its PR2 robots for free simply to help enliven the industry and generate more innovations for its platform (known as the PR2 Beta Program). There’s little doubt that Willow Garage is taking a long term vision for its own growth, and it knows it can create the personal robot of the future much faster if the entire global collection of robotics engineers work together.

The medium for that cooperation is ROS. The ROS library recently turned three years old, and its collection of code packages and robotics algorithms has been growing at an exponential rate. Partly that growth is due to the work of Willow Garage engineers and their constant additions to the library. Mostly, however, it’s due to the ever expanding group of research institutions that are using ROS. Because it is open source, engineers can download, use, update, and upload ROS code, and adapt it for any number of different robot platforms. Samsung, University of Washington, George Washington University and LAAS-CNRS will be able to jump right into major development because so much of the groundwork in hardware and software has already been created. According to an engineer at Samsung, “we were programming on and navigating the PR2 in less than one day.”

There are more than 50 bots that use ROS, but the PR2 is definitely the flagship of the group. It has advanced sensors, hands, and stability that give it a wide range of possible applications. In the following video you’ll see the PR2 fold towels, fetch beers, and perform dozens of other tasks. Many of these clips show work performed at Willow Garage, but you’ll also see results from the 11 teams that received PR2s during the Beta Program. Make sure to watch at 2:18 where you’ll see a great preview of a new project featuring the XBox Kinect 3D sensor:

So if the PR2 and ROS are such an awesome combination, what are the four new teams going to do with them? According to Willow Garage, Samsung will be using its purchase “to enhance their existing robotics research.” That’s a little vague, but as WG points out, Korea is looking to put a robot in every home by 2020 – we should expect some exciting developments in personal robotics out of Samsung in the years ahead.

Professor Joshua Smith received the PR2 at the University of Washington on November 2nd. Smith has a history of working with sensors, and finding new ways to power them. He was recently at Intel working on wireless energy links. It should be interesting to see what he can do with the PR2′s collection of IR sensors, lasers, and cameras.

Rachid Alami, part of the Robots and Artificial Intelligence group, will be receiving the PR2 at LAAS-CNRS. According to Willow Garage: “Alami and his colleagues have earmarked the PR2 for the development of high level interactive and cognitive functions in the context of an ambient intelligent system for assistance, such as housekeeping for seniors.”

At George Washington University, the PR2 will be greeted by Evan Drumwright. His interest is to get robots performing occupational tasks. We’ve seen many developers looking for ways to incorporate robotics into unexplored realms of manual labor, and Drumwright is no exception. Willow Garage tells us we should look to Drumwright for “advances in dynamic robotic simulation, motion planning, and collision detection algorithms.”

Currently there are just 16 research institutions with PR2 robots around the world (4 new buyers, 11 Beta Program recipients, and Willow Garage themselves), but the group of ROS developers is much larger, and the general open source robotics community even larger still. Every team that contributes to the open development of robotics helps accelerate the growth of the industry as a whole. The four new entrants have valuable skills that could strengthen the open source robotics community and deepen the level of tasks that personal robots can accomplish. Even if none of the new PR2 recipients contributes a single packet to ROS, however, their purchase of the robot alone adds to Willow Garage’s clout. Bottomline, one way or another these sales are a good sign that open source robotics is working. With any luck, open robotics will keep growing and the shared code will continue to pile up exponentially.

[image and video credits: Willow Garage]
[source: Willow Garage]

LEFT: Bad robot, no crushing! RIGHT: That's it, treat the Sunkist with respect.

If Sam Waterson taught me anything it’s that I need to fear the powerful grip of robots. Because robots are made of metal…and they’re strong. Thankfully Silcon Valley robotics startup Willow Garage has worked to relieve my anxiety by teaching their PR2 bot how not to crush things – including humans. This summer UPenn grad student Joe Romano instructed the PR2 how to use tactile feedback (aka haptics) to hold delicate objects. Sensors in the grippers allow the robot to easily pick up and place objects, or hand them off to a human. Other researchers can freely incorporate Romano’s work because, as with all Willow Garage code, it’s made available via the open source Robot Operating System library (ROS). The PR2 already had some safety features on its movements that kept it from banging into people, but the new haptic feedback is a big improvement. Making sure a robot won’t crush things is a necessity if you want to sell it to researchers all over the world, as Willow Garage is aiming to do. Catch the video below to see Romano showcase the PR2′s human friendly grip. Make sure you watch until the end to catch a great series of high five shenanigans. Double fist pound into explosion – Ha! Priceless.

[screen capture and video credit: Willow Garage]
[source: Willow Garage Blog]

The PR2 contemplates its worth.

How much would you pay for a robot that can plug itself in, beat you in a game of pool, and fetch you a beer to soothe your wounded ego? Willow Garage is hoping the answer is $400,000. The Silicon Valley based robotics firm has officially begun selling its PR2 platform. After a preliminary announcement a few weeks ago, Willow Garage already has more interest than production capacity, but you can place a deposit and put your name in the queue for the next available PR2 to come off the assembly line. I had a chance to talk with Willow Garage representatives about PR2 sales, and what the development may mean for the future of the company. As a champion of open source robotics, Willow Garage’s distribution of the PR2 may be more about sending open source ambassadors into the market than it is about generating profits. Those with proven dedication to the open source community may be getting a copy of the bot for just $280,000.

The PR2 has continually impressed me with its capabilities. It has a wide array of sensors and its grasping arms are top notch. But even more impressive than its hardware are the tasks the bot can perform with its open source code (all available via ROS). We’ve seen the bot fold towels, pick up trash, and play music as well as the autonomous recharging, pool sharking, and beer fetching we mentioned above. Each of these accomplishments are built off of earlier ROS software tools and each will enable further innovation in the open source community. When you try to place a value on the PR2 you have to include access to this community and the ability to help shape it. The growth of that community has largely been Willow Garage’s focus up to this point. They even gave away 11 of their PR2 platforms to various research teams around the world just to advance the cause of open source robotics. That’s a $4.4 million investment.

Talking with Willow Garage representatives, it certainly seems that they are looking to increase that investment. A few awards (no idea how many) will be offered to PR2 customers with a proven history to furthering the open source robotics community. Those awarded will be given a chance to purchase a PR2 for just $280,000. Willow Garage can’t require anyone to use a purchased PR2 exclusively for open source research but the award is a nice way of encouraging customers in that direction.

30% is certainly a nice discount, but I’m not sure how many first wave customers would NOT qualify. Most are expected to be research teams, academic and industrial, looking to get their hands on a cutting edge platform with an ever expanding library of open source code (in the form of ROS). While some of these will undoubtedly be exploring open source for the first time, many will have some familiarity with it. Could Willow Garage sell more PR2s at $280k than at $400k? Possibly, but I doubt it. Picking who gets those awards may be difficult.

Willow Garage’s sales of the PR2 may mark a (very gentle) shift towards profitability at the company. For the first time, I’ve gotten a better sense of the long term financial goals and possibilities for Willow Garage. Although they wouldn’t specify exactly how much, the sales of PR2 at $400k will net the company a profit, (considering the aforementioned award size, ’30%’ does seem to suggest itself fairly readily, doesn’t it?). The PR2, while utilizing open source software, is not open source itself. The module specs are all online, but you can’t build a PR2 on your own (I’m not sure I’ve ever made this clear in past PR2 coverage, and I apologize if I misrepresented the PR2 as open hardware). As such, Willow Garage is likely to recoup some of its investments over the past few years with upcoming PR2 sales. Because so many of their prospective customers are in academia, Willow Garage is already in talks with the large institutions that distribute scientific funds via grants about how best to provide the robots to those researchers who want them most. We may see nations purchase PR2s and lease them out to research teams, though that’s just speculation on my part.

Why would a country, or any research institution, make such an investment? Simply speaking, the PR2 is one of the most advanced robotic research platforms in the world, and the ROS library has the potential to accelerate progress in robotics considerably. Willow Garage isn’t the only group sponsoring open source robots (URBI, CARMEN, and Orocos just to name a few) but it is one of the best funded. The Willow Garage YouTube channel, and their blog, is constantly announcing new innovations from their team of engineers. Many are too technical for us to regularly cover here on Singularity Hub, but you can check them out for yourself and see the steady advancements Willow Garage is making. With this dedication to robotics research, and with the 11 PR2 robots in major institutions around the world, Willow Garage is likely to be able to prove the success of its approach in the next few years.

Research teams may be lining up to purchase PR2s now and join the development of the next generation of robot software, but it’s unclear when Willow Garage will start to generate a company-wide profit. Talking with their reps, they certainly still seem focused on long term goals over short term gains. There was indication that the company is moving towards finding profitable robotics applications, and to enable further research in that direction, but it may be a while before we know what any of those apps could be.

The more research teams that use the PR2 and generate code for ROS, the sooner we are likely to see robots that can actually perform the tasks we’ve dreamed about and seen in science fiction. My only concern (and it’s a big one) is that no one, Willow Garage included, seems to be focused on providing this sort of accelerating research at the retail level. Where’s the affordable personal robot that everyone can own? Maybe Willow Garage’s development of the PR2 and ROS will enable others to pursue that market. In any case, I look forward to seeing how the first round of PR2 customers will expand open source robotics. I was excited about the 11 teams that received the free PR2s, now they may have dozens of colleagues working towards the same goals.

[image credits: Willow Garage]
[source: Willow Garage]

Ze'ev Klapow taught Willow Garage's robots some new tricks as a summer intern..

I’m not sure what you’re favorite summer job was during high school but I bet it wasn’t nearly as cool as Ze’ev Klapow’s. He’s one of the student interns at Willow Garage and this summer he got to spend his time hanging out with the company’s most popular robots. Klapow developed a prototype IR sensor for the Texai, a telepresence bot, so it could detect and avoid ledges. No more falling to its death whenever it neared a set of stairs or other sharp drop. He also developed code for the PR2 research robot that allows it to calibrate its sensors with almost no outside help. Klapow’s work is available as part of the Robot Operating System (ROS), free for anyone else to use, improve, and adapt. Check out Ze’ev explaining his work in the video below. Robotic suicide avoidance and auto-calibration, not bad for an intern.

I enjoy writing about Willow Garage mostly because I like their robots and open source philosophy, but I’m becoming more and more enamored with their work ethic as well. They get a lot of effort out of their engineering staff, as evidenced by their fast and furious ‘hackathon‘ programming sessions we’ve covered this summer. It’s clear that they get just as much dedication from their students/interns as well. Yes, Klapow’s results aren’t mind-blowing, and I’m sure for a mature robotics engineer these tasks would be relatively straight forward. But for a high school student this is top notch work, and valuable too. Klapow’s code is now available to all the research groups using the PR2s around the world, and will be a part of ROS forever. His work with the Texai is equally useful and might prove valuable if Willow Garage ever ends up bringing the telerobot to market. Kudos to Klapow for these two successes and kudos to Willow Garage for its dedication to the robotics community, whether it be through open source code or training future engineers one summer at a time.

[screen capture and video credit: Willow Garage]
[source: Willow Garage]

Someone please lend me a few hundred thousand dollars, I have something I need to buy. Willow Garage just announced that its PR2 research robot would go on sale starting next month. The PR2 is a world class open source robot whose accomplishments include plugging itself in, playing pool, and light house-work. Willow Garage spent years, and reportedly millions of dollars, developing the PR2 as an advanced research platform and recently gave away 11 of the robots for free to scientists around the world. I talked to the Silicon Valley company to try to find out more details about the sale, but they’re being very tight lipped at the moment. I do know that the price will be around $400k, that prospective customers can contact Willow Garage for a quote and that there will be a more formal announcement as they get nearer to the sales launch. Who’s going to spend nearly half a million dollars on a research robot? Best guess: universities and private research firms looking to accelerate their robotics programs. Second guess: people who don’t want to get off the sofa to get a beer.

[image credit: Willow Garage (modified)]
[source: Willow Garage]

Robots fetching beer... Super Bowl ads will never be the same again.

Not convinced that robots are the wave of the future? Maybe an ice cold lager will change your mind. Willow Garage has pulled off the ultimate engineering feat: teaching a robot to fetch you a beer from the fridge. As part of their summer ‘hackathon’ series, a team of Willow Garage engineers toiled for a week straight to develop a new demonstration of their PR2 robot’s prowess. They succeeded with flying colors. Not only can the PR2 select the correct brew from the fridge, it can deliver, and even open the beer as needed. That’s right, all you humans have to do is drink and relax. Prepare yourself for some major robot-envy as you check out the PR2 deliver much needed refreshment in the video below.

This is the third hackathon we’ve seen from Willow Garage this summer – the PR2 has already learned how to shoot pool and clean up after a party. It’s starting to seem like the Silicon Valley startup is getting ready to open the coolest robot bar ever. (O’Asimov’s? Bar2D2? Toasters?) I continue to be impressed with these hackathon projects, not just for their fun content, but for the incredible speed at which they are accomplished. One week’s work is practically nothing in terms of research and development, yet Willow Garage engineers consistently pull off some great feats. How are they are able to work so quickly? It could be that the open source library (ROS) provides a means of recycling previous work and enabling future progress. Or it could be that engineers will work harder when they have the promise of a liquid reward:

This ‘Beer Me, Robot’ hackathon highlights several noteworthy capabilities of the PR2, some of which we’ve seen before. A web-browser interface allows a user to order a beer through the robot, essentially sending commands to the bot via the internet. The robot identifies the various beer bottles while using its body to hold the door open. I’m most impressed that the robot can look for and be cued off of recognizing a human face – when the bot delivers the beer it waits for you to approach before it lets you pull the beer out of its hand. That ability is going to see a lot of use in the years ahead, and not just in deliveries. There are many applications where you want a robot to wait for and interact with humans, especially in medicine.

The Beer Me, Robot hackathon didn't use a typical fridge. Though, now that I think of it, maybe all fridges should look like this.

Often with these hackathons, engineers use a few cheats and have to leave in a few limitations. Clearly, the PR2 had to have some hardware add-ons to make beer delivery possible. Cupholders were added to the base, and the refrigerator was fitted with a sloped beer shelf. Those are fairly reasonable hacks, but very task-specific. Ultimately we want a humanoid robot to accomplish the vast majority of its chores without any modifications. We also want our beers not to be shaken so much that they spray all over us. But, y’know, I’m not going to look a gift robot in the mouth.

As with the other week-long engineering projects coming out of Willow Garage this summer, the Beer Me Robot endeavor is equal parts demonstration and innovation. Clearly it’s fun to watch a robot fetch someone a cold beer. However the real story is that ROS and the open source coding of the PR2 allow for developers to share and build off each others work. That enables the speed of the hackathon and insures that whatever usable code was created for this project can be recycled as needed in future hacks. One day the ROS library will be so full of helpful code that robot programmers will have to do little more than string together previous work. Which will be important, because by that time those programmers are probably going to be pretty buzzed.

Maybe the next hackathon should be: “I’m home. Take me drunk, Robot.”

[image credit: Willow Garage]
[video credit: Willow Garage]
[source: Willow Garage Blog]