Foldit is an online game that rewards you for figuring out how proteins fold. Earn points and change science forever. Seriously.

When someone whines that your playing video games is a waste of time, tell them you’re doing it for science. Researchers at the University of Washington have successfully leveraged the power of gamers to solve a biochemical puzzle: the structure of a complex protein related to the development of AIDS. By playing an online game called Foldit, teams of average citizens were able to make a breakthrough discovery in how this protein was shaped even though scientists had stumbled over the question for more than a decade. A combination of computer-created predictions and human 3D spatial reasoning transforms simple game playing into a mighty problem solving engine. Their discovery could help develop new anti-viral drugs for HIV. Watch how Foldit is played in the video below. While applications are likely to remain limited for years to come, programs like Foldit demonstrate the vast potential of crowd-sourced solutions in science. Never doubt it – video games can be powerful tools for change.

Researchers from the University of Washington described their unique method for solving the structure of the AIDS related protein in a recent report published in the journal Nature Structural and Molecular Biology. The compound in question was Mason-Phizer Monkey Virus (M-PMV) a retroviral protease that affects how the HIV strains mature and replicate. Like so many proteins, M-PMV’s crystalline structure is determined by very complex interactions between the amino acids in the molecule, as well as preferences for parts of the protein to avoid or attract water in its environment. Predicting how proteins fold up is a computer-heavy problem, and various groups have explored distributed computing solutions so that people all over the world can donate processor time towards determining protein structures.

Foldit has a very similar goal, only it’s using your brain, not your computer to help it figure out how proteins fold. In the style of a game, Foldit players are rewarded points for finding solutions to protein folding problems. There are always many different possible solutions, so Foldit rewards players, or teams of players, based on various requirements set by the game designers in the University of Washington computer science department. Essentially players look at complex proteins and use their 3D spatial reasoning skills, as well as healthy doses of trial and error, to find the the folding pattern that gives them the most points. It’s up to the UW team to determine how to award points in a way that makes the gamers’ efforts most useful.

In this study, University of Washington researchers determined that there are some conditions in which Foldit doesn’t work very well. Sometimes players get stuck pursuing dead-ends, other times they try to tweak a possible solution when they really need to make big changes to find the ideal answer. The Foldit puzzles in this study gave players starting points (called Rosetta structures) as well as the means to manipulate portions of the protein and compare them to earlier (validated) solutions. By changing the starting points, and giving players more freedom in exploring the possible folding patterns, the University of Washington enabled Foldit teams to get very close to finding the M-PMV structure. The gamers’ answers were so good that UW could then step in and use more traditional computer driven methods to find the finalized solution to the problem. Scientists had been working on M-PMV for years, this new style of guided-gaming figured it out in just three weeks.

It’s tempting to call the Foldit participants “untrained” or “typical video game players” because that makes for a good story. The truth is, however, that part of what Foldit does is teach you how to solve protein folding problems. It just does it in a way that you’ll find fun and challenging. It’s not that the University of Washington isn’t training the thousands of players who participate in Foldit, it’s that the training is so vastly different than normal scientific education. You don’t have to know why some atomic bonds lead to lower energy states when arranged in new patterns, you just have to play with these twisted proteins until you get enough points to win.

In the short term, the combination of human and computer processing that the University of Washington demonstrated could be put to extraordinary use. Humans have 3D understanding that computers just can’t handle yet, and computers have the number crunching capabilities that humans can’t possibly compete with. Put the two forces together and you’ll find new avenues for anti-viral medications (as with the M-PMV case) or possibly solve any number of other biochemical mysteries. We’ve seen similar pairings of human and computer expertise suggested for things like image tagging. There’s a whole application space waiting to be explored by inserting large numbers of humans into a problem solving system when computer power alone doesn’t cut it. Video games are a great tool for leveraging the human mind, especially as massive multiplayer online games and other internet entertainment are teaching us to enjoy solving challenges in crowd-sourced environments simply for the sake of earning more points (and prestige).

Looking further down the line, however, intellect-harnessing video games like Foldit are only going to be useful as long as there are skills that human brains possess that computer processors do not. We’ve probably got several decades of that being the case, but eventually it will change. So for now we should really enjoy the power of video games. Used correctly they can transform even the most lackadaisical layabout into a mighty instrument of science. So your mother was wrong: you’re not a coach potato, you’re a biological processor that really enjoys its work.

Speaking of which, there’s some scientific work I really need to get done. Anybody know the cheat codes to Ebola?

[image and video credit: FoldIt]
[source: Khatib et al Nature Structural and Molecular Biology 2011, University of Washington News]

Willow Garage announces it's first sales for their PR2 robots. Get dressed boys, your new parents are ready to take you home!

More institutions are buying into Willow Garage‘s vision for accelerating the field of robotics. The Silicon Valley startup announced that it recently sold its PR2 robots to four new research groups. Willow Garage has already shipped new bots to Samsung’s resarch center in Suwon, South Korea, and the University of Washington in Seattle. Soon others will arrive at George Washington University in DC and the CNRS Laboratory of Analysis and Architecture of Systems (LAAS-CNRS) in Toulouse, France. Quite the global spread. At $400,000 each these robots aren’t cheap, but they come backed by an ever expanding collection of professional quality open source software via the Robotic Operating System (ROS) library – a free resource that can be utilized by many different robots, including the PR2. With code found on ROS, the PR2 robot has been programmed to accomplish an incredible variety of tasks. Willow Garage highlights these successes in their newest video montage shown below. Four more world-class research institutions adopting the PR2 platform and ROS mean that the global community of mutually supportive robotics engineers are receiving a valuable boost.

When Willow Garage announced it would begin sales of the PR2 back in September it also described a sharp discount (down to $280,000) for institutions with a history of open source contributions. It’s no surprise then, that some of the four new buyers (Willow Garage will not comment on which) will be receiving their new robots on the cheap (relatively speaking). Why would Willow Garage voluntarily cut into its profit margins? Community building is more important. Earlier this year, the company gave away 11 of its PR2 robots for free simply to help enliven the industry and generate more innovations for its platform (known as the PR2 Beta Program). There’s little doubt that Willow Garage is taking a long term vision for its own growth, and it knows it can create the personal robot of the future much faster if the entire global collection of robotics engineers work together.

The medium for that cooperation is ROS. The ROS library recently turned three years old, and its collection of code packages and robotics algorithms has been growing at an exponential rate. Partly that growth is due to the work of Willow Garage engineers and their constant additions to the library. Mostly, however, it’s due to the ever expanding group of research institutions that are using ROS. Because it is open source, engineers can download, use, update, and upload ROS code, and adapt it for any number of different robot platforms. Samsung, University of Washington, George Washington University and LAAS-CNRS will be able to jump right into major development because so much of the groundwork in hardware and software has already been created. According to an engineer at Samsung, “we were programming on and navigating the PR2 in less than one day.”

There are more than 50 bots that use ROS, but the PR2 is definitely the flagship of the group. It has advanced sensors, hands, and stability that give it a wide range of possible applications. In the following video you’ll see the PR2 fold towels, fetch beers, and perform dozens of other tasks. Many of these clips show work performed at Willow Garage, but you’ll also see results from the 11 teams that received PR2s during the Beta Program. Make sure to watch at 2:18 where you’ll see a great preview of a new project featuring the XBox Kinect 3D sensor:

So if the PR2 and ROS are such an awesome combination, what are the four new teams going to do with them? According to Willow Garage, Samsung will be using its purchase “to enhance their existing robotics research.” That’s a little vague, but as WG points out, Korea is looking to put a robot in every home by 2020 – we should expect some exciting developments in personal robotics out of Samsung in the years ahead.

Professor Joshua Smith received the PR2 at the University of Washington on November 2nd. Smith has a history of working with sensors, and finding new ways to power them. He was recently at Intel working on wireless energy links. It should be interesting to see what he can do with the PR2′s collection of IR sensors, lasers, and cameras.

Rachid Alami, part of the Robots and Artificial Intelligence group, will be receiving the PR2 at LAAS-CNRS. According to Willow Garage: “Alami and his colleagues have earmarked the PR2 for the development of high level interactive and cognitive functions in the context of an ambient intelligent system for assistance, such as housekeeping for seniors.”

At George Washington University, the PR2 will be greeted by Evan Drumwright. His interest is to get robots performing occupational tasks. We’ve seen many developers looking for ways to incorporate robotics into unexplored realms of manual labor, and Drumwright is no exception. Willow Garage tells us we should look to Drumwright for “advances in dynamic robotic simulation, motion planning, and collision detection algorithms.”

Currently there are just 16 research institutions with PR2 robots around the world (4 new buyers, 11 Beta Program recipients, and Willow Garage themselves), but the group of ROS developers is much larger, and the general open source robotics community even larger still. Every team that contributes to the open development of robotics helps accelerate the growth of the industry as a whole. The four new entrants have valuable skills that could strengthen the open source robotics community and deepen the level of tasks that personal robots can accomplish. Even if none of the new PR2 recipients contributes a single packet to ROS, however, their purchase of the robot alone adds to Willow Garage’s clout. Bottomline, one way or another these sales are a good sign that open source robotics is working. With any luck, open robotics will keep growing and the shared code will continue to pile up exponentially.

[image and video credits: Willow Garage]
[source: Willow Garage]

A small humanoid robot (left) is given general commands based on brain signals of a user (right).

Mind controlled robots, wheelchairs, or cars… the difficulty really comes from the mind-reading, not the automation. While ASIMO’s venture into brain-control had users make direct commands (lift left arm, stick out tongue, etc), and Braingate directly measures motor neurons, Rao’s team takes a broader approach to mind-control. Surface sensors measure a very narrow range of brain activity and basically just report which of several objects/locations you show interest in. This command-level approach is less sensitive than the other systems (it also was developed years earlier), but it has important implications. When we see robots directly controlled by human minds (as in the movie Surrogates), we are shown a direct thought to action connection. I want the arm to lift, the robot lifts the arm. But what if you just thought: “I want that ball” and the robot handled the arm lifting and grasping on its own? Precision is important, but directly controlling all the myriad functions of a robot may be too difficult for many users. After all, many of us have coordination problems in our own bodies.

The video lacks sound, so here’s a quick play by play of what you’re seeing:
0:05 – The user is hooked up with non-invasive surface sensors that read brain activity.
0:25 – The robot identifies two objects on the table.
0:56 – The user is shown both objects, with a box alternatively flashing around each. Brain activity peaks when the flash is perceived, thus the system knows which object is being actively focused on.
1:15 – After the selection is interpreted, the robot proceeds to approach and lift the chosen object. (This takes a while).
2:00 – Again, using flashes to record the user’s focus, the robot is told to place the block on the blue platform with the gray square.
2:45 – Success!

[photo and video credit: Rajesh Rao, University of Washington]

Augmented Reality is getting much closer. This contact lens has embedded metallic circuits that could one day be used to project images directly onto your eye.

When you drive your car your dashboard instruments display the speed, amount of fuel left, and distance traveled. You can use Google Maps on your smart phone to find restaurants, post offices, or other important landmarks all around you. Why can’t this sort of information be given to you all the time, streaming directly into your field of vision on a contact lens? That’s the question University of Washington Prof. Babak A. Parviz asks in his recent letter to IEEE Spectrum. Parviz and his team have been developing miniature circuits and simple LED displays and integrating these elements onto a contact lens-like polymer. They’ve tested them on rabbits who can wear the devices without harm. As Parviz points out, introducing Augmented Reality onto a contact lens is just a matter of time and effort.

Augmented Reality applications are starting to crop up everywhere, from table top games, to toy store displays. AR allows digital images and information to be blended with streaming video in real time. Using a computer screen or TV, the digital world and the real world overlap before your eyes. Pavik wants that statement to be literal: the overlap should be right before your eyes. As electronic elements become smaller and able to function at ultralow power, there should be little reason why electronics can’t function directly on your body. AR contact lenses could also record information too. Biosensors on your eye would be able to monitor your health and then display that data to you through the AR interface. These devices would not only grant you increased knowledge, they could lead to bionic eyesight. High powered cameras could beam their recorded images directly onto your eyes through the lens, letting you see further, sharper, better.

Well what do we have here?

To date, Parviz’s team has made some great first steps into proving that the AR contact lens concept is possible. Here’s a quick summary of the accomplishments they list in the IEEE article:

• Parviz has developed several micro circuit components suitable for use on a contact lens. These include: control circuits, power circuits, communications circuits, an antenna, and a LED. That LED was activated to provide a single pixel in the field of vision.
• All of these elements consisted of semitransparent hardware that was embedded onto a polymer or glass that emulated a traditional contact. The AR contact prototype was then embedded in a biocompatible polymer to keep toxins inherent in semiconductor materials from spreading onto the eye.
• The entire structure was tested on a rabbit eye for 20 minutes without ill effect and the LED was lit.
• The team has also developed a simple biosensor that could be used to determine blood glucose levels by monitoring the eye’s surface.

Parviz and his team have already started developing ways to get integrated circuits onto a contact lens.

Yet these are just the first steps to actually creating an AR contact lens. That single LED is just one pixel, ideally you would want many more. According to Parviz, the useful image area of a device directly on your eye is about 1.2 mm. The current LED setup is about 300 microns which creates a single pixel that is a donut 112 microns in diameter and 60 microns thick. The goal is to create an array of 3600 pixels each 10 microns in diameter and 10 microns apart.

Once you generate an LED screen, you still have to get your eye to focus on it. Parviz reccommends the use of several thin micro-lenses that would fit with the AR contact. Essentially you would have just a few more layers of polymer that act as a telescope to get the LED image correctly seated on your retina.

Powering a contact is going to be interesting. Obviously a wireless system would be best. The University of Washington team created an RF circuit that could pick up energy broadcast from a battery pack in the 900 MHz to 6 GHz bandwidth – ideal for not frying your body with power. Parviz also suggests that a miniature solar cell could provide energy for the contact. Either way, the amounts of power supplied would be small (100 microwatts through RF, maybe 30 microwatts from the solar cell). The good news is that very little power is really needed. A LED situated directly on your eye shouldn’t be very bright in order to be work. In fact, we could see a passive display (light is blocked instead of generated) work just as well as an active one. A passive system would work much like an LCD screen, where the ambient light of what you were looking at would function as a backlight.

What I find exciting about Parviz’s work, and his ideas for the future, is that the AR contact is an ideal level of technology for many users. While we could one day have images broadcast into our brains, or enhance our eyes through genetic engineering, I find those concepts a little alien and uncomfortable. What happens if someone hacks your brain feed, or if there’s a problem with the genetic manipulation? AR contact lenses could provide bionic eyesight, an integration of digital and real world images, and health monitoring as effectively as these other hypothetical systems. And if anything goes wrong, or if you want to upgrade to the newest technology, well, you just take the contact off and replace it.

As Parviz points out, his team still has many hurdles to overcome. They’ve tested circuit elements, but they haven’t actually created the integrated chip they would need for AR. They’ve lit a single LED, but they haven’t actually generated an image. There are many more parts of the puzzle that have to be created and then miniaturized to fit in the tiny space allotted. It will likely take years if not decades for these engineering problems to be solved. Still, I have high hopes for the technology. Parviz has a healthy National Science Foundation grant and I think the demand for the kind of device he describes is great enough to encourage others to enter into the field as well. Like the 100 million other contact wearers out there, I put plastic on my eyes every morning to improve my vision. Someday, with the help of AR technology, I hope to improve it even further.

[photo credits: Parviz Research Group, University of Washington]