A new brain chip that allows monkeys to control a virtual hand and sense touch may be the breakthrough needed to give paralyzed patients the protheses control they need.

Scientists have created a brain implant that not only allows monkeys to control a computer with their thoughts, it also allows them to “feel” the virtual objects. The new, two-way, brain-machine-brain interface represents a major breakthrough in the field of neuroprosthetics. Bolstered by the results, the scientists plan to test the technology on a quadriplegic in just three years.

There have been a number of advances in brain-machine-interface (BMI) research in recent years. Implants have allowed monkeys to control computer cursors and even a robotic arm with impressive precision. In the current study, two macaques were trained to control a virtual arm on the computer screen and use it to “grasp” virtual objects. What separates these macaques from past BMI trainees is that, when their virtual hands contacted the virtual objects they were able to “feel” the objects.

The macaques were shown three identical objects on the computer screen. They were trained to pass the virtual hand over the three objects and to choose the one that evoked a sensation. Controlling the hand was enabled by electrodes implanted into the motor cortex, a major part of the brain for movement control. First, the activity of hundreds of neurons in the region is recorded while the monkeys control the virtual hand with a joystick. In this way the computer learns what “left” and “right,” etc. means in terms of brain activity. Then the joystick is taken away, and the monkeys are trained to control the hand with their thoughts.

The sensation indicating the correct object caused by electric stimulation through a return connection from the computer to the brain. These electrodes were inserted into the somatosensory cortex, the part of the brain that senses touch. As the virtual hand passed over the correct object the somatosensory cortex was stimulated. If the monkeys chose the correct object they received a food reward. They learned the task quickly. One monkey showed improvements after nine trials, the other after only four. After mastering the task the monkeys were picking the right object nine times out of ten. This shows that they can sense and object without any stimulation to the skin.

The following video shows the avatar arm in action.

“We don’t know what the animals perceived,” Miguel Nicolelis, neuroprosthetic aficionado and lead author of the study, told Nature, “but it was a sensation that was created artificially by linking the virtual fingers to the brain directly.” The study was published in Nature October 5th.

Managing the two signals – out from the motor cortex and into the somatosensory cortex – proved tricky for the researchers. The two areas are close enough together in the brain that the electrical stimulation in the somatosensory cortex would actually leak over to the motor cortex and disrupt control of the virtual hand. They got around this by alternating between recording and stimulating every 50 milliseconds. Of course, normal brain activity usually doesn’t take turns, but the fact that the monkeys were still able to learn the task means the constrained methodology still worked.

To give prosthetics sensory feedback would be huge. Some labs are currently trying to develop prosthetics with sensory feedback functions. One example relays feedback signals through the stump ending nerve terminals. Due to variability in the health of the stump tissue and severed neurons, this approach is less than idea. The two-way communication of BMBI, straight to the brain’s pristine movement and sensory centers could make feedback a reality. If BMBI can be used effectively in humans – a big ‘if’ – it would surpass current BMIs that up till now have relied solely on visual feedback for control. “If you want to reach and grasp a glass, visual feedback won’t help you,” Quian Quiroga, a neuroscientist at the University of Leicester, UK said in a commentary in Nature. Quiroga, who was not involved in the study, added, “It’s the sensory feedback that tells you if you have a good grip or if you are about to drop it.”

The Walk Again Project exoskeleton will be controlled entirely by thoughts.

With the successful BMBI demonstration, Nicolelis and his colleagues are ready to take neuroprostheses to the next level. The Walk Again Project is a collaborative effort involving many scientists from the world’s leading research centers to, through BMI/BMBI technology, do no less than “restore full mobility to patients suffering from a severe degree of paralysis.” Of course, to achieve such a superhuman feat will require superhero suit. In collaboration with Walk Again, Nicolelis’ lab at Duke University are putting together a “wearable robot” or “exoskeleton” designed to support the patient’s body and allow he or she to move at will. As in the study, electrodes will be implanted to give the patient both movement control and the all important feedback. The electrodes will be chronically implanted in at least five different regions on each side of the brain. Whereas a few hundred neurons were sufficient for the macaques to control their virtual arms, the larger human brain requires that a couple thousand brain cells be recorded for proper movement. It’s a major challenge that the researchers will have to overcome before patients can begin donning the exoskeleton. The electrodes will have to be stable for up to a decade, and then there’s always the risk of infection.

The brain chips – if they work – will be a technological triumph by themselves. Custom designed, the brain chips will be low-power and wireless, transmitting their signals to a processing unit worn on the patient’s belt about the size of a cell phone. That brain activity will then be translated to digital motor signals which will control the actuators across the joints of the exoskeleton. Force and stretch sensors throughout the exoskeleton will signal back to the patient’s brain the whereabouts of his or her joints and limbs. They predict that it would only be a matter of weeks before the patient was used to the suit, off on his or her own, experiencing the world like never before.

It’s an ambitious project, to say the least. The scientists are currently working hard to ready the suit for clinical trials over the next three years. You see, they have a deadline: the 2014 World Cup in Brazil. Nicolelis, a native Brazilian, hopes to test the exoskeleton on a grand stage by having a young quadriplegic Brazilian deliver the opening kick.

[image credits: The Guardian and Walk Again Project]
[video credits: DukeMedicine via YouTube]
image 1: Virtual arm
image 2: exoskeleton
video: BMBI

Implanted electrodes in the speech center of the brain can communicate wirelessy via FM transmission with a computer. This allows a computer to inteprete brain activity into sounds using a speech synthesizer.

Remarkable news keeps coming for those who are trapped in their own bodies. People with locked-in syndrome, a condition where a healthy mind is unable to express itself due to brain damage, are slowly being opened up through direct contact with their motor neurons in the brain. Frank Guenther at Boston University’s Speech Lab has teamed up with Phillip Kennedy at Neural Signals to measure activity in the speech centers of the brain through implanted electrodes. These electrodes can then relay the information to a sub-dermal amplifier and then to a computer via wireless FM transmission. The results: a patient has demonstrated the ability to form rudimentary vowel sounds on a synthesizer using just his thoughts. It’s a small step, but research like this may one day allow someone to simply think of the words he wants to say, and have a computer do the talking for him. We have some videos of the wireless brain signal to speech test results after the break.

Brain-machine interfaces (BMIs), aka brain-computer interfaces (BCIs), are in development by several different teams across the globe. The Braingate project uses similar wireless transmission technology to connect electrodes in the brain to cursors on a computer, or even the controls of an electric wheelchair. Like many such projects, Braingate uses motor neurons to control movement. We’ve seen other teams work with robotic arms and prosthetic limbs. The Speech Lab/Neural Signals BMI is somewhat rarer because it is translating those signals which might inform mouth/tongue/vocal chord movement and directly interpreting them as sounds. This layer of interpretation is difficult to perfect but its pursuit gives us hope that one day we could see devices that actually “read” our thoughts and translate them into images, sounds, and other sensations. Once we achieved that level of “mind-reading”, there could be a direct conduit between our mental and digital worlds. Totally immersive virtual reality, surrogate bodies…the possibilities really expand at that point.

For now, research into turning thoughts into sounds is still at a rudimentary level. Neural Signal designed the hardware (electrodes, amp, receiver) and implanted it, but the Speech Lab had to develop the software routines to interpret the information into sounds. As described in the paper published on PLoS ONE, the joint team of scientists was able to produce a system that provided the patient with the means to make vowel sounds. A synthesized voice produced on a computer gave the patient auditory feedback so that he could hear how his “thoughts” were being translated and could focus on correcting them as needed. That feedback was remarkably fast, about 50ms, on par with the normal speed of talking. After practice, the patient’s ability to listen to vowel sounds and then repeat them improved from 45% to 70% (and beyond). In the following video from New Scientist, you can see how the patient is given audio promptings and then repeats the vowel sounds using his thoughts and the BMI. The second video, from Wired, gives you a better idea of what appears on the screen. You may want to turn down the volume before watching as the synthesized voice is fairly loud and atonal.

When I first heard about this project, I sort of expected to see a video with, you know, a little more pizazz. Maybe someone with a Darth Vader voice, or a Stephen Hawking look a like. The Speech Lab/Neural Signals BMI is at once less impressive and more important however. No, the patient isn’t making clear intelligible speech yet. No, the voice doesn’t carry human emotion. But he’s talking with wires in his brain, people! Wires in the brain and a little FM transmitter are all that connect this guy’s speech centers to the outside world. It’s not Darth Vader, but it still gives me chills.

For now, just a few electrodes and wires are used in the BMI. Eventually, patients could have 32 electrodes to provide better analysis and better speech.

Guenther and Kennedy plan to make some major improvements in the BMI in the next few rounds of experimentation. First, the number of implanted electrodes will increase from 2 or 3 up to 32. This will allow for the interpretation of neural signals that describe more complex mouth movements. Patients will be able to manipulate a “virtual tongue” with their brain activity and thus be able to form consonants (and hopefully full words). Also, the BMI currently requires lab computers to work, but the researcher team is hoping to move this functionality to a laptop.

As always, research like this doesn’t happen in a vacuum. The data gathered in this experiment will inform others and vice versa, hopefully accelerating the development process. Already we’ve seen how research into Broca’s Area in the brain is providing insight into how we speak. The Speech Lab/Neural Signals BMI, which also deals with Broca’s Area, could benefit from that research. This sort of basic neural research however, is likely to have a long lead-up time before we see tangible effects outside of the lab. In the short term, non invasive devices like Audeo, or the artificial larynx are more likely to make measurable improvements in a larger number of people’s lives. For those with locked-in syndrome, however, BMIs are really the only non-biological solution. As we learn how to release their minds into the outside world, we will find the key to unlock our own as well.

[photo credits: Speech Lab]
[video credits: New Scientists, Wired]

Above: Closeup shots of the head gear that reads blood flow in the brain and electrical signals

Above: An individual uses thoughts to send four simple commands to a nearby Asimo robot

Honda has demonstrated a person sending four simple commands to the robot simply by thinking. The commands are “lift right hand”, “lift left hand”, “move legs”, and “stick out tongue”. Pretty cool you say? Not really! Earlier this year we reported on a team at Carnegie Mellon that is able to read substantially more than four thoughts from an individual with at least the same accuracy as Honda is claiming.  The Carnegie Mellon team was not using these extracted thoughts to control a robot or anything else, but they easily could have.  The hard part is extracting thoughts from a person’s head.  After the thought is extracted it is a simple matter to automate an action based on these thoughts.

Not only have we already seen better non-invasive thought extraction than in the Asimo demonstration, but also earlier this year we saw something far better: invasive, direct access to the brain through implanted electrodes.  In a previous story we saw that a monkey implanted with electrodes was able to control the walking movement of a robot simply by thinking about walking.  The monkey demonstration was far more advanced than the Honda demonstration, allowing for a modulated spectrum of commands to be sent, such as slow down, speed up, and stop.  This goes far beyond sending a simple command such as “walk”.

The stories referenced above are just the tip of the iceberg.  All across the world researchers are rapidly breaking down the barrier between our once private thoughts and the outside world.  Of course we are a long way from completely reading someone’s entire mind, but on the flip side reading four simple thoughts from a person’s mind is already old news.

Don’t get us wrong about Honda though!  We are thrilled to see Honda joining the brain interface party with investment, ideas, prototypes, and more.  Honda’s success is no small feat and is a valid contribution to the field.  But we also want readers to have some perspective of where the Honda demonstration fits in with what the rest of the industry is doing.

The field of interfacing with the human brain is literally exploding.  It is going to be an amazing journey and the rewards will be great.  Honda says they want to “open the car trunk simply by thinking”.  We say forget the trunk!  How about allowing a paralyzed person to walk again once we can extract their thoughts and re-route them to a pair of robotic legs.  Now that is what we’re talking about!