Implanted electrodes in the speech center of the brain can communicate wirelessy via FM transmission with a computer. This allows a computer to inteprete brain activity into sounds using a speech synthesizer.

Remarkable news keeps coming for those who are trapped in their own bodies. People with locked-in syndrome, a condition where a healthy mind is unable to express itself due to brain damage, are slowly being opened up through direct contact with their motor neurons in the brain. Frank Guenther at Boston University’s Speech Lab has teamed up with Phillip Kennedy at Neural Signals to measure activity in the speech centers of the brain through implanted electrodes. These electrodes can then relay the information to a sub-dermal amplifier and then to a computer via wireless FM transmission. The results: a patient has demonstrated the ability to form rudimentary vowel sounds on a synthesizer using just his thoughts. It’s a small step, but research like this may one day allow someone to simply think of the words he wants to say, and have a computer do the talking for him. We have some videos of the wireless brain signal to speech test results after the break.

Brain-machine interfaces (BMIs), aka brain-computer interfaces (BCIs), are in development by several different teams across the globe. The Braingate project uses similar wireless transmission technology to connect electrodes in the brain to cursors on a computer, or even the controls of an electric wheelchair. Like many such projects, Braingate uses motor neurons to control movement. We’ve seen other teams work with robotic arms and prosthetic limbs. The Speech Lab/Neural Signals BMI is somewhat rarer because it is translating those signals which might inform mouth/tongue/vocal chord movement and directly interpreting them as sounds. This layer of interpretation is difficult to perfect but its pursuit gives us hope that one day we could see devices that actually “read” our thoughts and translate them into images, sounds, and other sensations. Once we achieved that level of “mind-reading”, there could be a direct conduit between our mental and digital worlds. Totally immersive virtual reality, surrogate bodies…the possibilities really expand at that point.

For now, research into turning thoughts into sounds is still at a rudimentary level. Neural Signal designed the hardware (electrodes, amp, receiver) and implanted it, but the Speech Lab had to develop the software routines to interpret the information into sounds. As described in the paper published on PLoS ONE, the joint team of scientists was able to produce a system that provided the patient with the means to make vowel sounds. A synthesized voice produced on a computer gave the patient auditory feedback so that he could hear how his “thoughts” were being translated and could focus on correcting them as needed. That feedback was remarkably fast, about 50ms, on par with the normal speed of talking. After practice, the patient’s ability to listen to vowel sounds and then repeat them improved from 45% to 70% (and beyond). In the following video from New Scientist, you can see how the patient is given audio promptings and then repeats the vowel sounds using his thoughts and the BMI. The second video, from Wired, gives you a better idea of what appears on the screen. You may want to turn down the volume before watching as the synthesized voice is fairly loud and atonal.

When I first heard about this project, I sort of expected to see a video with, you know, a little more pizazz. Maybe someone with a Darth Vader voice, or a Stephen Hawking look a like. The Speech Lab/Neural Signals BMI is at once less impressive and more important however. No, the patient isn’t making clear intelligible speech yet. No, the voice doesn’t carry human emotion. But he’s talking with wires in his brain, people! Wires in the brain and a little FM transmitter are all that connect this guy’s speech centers to the outside world. It’s not Darth Vader, but it still gives me chills.

For now, just a few electrodes and wires are used in the BMI. Eventually, patients could have 32 electrodes to provide better analysis and better speech.

Guenther and Kennedy plan to make some major improvements in the BMI in the next few rounds of experimentation. First, the number of implanted electrodes will increase from 2 or 3 up to 32. This will allow for the interpretation of neural signals that describe more complex mouth movements. Patients will be able to manipulate a “virtual tongue” with their brain activity and thus be able to form consonants (and hopefully full words). Also, the BMI currently requires lab computers to work, but the researcher team is hoping to move this functionality to a laptop.

As always, research like this doesn’t happen in a vacuum. The data gathered in this experiment will inform others and vice versa, hopefully accelerating the development process. Already we’ve seen how research into Broca’s Area in the brain is providing insight into how we speak. The Speech Lab/Neural Signals BMI, which also deals with Broca’s Area, could benefit from that research. This sort of basic neural research however, is likely to have a long lead-up time before we see tangible effects outside of the lab. In the short term, non invasive devices like Audeo, or the artificial larynx are more likely to make measurable improvements in a larger number of people’s lives. For those with locked-in syndrome, however, BMIs are really the only non-biological solution. As we learn how to release their minds into the outside world, we will find the key to unlock our own as well.

[photo credits: Speech Lab]
[video credits: New Scientists, Wired]

Braingate reads signals in the motor cortex and translates those signals into movements of a cursor on a screen.

The procedure for implanting Braingate may seem pure science fiction, but it works. Hair-thin gold wires are connected to individual neurons in the brain’s motor cortex. These wires are gathered at a small silicon array and connected to a “pedestal” embedded in the skull. This metallic interface is easy to spot (it’s a big metal nub on the top of the head). From the pedestal, signals can be sent to a computer for translation. By interpreting the motor cortex signals, scientists can determine what your brain would be trying to move (arm, hand, finger, etc) if you weren’t paralyzed.

So you have a metal nub in your head, and some wires poking into your brain, what’s the pay off?  How about the most intuitive mouse ever: by thinking about raising or lowering their hands, patients can move a cursor on the screen of a PC. Squeeze their imaginary hand, and the cursor clicks. The brain signals aren’t completely mapped out yet, and keeping track of one’s thoughts isn’t an easy task, so the cursor tends to jiggle a little and can be hard to move quickly. That being said, it allows individuals who have a hard time even blinking to be able to communicate with others and manipulate devices from their computer. Check out Kathy Hutchinson, one of the first patients, in this story from 60 minutes, the cable connected to her skull seems to be straight out of the Matrix:

Moving Right Along

This amazing technology is the work of many different collaborators, chief among them are Dr. John Donoghue from Brown University, who is also the head of Cyberkinetics, and Dr. Leigh Hochberg from Massachusetts General Hospital. While most of the videos you can find of these two are more than a year old, their work is still developing quickly. Dr. Hochberg began the long process of pilot clinical trials back in February of this year. With the clinical trials will come a better understanding of how to interpret motor cortex signals and increase the tasks able to be performed by patients. Cyberkinetics is already testing a motorized wheelchair, and has plans to develop methods for regaining breathing, bladder, and bowel control.

Expectedly, Braingate is well received publicly and Dr. Hochberg is seeking Investigational Device Exemption from the FDA. Like the Humanitarian Device Exemption given to other implants, this allows the research to continue with human trials quickly. And the technology is developing at a rapid pace. It was only 2005 when we first heard about the beginning experiments to map signals from the brains of rhesus monkeys. The next four to five years will likely see another flurry of development.

I think that this technology is on the brink of runaway growth and success. As Braingate moves forward and is refined, it is poised to mesh with dozens of other related technologies. Singularity Hub has shown you the prosthetic devices, robotic exoskeletons, brain controlled robots, and fMRI mind-reading systems already on the horizon. Soon, I think we’ll see a convergence of these various tools that, while developed separately, have a similar goal: allowing human thoughts to directly affect real-world objects. Once these technologies function better than normal human equivalents we will seem them transition from therapies to everyday utilities.

For now, Braingate returns a precious commodity: control. For many locked-in their own bodies, the best hope they had would be to communicate by blinking. Using a direct neural interface, these same patients have the prospect of writing letters for themselves and maybe even guiding their own wheelchairs. In the future, those prospects may expand to include walking with the help of an exoskeleton or commanding a helper robot. Without a doubt, brain signal technology is taking small steady steps forward every day. Like the Count de Monte Cristo, scientists are slowly digging an escape from the prison that these patients are held in. Together with their patients, they prove that even greater than the terror of being buried alive is the determination to one day be free again.