That's Not a Horse! It's Just a Man Behind You With Two Coconuts!

For those in the non-invasive camp, however, it will be a bit of a wait (check out Braingate for the status of invasive techniques). As is proven with the Emotiv Epoc, the technology is not quite there. Thought controlled computers are still a bit of a chore for the able-bodied but, for the disabled, are already making quite an impact. The newest gadget in personal mobility is the thought-controlled wheelchair, and this iteration seems to have quite a complex navigation system. Rather than go with the smile forward, blink to turn method that Cuitech Inc. took with the Epoc-equipped chair, researchers at the University of Zaragoza adopted a method similar to the brain-Twitter interface.

The user focuses on a point onscreen and, as the point blinks, the headset can determine at which area of the screen the user is looking. For the wheelchair interface, a laser is used to scan for obstacles and the user is given options by way of a 3-D map on the computer screen. Although the wheelchair is limited to about two processed thought-commands per minute, the route is already planned into the chair’s navigation system, so there is not much need for more intense user input. The slow input time is presumably due to the accuracy of the computer system, where because the electrical impulses that the headgear measures are so small, many measurements need to be taken for the same action. Take a look at the video for a demonstration.

As the technology improves, researchers are hoping that they will be able to process more commands in less time with the same accuracy. This would allow directional changes on the fly, rather than picking a pre-charted destination. The laser system that scans for obstructions also boasts a crash avoidance system, which allows for safe travel in the vehicle even though it is relatively slow to respond to driver instructions.

So far, the system has been tested on able-bodied and disabled people with great success. Currently, use of the wheelchair is limited to about two hours because the conductive gel that is used to create a sound contact between electrode and scalp begins to dry out. Researchers are looking into a more long-term method of increased conductivity, allowing the chair to be used all day.

Technologies such as these are the first step in producing a thought-controlling society. Even though two commands per minute or 10 tweeted characters per minute may seem slow to most, those who do not have the luxury of such rapid communication and transit can now benefit. Even this technology, however, may just be an interim solution until the invasive procedures are perfected, allowing a direct linkage between the brain and the computer. Nobody that we know can tell the future, but our tea leaves are telling us that the brain will eventually be wired directly into a computer through an invasive solution. The only question is: how soon until we get there?

Now only in their infancy, devices that can read people’s minds are on track to improve dramatically in the coming decades.  These devices may open the doors to lie detection and telepathic communication, even as they challenge our already weakening boundaries of personal privacy.  Bolstering this vision is our recent story documenting the first time that a defense has attempted to bring lie detection evidence from a brain scanning device to a courtroom in San Diego.

Scene from the Signs Movie

In an effort to learn more about the field of reading minds we setup a chat with researchers at Carnegie Mellon University (CMU) where some of the most cutting edge work in the field is being performed.   We spoke with Tom Mitchell, founder and Chair of CMU’s fabulous Machine Learning Department, Marcel Just, Co-Director of CMU’s Center for Cognitive Brain Imaging, and Mark Palatucci, Ph.D student within the CMU Robotics Department.  Below is a summary of our conversation:

How it works

fmri brain scanner: one day we will laugh at how big it is!

Using fMRI brain scanners, the team at CMU is able to determine the activation level of roughly 20,000 three dimensional regions within the brain when a person is thinking about something.  These three dimensional regions, called voxels, are logical cubes of neurons roughly 3 mm on a side. When we think of an object like an apple or a chair, only a subset of the brain’s 20,000 voxels light up with high oxygen levels, revealing a neural activation pattern that is unique for that word.

fMRI is non-invasive, meaning the entire procedure can be performed without having to go inside your brain.  The limits of fMRI based mind reading are unknown, but results are already impressive.  Whatever  the limits of fMRI turn out to be, we can always take things a step further by connecting electrodes directly into the brain,  a vibrant area of research that is being closely followed here at the Hub.

The Truth, Please

One of the first things we wanted to discuss with the CMU team was the controversial fMRI based lie detection case in San Diego.  To our great surprise they were reluctant to discuss the matter.  Although their research deals with reading people’s minds,  the researchers at CMU say that detecting lying is very different from detecting what a willing subject is thinking.

“Lying is a completely different ballgame”, says Just regarding the issue.

“This is an empirical question that we will know much more about in the next decade.” says Mitchell.  “People who dismiss it as obviously impossible are not thinking carefully, and those who are embracing it as already solved are not thinking carefully either.”

We Really Do Know What You Are Thinking

Lie detection aside, the team at CMU has achieved some very serious success in reading the minds of willing participants.  Palatucci describes his own work as follows:

“Basically, I’m trying to train a program that predicts what specific word a person is thinking of, rather than just the high level category of that word. I’ve found that in some cases, it’s actually possible to predict certain words, even at this fine grain level from a large set of possible words that the computer could predict.”

One of the most amazing findings of this work is that the brain activity of a person thinking about an object, such as a hammer, is very similar to the brain activity of a completely different person that is also thinking of a hammer.  People grow up in different places and have completely different experiences, yet thoughts of common objects, such as hammers, seem to be held within our brains in a representation that is roughly consistent among all of us.

The CMU researchers are making impressive strides in determining what word you are thinking of if their program has been trained to know how to look for that word.  But what about words that their program has not been trained on – words the program has never seen before?  Believe it or not the team at CMU is making progress on this task as well.  Give the program a word it has never seen before and the program can make a prediction of what the fMRI neural pattern of your brain should look like when you are thinking about that word.

It seems like an impossible problem at first: how is a program supposed to know what the neural activation in your brain should be for a word that it has never seen before and has no understanding of?  The answer turns out to be pretty cool: give the program a large sample of human language to sift through (say 1 trillion words or so!) and soon the program gains a basic understanding of what the word “means” based on its co-occurrence with other key words.

“Take the word briefcase“, says Mitchell.  “In the English language the word briefcase is likely to be found near words such as ‘lift’ or ‘hold’ and not so likely to be found near words such as ‘eat’ or ‘sleep’.  By recognizing these types of associations a program can quickly assemble a reasonable understanding of the word.  The program will realize that briefcase is similar to the word purse and not so similar to something like apple.

Even assuming the program can figure out what a word means, how is it then supposed to predict the neural activation in your brain for that word?  Is it really true that the neural activation for the word briefcase is similar to the neural activation for the word purse, or is the neural activation for each word in our brains stored in its own completely unique activation pattern?  Amazingly the team at CMU has discovered that similar words or concepts, such as purse and briefcase, are indeed stored in the brain with similar neural activation, using many of the same voxels within the brain, but varying in the intensity of the voxels’ activation level.

Discovering How The Brain Works

The implications of this discovery offer far reaching possibilities in attempts to unlock the secrets of the brain.  The brain is an almost infinite web of interconnections, a massive spaghetti of neurons, synapses, dendrites, and so forth.  If we have to analyze each one of these individual connections to reverse engineer how the brain works we may be in for a long and grueling ride.  Yet could it be that to understand the brain we don’t need to go down to this level of granularity after all?  Perhaps all we need to understand how the brain works is to look at higher level pieces, such as  the voxels that the CMU team uses to read minds.  Mitchell and Marcel seem to think so, and they have described their idea with an analogy:

Architects work with rooms and hallways when they are designing a building.  They don’t look at the individual bricks.  When looking at the brain we can think of neurons as the bricks.  We can look at clumps of neurons, or voxels, as the hallways and rooms.  Unraveling the mysteries of the brain may not require us to analyze it neuron by neuron.  Perhaps all we need to crack the secrets of the brain is to understand the voxels and other higher level components.

Taking the architecture analogy one step further, if we can learn the architecture of the brain well enough to read people’s minds, wouldn’t this perhaps bring us within shouting distance of recreating human intelligence?  With this thought in mind, I asked Just if he thought mankind would create an artificial intelligence that will  match or exceed human ability in the next 50 years.  We close with his answer paraphrased below:

The best chess playing programs already exceed human players, so if it’s just quality of performance then in some arenas computers are already better.

My late colleague Herb Simon used to talk about how aircraft are artificial birds.  In many ways aircraft exceed birds in their ability to fly, but in many other ways they do not.  The path to recreating human intelligence may deliver a similar outcome.   Just as birds and aircraft are similar but different, we may create artificial intelligence in the future that mimics human brains, yet also differs greatly in its implementation and capability in a variety of arenas.