Jon Rodriguez and Erick Miller are the crazy-like-a-fox founders of Vergence Labs.

There are few fuels as potent as youth and genius, and Vergence Labs is sitting on a powder keg. The fledgling startup in Palo Alto is comprised of Erick Miller and Jon Rodriguez a dynamic duo that want nothing less than to redefine the human-computer interface for the 21st Century. Vergence Labs, recently accepted into Stanford’s prestigious StartX program, is looking to create commercially available devices which will bridge the space between augmented reality, biometric data, computer vision, and digital information in the next few years. They’ve already produced demos for an augmented reality visor that will tell you the names of your friends as you look at them (see video below). Their first real product, however, will be a pair of glasses that allow users to capture video and share it socially, and they’ve just launched a Kickstarter project to help start production. Soon thereafter they’ll be integrating biometric data from wearable sensors, and moving on to bigger, better ways to merge humans and machines.

I talked with Miller and Rodriguez about the future of their new company. Augmented reality, immersive digital/social experiences, telemedicine, preventative health care monitoring…Vergence Labs has so many potential applications for their devices lined up that it’s not really a big concern. These guys don’t want to find a killer app, they want to build the bridges that will unite us with computers. Or, as Miller says, “Reality is our platform.”

Singularity Hub first ran into Vergence Labs at this year’s FutureMed at Singularity University, discussing some of the many possible applications for human-computer interactions in the healthcare industry. It’s hard to convey in words the enthusiasm and surprising depth of vision that Miller and Rodriguez provide in person, so a few videos are in order. The first is an introduction from Vergence Labs that quickly jumps through some of their concepts of augmented reality devices, and then demonstrates a great prototype facial-recognition system that culls info from Facebook to identify friends as you look at them.

In the second Vergence Labs video Rodriguez presents another video glasses prototype that augments reality in a few fun ways:

The latest Vergence Labs creation: electric sunglasses. A nice little demonstration of the hardware expertise in this team:

There’s a breadth to Rodriguez’s experience which is astounding for someone so young. Besides robotics, AR, and social media (a former intern at Facebook), he’s also worked with synthetic biology, DNA sequencing, and mobile apps. His honor’s thesis at Stanford this year (yes, he’s a college senior) will focus on harnessing wearable sensors to create a bioelectric interface, “assembling an augmented reality computer with video screens and headphones as output, and inputs consisting of gaze tracking, EEG, finger tracking, and GPS.” He’s most of what one could ask for in a Chief Science Officer and he hasn’t even finished his undergraduate degree.

Here’s Rodriguez, while still in high school, discussing a system to adapt reconnaissance robots with IR vision for military applications. His work here won him an award at the 2008 International Science and Engineering Fair.

Miller’s career is no less impressive. His experience in computer graphics, computer vision, design, and visual effects includes time with some big names like Walt Disney Animation Studios, and Sony. Fantastic Four: Rise of The Silver Surfer, 300, X-Men: The Last Stand, Beowulf, I Am Legend – Miller’s work or leadership is seen in all those films. But he’s not just some movie guy, he’s a recent MBA with a great background in shaping digital worlds ready to take a bigger role in deciding how people actually experience reality.

Why am I spending so much time on Miller and Rodriguez’s backgrounds? Because when you start talking about Vergence Labs it’s easy to get bogged down in the short term applications, and there’s a lot more going on here. Vergence Labs’ first product is going to be a set of camera glasses that share video through social media – that’s a space ZionEyez has tried to enter (I’m still waiting to see if the former Flip camera crew can pull that off). The first demo we see of the Miller-Rodriguez team is a prototype device that uses Facebook to recognize people in real-time – KLIK from Face.com is already in that market with a mobile app. The augmented reality goggles and software that Rodriguez shows off in the second video are fun, but Vuzix hardware, and a variety of different software programs have been available for years. Not only that, but Google just starting showing off their Google Glasses concept and the search engine giant looks like it might try to dominate the field in the next few years. Social video eyewear is going to be competitive to say the least.

So if one just looks at Vergence’s near term projects as evidence one would be tempted to say: “cool, a couple of smart guys are working on some of the neat visual tech I’ve seen before, maybe they’ll help provide some good competition to other companies.” The real message to be received however is “Vergence Labs could be the breakout company that merges our digital and physical senses”.

Because it’s about more than just cameras and augmented reality mobile apps. Vergence Labs is aiming broader in data collection – not just video and audio, but synchronized biometric sensors. In the short term that means that a Vergence system could have applications beyond social media, including preventative healthcare, body monitoring, and personal medicine. Miller threw out tons of great apps in this space including glasses that use computer vision to see and track what you eat, even suggest when you should eat something else and sensors that correlate what you do (as seen in video) with how your body feels, sleeps, or moves. Rodriguez’s honor thesis suggests even more possible ways of transforming a wearable computer (like the glasses and sensor) into a “Total Biolectric Interface” that can provide information the user needs to react to the environment around them. Imagine a pair of glasses that hears everything said to you, converts it from speech to text, searches for keywords, and then overlays graphics and text that you can select using just eye movements – hyperlinks to your reality.

Those sort of long term applications hint at the kind of platforms that Vergence Labs wants to create: observant, immersive, intelligent products that unite physical experience with the growing database of digital information available online. Everything is synched through the cloud, everything can be shared, and everything you want to know can be in front of your eyes in just seconds. Their first products are low-hanging fruit – real time social video through web and mobile computing, augmented reality, and basic body monitoring. Give them a few years of success, however, and Vergence Labs could be creating the prototype devices that just scream of the Technological Singularity and the merger of man and machine.

Miller showing off a concept immersive display system. Vergence is already thinking big in terms of hardware, software, and overall user experience.

Will they make such a transition? Well, Miller and Rodriguez certainly seem to be off to a great start. They’ve each racked up some considerable experience in a wide variety of fields as described above. Vergence Labs has also gained some great attention and support with their acceptance in StartX. For those unfamiliar with Stanford’s startup accelerator, StartX provides resources, mentors, and services (legal, accounting, etc) to let fledgling companies bootstrap themselves into the market. Alexa Lee from StartX, who joined us for our chat, said that there was “no other company like [Vergence Labs] in this session”, a group that will emerge from the incubator in an few months. Rodriguez already has a small Kickstarter fund-raiser under his belt, and Vergence Labs is ready now for another. They’ve just started a KS project to raise $50k for computer-enabled eyewear. They’ve already attracted some high-end backers and raise $4k in their first day. Check out the video below to learn more:

More general (equity) fund-raising will happen soon, and in fact would have happened earlier but Miller delayed the round to take better advantage of StartX’s network. The actual products are close as well. The technology to build the immersive displays Rodriguez and Miller discuss in that first video is almost completely ready to go, except for battery/power concerns. Yet even there Vergence Labs is hopeful, looking to extend their current two hour battery life using a suite of efficiency tricks that Rodriguez reverse engineered from the XO laptop (another one of his school projects). If batteries are the weakest link, I asked the two entrepreneurs which were the strongest bits of tech in their stable: “Our screen technology is state of the art…and with computer vision, we are really pretty damn cool.”

In other words, Vergence Labs has some technological surprises up its sleeve, and the vision to apply those innovations with immediate and long term goals. The talent here is wide and deep, and the enthusiasm is like a enormous pitcher of Kool-Aid. Should you drink it? Well, listen to their latest Kickstarter pitch and judge for yourself. These guys may simply be another young company that sets off to change the world and gets stuck making marketable gizmos…or they could be the real deal – one of the hundreds of innovators that history will place at the foundation of the development of the Singularity. Either way, I find myself very hopeful about what Vergence Labs will be able to show us in the years ahead.

[image and video credits: Jon Rodriguez and Erick Miller (Vergence Labs)]

KLIK not only knows where a face appears on the screen, it knows who that face belongs to.

I never forget a face. Neither, apparently, does my iPhone. KLIK, the exciting new app from Face.com, can automatically recognize faces through the smart phone’s camera. Just open the app, take a photo, and KLIK will search through previous pics to correctly tag the face(s). If linked to a Facebook account KLIK becomes even more powerful, correctly identifying friends based on their shared photos. KLIK is so fast that, with a good internet connection, it can accurately identify someone before a picture is taken. Not only that, but it’s absolutely free! After downloading the app on an iPhone 3S and testing it for several days, this writer is amazed at how quick and accurate KLIK can be…when it works. There are definite bugs to be worked out of the system. Still, Face.com’s foray into mobile facial recognition for social media is stunning when it succeeds, and its launch hints at the absurd power this technology will have in the very near future.

As Singularity Hub has discussed in the past, Face.com is a well known innovator when it comes to facial recognition. They are one of the leading sources (if not THE source) of facial recognition on Facebook, and their API allows for a wide range of applications. Automatic tagging of photos, locating faces for photo manipulation, even characterizing faces by their apparent emotions – Face.com can do it all, and it can do it very well. So well, in fact, that their technology has been used to help identify ‘anonymous’ looters from the most recent London riots. While Face.com technology isn’t fool proof when transforming blurry surveillance footage into quality court room evidence, it is remarkably powerful when dealing with the high quality pics taken by smart phones.

Which is readily apparent with KLIK. The new social app is free and available for phones running iOS 4.3 and beyond. After downloading KLIK I hooked it up to my Facebook account and then pointed it at a friend. While I was still getting the photo lined up, KLIK superimposed my friend’s name on the screen. I hadn’t even taken the shot yet. It didn’t matter, KLIK had automatically searched through my data and found that friend’s face. It suddenly became clear why this app’s development codename was “Project Badass”. Automatic facial recognition and photo tagging on my iPhone for free? Both my friend and I were suitably impressed. The word “scary” was bounced around more than once.

KLIK is aimed at making your photos social..

In the days since I have found that KLIK is both badass and fallible. It fails to recognize some of my close buddies on FB, even though they have plenty of clear photos available. Yet it also learns as I tag photos by hand, and the app can search through both the camera roll on the phone as well as online Facebook pics. KLIK can crash on the iPhone 3S seemingly without any warning, and the processing of images is sometimes horribly slow one minute then lightning fast the next. KLIK is aimed at making your photos social. You can share your pics through KLIK itself with locations powered by FourSquare. You can also upload photos (properly tagged, of course) to Facebook or Twitter. While KLIK can operate without being linked to social media, the interface prompts the user to share every time a picture is taken, and there is some worry that one could accidentally post a photo publicly while trying to maneuver through KLIK’s bare bones (and icon heavy) UI. That shouldn’t be surprising as, according to the Face.com blog, KLIK is “still in the initial stages of development but [we] wanted to share it with friends, get your feedback and continue to improve.”

Even if KLIK isn’t quite ready for prime time, the underlying technology of this app is not to be ignored. Automatic facial recognition tagging was going to come to mobile sooner or later (and has in other guises) but KLIK shows that “sooner” means “today”. Not only that, but KLIK is just the most overt application for this branch of facial recognition. Face.com’s real strength seems to be in packaging their software into accessible, and widely used, APIs. How long until something like KLIK is an integrated portion of every social media app? Facebook’s already using their tech on the back end, this mobile form of automatic tagging could clearly be a more overt feature offered very soon. Apple already has a similar sort of facial recognition embedded in their iPhone 4S (adapted from tech seemingly acquired from Polar Rose). The newest Android phones already let users unlock the devices with just their face as a pass code. It’s clear that smart phones can, with some nominal rate of failure, identify people quickly and effectively. That’s not years or even months in the future, that’s today.

KLIK is fun, but I don’t really need it in my day to day life, and I don’t see the app itself as becoming a household name. Yet there’s no doubt that this technology is a game-changer. Automatic tagging in mobile phones through augmented reality interfaces – as awesome and futuristic as that sounds, it’s just the tip of the iceberg. Security applications for facial recognition have been widely discussed before, but again, this technology now seems like it could be universal in just months or years. The integration of facial ID with the social network is both amazing, and as my friend said “really, really creepy.” Is there a picture of you on the internet? Better assume it’s been automatically tagged with your name. Did someone point a smart phone at you in public? Same conclusion. While KLIK is limited to those photos shared by you and your friends on Facebook (or taken on your private camera roll) future versions of this tech are almost certainly going to be more widely applied. That means that in the next few years (maybe much sooner), any camera that sees you will know who you are. You are your face, and your face is public. If not today, then very, very soon.

Welcome to the future. We got rid of the old sense of anonymity with a single click.

[image credits: Face.com]
[source: Face.com Blog]

Police have released dozens of images of people involved in the London riots. Will facial recognition software identify them?

There are more than a million CCTV cameras in London, and it’s time for them to start paying off. The recent riots have left local police and Scotland Yard with untold hours of video footage of looters and rioters. Advanced facial recognition software that was originally planned for the 2012 Olympics is now being applied to filter through the colossal heaps of video evidence in hopes of pairing criminals with their identities. While hundreds of people have already been arrested for their actions during the riots, it’s unclear how many, if any, will be prosectured based on CCTV camera and facial recognition evidence. Amateur attempts to use commercially available face scanning software have failed despite fervent hope from the public. While the technology has grown more powerful, and more useful in controlled situations, the quality of images from CCTV cameras may be insufficient to convict many of those involved in the looting. Law enforcement enthusiasm for facial recognition is strong, but the chances of face scanning providing immediate help in the aftermath of the London riots seems slim.

Operation Withern at the Metropolitan (London) Police is an ongoing attempt to gain valuable information about those involved in the London riots. They’ve released photographs to the general public in the hopes that witnesses will come forward to identify suspects. Facial recognition may also be able to link some of these suspects with images available on Facebook and other social media sites. Despite hours and hours of CCTV camera footage associated with the riots, however, Operation Withern has published less than 200 images of persons of interest. Where are all the other pictures of suspects?

Well, CCTV footage taken during the recent riots is often quite poor. To demonstrate, here are two typical clips that have been made widely available to the public. The first shows a group of looters breaking into a T-Mobile store, stealing some products and destroying many others. Our second clip was released by the London Police. It shows a group of police officers chasing a band of rioters only to be struck by a fleeing car. Do your best to try and discern any faces in either clip:

As we mentioned two years ago when reviewing the UK’s massive investment in CCTV cameras, the video from these devices is rarely of sufficient quality to actually convict someone. Many are located high above streets, positioned to show the backs and bodies of those below much more than their faces. In those rare instances when faces are clearly seen, the resolution of the picture may not allow police officers to reliably identify perpetrators no matter what facial recognition technology they use. It’s important to remember that in the recent history of London CCTV, there was an average of only 1 crime solved per 1000 cameras. Even with the massive amounts of video accumulated during the riot, CCTV cameras simply may not make much of a difference in pursuing those responsible for crimes.

But CCTV cameras are just a single part of the equation. News crews and private surveillance often captured higher quality recordings of the violence that unfolded. Here’s a clip of a BBC camera being stolen from a news team. While many faces are covered or otherwise unseen, those captured on this video would have a much higher chance of being recognized by software:

The greatest hope for identifying rioters may come from the general public. During the riots thousands of people took footage of crimes using their personal mobile devices. That footage has been shared on Facebook, and other public forums, leading to a few arrests. A vigilante group hoped to leverage commercial facial recognition software from Face.com to help them identify people from some of the many images acquired on CCTV, mobile devices, and other cameras. However, after only a few days, the vigilantes have called it quits. While Face.com works wonders on Facebook, as we’ve seen before, it simply wasn’t reliable enough to distinguish faces captured during the riot. Facial recognition technology developed for UK law enforcement should be considerably better than Face.com, though it’s yet to be seen if it will be able to quickly identify people captured by public devices.

Facial recognition has garnered the praise and interest of police everywhere. Heathrow Airport will be employing it soon, the US is toying with taking high quality facial scans after arrests, and private companies are developing the technology for even broader applications in the near future. Yet most of these applications are in controlled environments. You rarely see someone wearing a mask and running with a lit torch in an airport (if you do, it’s probably time to leave). Face scanning and recognition programs will work best when you can pass everyone through choke points. A single archway in a bus station, the entrance to a stadium – identifying people is going to be infinitely easier to do when you can ensure that everyone’s face is uncovered, aimed towards the camera, and moving at a reasonable speed.

For now I think that means efforts by Scotland Yard and the Metropolitan Police are simply going to fall far short of expectations. Some day video resolution and facial recognition technology will be so advanced that anyone can be identified whenever they pass by a camera. That day is simply not here yet.

Which I think is evident in the behavior of law enforcement in London. Several stories have emerged in the major press that London officials are using mobile phone records, and social media activity to identify rioters rather than trying to match their faces to names. Did you call someone from inside the riot? What did you say? Did you use Facebook or Twitter to encourage others to come with you to do some looting? Well then you’re screwed. Public admissions of involvement, even just that which can be inferred, may be much more damning evidence than your blurry face taken by a CCTV camera.

Which, if you’re a privacy person, should be even more threatening. Recognizing your face (or your gait, or your body outline, or your iris, etc) is intrusive, but it can be overcome with masks and costumes. The UK is talking about shutting down mobile connectivity, and social media sites, whenever they feel such resources are being used by those opposed to law enforcement. We’ve seen similar efforts by the Bay Area Rapid Transit here in San Francisco. BART turned off phone service in its stations to deter protesters. Faced with a disquieted public, law enforcement thinks of hindering resources that empower the public. Worry about facial recognition if you want, but that technology is years away from being the repressive tool people fear it to be. Social media is already here, and it may be leveraged against the public far more readily.

[image credit: Operation Withern ]
[video credit: The Telegraph, Models and Music, BBC]
[sources: CBS News/Associated Press, Forbes]

Face.com's API can now detect your mood and expression. Awesome or creepy?

Does the internet know when you’re smiling? That’s a rhetorical question. Of course it can. Face.com, makers of a top notch facial recognition API, recently announced it was now capable of detecting the moods and expressions of people in photos it scans. Now, not only can the API tell who you are, it can say whether you were happy, sad, smiling, or even kissing. Face.com is the creator of the popular PhotoFinder and PhotoTagger apps on Facebook, so you may soon see that capability on the social network as well as among the 20,000 developers who use the Face.com API. In related news, Facebook (using its own software) has been automatically using facial recognition to tag photos you upload since December of last year. They’ve already prompted the use of such facial scanning 2.7 billion times in the past six months! Learn more about their push for automated tagging in the video below. Facial recognition has grown so sophisticated, and cheap, that it seems it will soon leave no photo untagged, no mood unrecorded. If that idea makes you uneasy, don’t worry, the social network of the future knows exactly how you feel.

Facebook quietly rolled out their in-house facial recognition (Photo-Tag Suggest) in the US last December, allowing users to tag their friends, teaching the social network who was who. Soon there after, Facebook could automatically suggest who was in each picture, making tagging quick and easy. It’s a pretty awesome feature, and as of early June it was available “in most countries.” On June 30th, Facebook announced it had prompted its 750 million active users 2.7 billion times to try the automated tagging process, often with the rather ambiguous box on your homepage labeled “Photos are better with friends.” Naturally some privacy activists groups are crying foul, worrying that although the Photo Tag Suggest only works on your friends Facebook is collecting huge amounts of data on our appearance. ABC News has more:

Mood recognition from Face.com

The Israeli facial recognition company ABC News interviews is Face.com, who we discussed a little over a year ago with the launch of their API. According to a recent newsletter, in the past twelve months their API has gone from a few dozen developers to over 20,000. That’s very impressive, but when you look at all the Face.com API can do, it just makes sense. Not only is the code quite capable in finding faces and matching them against others in the databases you provide, it is now able to detect moods and expressions. The Face.com API can recognize when you are happy, sad, surprised, angry, or neutral and gives a confidence interval on that analysis. Now, when sorting through photos, a developer could use the API to organize them according to how you felt at the time the pic was taken. Similarly, the API can now detect smiles, parted lips, and kissing lips, further refining the classification. Among the suggested uses for the new capabilities would be reaction detection in live chats, filtering in dating sites according to mood in profile pics, and selecting the best photos for display.

When you start to imagine the potential applications for this technology, it gets really amazing. Sure, Facebook already is pushing to be able to recognize its 750 million users, but that just tells you what everyone looks like. Face.com could make it possible to guess at relationships based just on photos. Do you always smile when you’re with someone? Maybe it’s a sign you really like them. File that away in the metadata and maybe use it to make their suggestions mean more in searches, etc. Do thousands of people take geotagged pictures at a certain location every year? Maybe that location (stadium, landmark, etc) would like to find all the happy smiling people in those pictures and use it for marketing. Filing celebrity photos for press agencies, determining when customers are unhappy during recorded events, tracking how consumers feel during market research—there are tons of ways this technology could be applied.

Expression recognition from Face.com

And it’s (mostly) free. 20,000 developers have already found uses for the Face.com API, and that’s without any of the new mood and expression detection algorithms. As we’ve discussed before, many are simply cute apps for entertainment, but with more sophisticated capabilities we’ll see more sophisticated applications. Soon, social networks, and really all internet sites, could use facial recognition technology that puts most security networks to shame.

What gets me about all of this is really how ubiquitous facial detection and recognition has become. iPhoto has it, Facebook has it, Google’s Picasa has it – all the major players have brought it online, and we’ve seen it used in advertising stunts as well. Which means that it’s probably also in many private places we don’t know about – ATMs, office buildings, etc. Somewhere along the line, this fun and amazing capability sort of creeped in everywhere. Which I may not be totally against. Your face is part of your appearance, when you show it somewhere you are publicly displaying it. If you (knowingly) allow people to record you, you should recognize that they can use it to learn more about you, including your identity. That’s part of living in a world filled with cameras and digital databases. That being said, I’m not sure it’s as creepy as we may think. Most applications of this technology have, so far, been limited to applications you are aware of, not readily searchable by the public, and increasingly secure – Face.com now supports SSL on their API for instance.

If, however, we decide we can’t live with computers being able to recognize us and our emotions, we better act fast. Face.com’s API is only going to get more sophisticated in the next year, and we’ve seen similarly advanced detection algorithms for video as well. Give them much more time, it there’ll be no telling what they’ll be capable of…

...like shooting lasers out of your eyes!

*Photo created using Laser Eyes by OneByOne Design. Click image to try it for yourself.

[image credits: Facebook, Face.com, One by One Design]
[video credit: ABC News]
[source: Face.com, Facebook]

They might not light up our eyes, but long-range facial recognition cameras are sure to light up computer databases with our whereabouts, criminal records, possibly even our shopping tendencies.

Ever heard of a company called Digital Signal Corporation? Neither had I.

Well, remember the movie “Minority Report,” how cameras everywhere would scan Tom Cruise, instantly recognizing him to either welcome him into the shopping mall or send the cops after him? The same sort of facial recognition technology is exactly what Digital Signal Corporation is into–except these guys are for real.

Marketwire recently reported that DSC received $15 million to develop three dimensional long-range biometric facial recognition technology.

Hmm, I wonder what exactly DSC’s technology will be used for? Makes me think of the cameras being mounted on more and more police cars that are able to record virtually every license plate that passes by. Will facial recognition devices be used to keep track of everyone that goes to a baseball game, the supermarket, or a political rally? The DSC website doesn’t offer any information about what their cameras can do. There’s pretty much nothing there except a picture of a guy staring you down like “he knows…HE KNOWS!” In an attempt to get past the guy, I searched for a link, but found none. I eventually saw this: “Our website is currently under construction.”

How convenient.

I did the next best thing and googled the CEO, David Guttadauro. Didn’t find much though, except I noticed he had a Linkedin account. So I checked Linkedin and found DSC. You’re not going to believe this, but across six pages of employees, Guttadauro’s is the only profile with a picture! Wait, it gets spookier. Looking at his profile I saw that he and I are actually 3rd degree contacts!

Like Sebastian in the “Neverending Story” I shut my laptop and uttered to myself, “This is toooo weeeird.”

Digital Signals Corporation: even the name has a seemingly nondescript yet somehow Sci Fi draconian ring to it.

When I opened my laptop back up I finally managed to track down a quote of his on baltimore.citybizlist.com, talking about the need for facial recognition technologies. Referring to an “uncertain world” that is becoming increasingly interconnected, Guttadauro points out the ever-increasing need to identify terrorists, track down shoplifters, or members of an organized crime family. He also points out the need to recognize customers, friendly soldiers, and frequent travelers.

Another glimpse into the inner workings of this mystifying company comes from Bloomberg Businessweek’s company overview, which says that “Digital Signal Corporation engages in the research and development of three-dimensional facial recognition systems for the department of defense and other federal agencies.”

Well now, that’s interesting. And that may explain in part why information on the technologies DSC is developing is so hard to find. I’d like to know why exactly they got the $15 million. There are a lot of companies out there developing their own facial recognition technologies. Why did Columbia Capital, City Light Capital, SilverHaze Partners, Novak Biddle Venture Partners and Paladin Capital Group all go in big for DSC? Must be something pretty cool.

At any rate, facial recognition technology is something cool. The first step in 2D face recognition, which compares the faces in pictures, is to find the faces–that is, to differentiate between a face and background patterns. Once a face is acquired, different features of the face are measured including distance between the eyes, width of the nose, and the length of the jaw line. These measurements are then transformed into a numerical code, called a faceprint, which is stored in the database.

It’s tricky though. Early 2D recognition software required a straight-on image of a photograph, and any variation in light or angle would greatly affect accuracy.

Three-dimensional face recognition was developed in an attempt to improve accuracy. It measures curves on the face at a high-resolution, sub-millimeter scale. In 3D recognition, the curve measurements provide the template.

The hope is that the 3D data will ultimately be used to identify criminals, terrorists, and people in the crowd who are late on their taxes (just kidding about that last group, I think). But police departments don’t have 3D images, they have mugshots. It’s tricky business to match a 3D face to a 2D face because of the different types of measurements that went into creating the respective templates. Algorithms are currently being developed that will allow accurate comparisons between the 2D and 3D templates.

What DSC is developing–biometric facial recognition–is the newest generation of facial recognition technology. Rather than measure distances between the eyes and other nodes, this technology measures unique skin features. A picture is taken here too, but not of the entire face, only a patch of skin. This skinprint is then analyzed by a process called Surface Texture Analysis that parses up the skinprint into smaller sections and records lines, pores and skin texture. The software is so accurate it can differentiate between identical twins.

It may surprise you to find out that facial recognition cameras are already in use. A company in Japan actually put the things in sidewalk billboards. When a pedestrian walks by, the camera scans their face, takes its best guess as to the sex and age of the person, and changes the advertisement on the billboard to target the person’s demographic!

Remember that scene in “Minority Report” when Cruise enters the shopping mall: “Welcome back, Mr. Yamamoto. Would you be interested in trying out our new chinos?”

If privacy advocate groups like the ACLU had a problem with scanning license plates, they’re going to go berserk at the idea of automated, long-range face scanning. But, as I’m sure automated license plate recognition cameras for the police will soon be everywhere, facial recognition cameras will soon be too. Companies like Digital Signal Corporation are working hard to make sure we are watched, even if, despite our best efforts, we can’t watch them.

[image credits: rubinium.org and Digital Signal Corporation]
image 1: Minority Report
image 2: Creepy Guy

Nao is getting an emotional upgrade.

Science fiction has taught me to fear and respect robots with emotions. Now scientists are teaching robots to fear and respect humans with emotions. It’s a virtuous circle. Aldebaran’s widely successful Nao robot is being used in experiments led by the University of Hertfordshire that hope to have it learn emotions in the same manner as young children. Using facial and body language recognition, special Nao prototypes will form attachments to those humans which teach them the most. The robots will then pick up on emotional cues and mimic the way they are used. This is pretty much what human and chimpanzee babies due to learn about emotions. Just like a child, the prototype Naos will show distress if their preferred caregiver doesn’t assist them when confronted by a stressful situation. Bots that need your care, wow. The idea of an emotionally vulnerable robot is getting people charged up. Check out Fox News’ clunky interview of an Aldebaran exec below.

Robots that learn like children? Yeah, we’ve seen those. But iCub is developing motor skills, Myon is learning language, and Diego-san…well, I think it’s being taught that life is cruel. In any case, the prototype Nao robots out of Hertfordshire are specifically aimed at gaining emotional intelligence and that sets this project apart. A large part of our brains are dedicated to recognizing, understanding, and predicting the emotions of others. If we can get a robot to develop a high emotional IQ we may have gone a long way towards making the machines decidedly more human. Not only that, but we’d be preparing ourselves for the eventual rise of better than human AI by getting these smart machines interested in our emotional well being. If we’re going to create robot overlords we should at least make them compassionate, shouldn’t we?

What can the Nao emotional prototypes actually feel?  They have preprogrammed emotional responses for anger, fear, sadness, happiness, excitement, and pride. Many robots have had such preprogrammed emotional displays in the past, but the Naos will be learning when and how to display these emotions from their human teachers. In essence, they will have preconceived notions of what an emotion is like, but are very open on learning how and when to apply it.

The work at the University of Hertfordshire, led by Lola Canamera, is part of a larger European initiative known as FEELIX Growing (FEEL, Interact, eXpress). FEELIX is aimed towards generating reasonable emotional bonds between humans and robots. The Nao prototype will also be part of ALIZ-E, a project that seeks to expand the periods of time over which the robot can be expected to interact and learn. In other words, the Naos will be taught how to develop their emotional understanding over their entire lifetimes. According to the University of Hertforshire News, this will enable them to serve as companions to children in hospital settings. In fact, ALIZ-E isn’t the only such application for the emotional Nao. University of Connecticut’s Center for Healht Intervention and Prevention (CHIP) will be using Nao to help autistic children learn about emotion and emotional displays. CHIP believes that the simplified emotional displays of robots, and their ability to be controlled, will make them more accessible to the young patients. So, not only will Nao be learning from us, he’ll be teaching us as well. Awesome.

As exciting as Nao’s emotions may be, I’m not terribly impressed quite yet. We’ve yet to see one of these sensitive Nao prototypes in action proving its emotional chops. Will it appear more aware of emotional stakes or will it simply be slightly less chaotic in its emotional displays than other robot toys of the past. As brilliant as it is to have robots learn emotions the same way that children do, we can’t know the true potential for the technique until we see results. We may only be teaching the Nao how to mimic a single person, not human motion in general. Though, come to think of it, that would still be a pretty cool accomplishment. Which person should we base the first robotic emotions on? My vote’s for Ben Stein. You know, make it easy in the beginning. We can throw them Meryl Streep later.

[image credit: University of Hertfordshire]
[video credit: Fox News]
[source: Fox News, CHIP, University of Hertfordshire]

Face.com has launched the alpha of their new API. Now, almost any site could find faces on photos.

Face.com, the company responsible for Facebook applications Photo Tagger and Photo Finder, lets you take any photo and quickly identify who is in it and where they are in the photo. This facial recognition is a boon to those tagging photos, and now Face.com is ready to bring a similar capability to the rest of the internet. May 3rd saw the launch of their new open API capable of scanning images and rapidly identifying the location, orientation, and identity of human faces. The API platform is meant for web designers who want to include a facial recognition feature on their own website. With this API, any company could let you upload a photo of yourself and find other photos of you in their database. Now in alpha testing, registering to try the API is free and very quick. Face.com, operated by Israel-based Vizi Labs, is looking to share the API with the developer community to see if the next killer application for facial recognition will arise organically. Eventually, platforms like this one may help your face become an access point to all the digital data about you on the web.

The Photo Tagger and Photo Finder applications have been big on Facebook: billions of photos processed from over 50 million users. We’ve also seen competing software that provides facial recognition on Facebook, including a rather brilliant bit of marketing for Coke Zero. As Facebook continues to expand, a growing number of websites are starting to veer towards becoming social networks, or at least picking up aspects of social networking to stay competitive/current. And photos are probably one of the largest themes of these networks. It makes sense then, that Face.com is looking to sell its photo expertise outside the FB realm. The API seems fairly easy to use, so it could be adopted easily, though their forums are disappointingly quiet for a first week of launch.

Together you and I could overthrow the Emperor and rule as...Uh...I mean the API has some interesting near term applications that are likely to be popular.

In the near term the API launch could mean many more websites will have some facial recognition gimmicks. Face.com actually goes to the trouble of suggesting some of these in their examples: quickly tagging photos through a widget, placing your face in famous photos, and finding celebrity photos on the internet. While none of these is an exactly mind-blowing idea, they are the sort of things that sell well on the web (I must admit I was drawn into playing with the “Poster Yourself” face replacer for much too long). One hopes that developers, hungry for the next big way of drawing traffic, will invent some more insightful applications for the API, but even if they don’t, it’s likely that Face.com will get plenty of mileage out of the standard time-wasters.

In the long term, being able to incorporate facial recognition into almost any web page could be a very powerful tool. You could, with just a photo, conceivably track down all sorts of information about a person – their friends, contact information, or anything else they choose to make public. In fact, Face.com paired up with Comverse to create an iPhone App that does just that, albeit only for people you already know. This “social augmented reality” application was revealed earlier this year at the Mobile World Congress in Barcelona. Take a photo of a friend and it recognizes the individual and rapidly brings up social networking links and other data you may want to use:

Of course, privacy restrictions on Facebook, Twitter, and other social networking hubs greatly reduce your access to someone’s photos without their permission. There’s little danger that someone could take a photo of you and then find you via one of these more reputable sites. In a posting on their blog, Gil Hirsch (CEO) made it very clear that similar privacy protection would apply to the Face.com API. One day however, such privacy is likely to be overcome through crafty uses of technology or general apathy on the part of the public.

As I see it, facial recognition has two basic levels of application. You can use it simply to determine what thing in a photo/video is human, and where that human’s face is. Or, you can use it to determine which human, out of billions, is in a photo. Both have some very powerful applications. We’ve seen how video filtering software can use body and facial recognition to improve security camera surveillance. We may also see our faces, or other portion of our appearance, become our all-in-one easy access key. You can use your face to open a door, others may use it to become your friend on a social network. It’s hard to know how facial recognition will be harnessed in the years ahead, but it’s clear that with companies like Face.com it will certainly be present almost everywhere we go online.

[image credits: Face.com]
[video credit: dariusctc (via YouTube)]
[source: Face.com, Vizi Labs]

Which of the faces on the right match the one on the left? Questions like this helped researchers determine that you inherit your ability to accuractely recognize faces.

The quest for augmenting human intelligence is a fierce one. Our modern civilization is based on years of school attendance – nominally in pursuit of making smarter workers and citizens. It goes beyond that, however: we don’t just want to get smarter through hard work, we want an easy fix. Students regularly use nootropic drugs, herbal supplements are toted as keeping your mind young, and we continually report on studies that hope to find which vitamins and minerals are essential to brain health.

Yet the 21st century approach to making smarter humans is probably going to be genetic in nature. We’ve seen how a single manipulated gene can improve memory and cognition in rats. Now, the CFMT twin study suggests that genes are also responsible for one of our most basic social skills: recognizing faces. Politicians and business executives everywhere would line up around the block if a genetic therapy could be developed to augment facial recognition skills. There’s no sign that such a therapy would be possible yet, but this research suggests that the performance of at least one brain mechanism (possibly many more) is determined by our genes. Which parent doesn’t want a smarter child? Chalk this up as another pressure on society to begin producing designer babies.

It’s pretty rare for us bloggers to be able to participate in the scientific research we report on. Luckily, the CFMT is available to take online. It only asks for a minimum of identifiable information, and takes 5-10 minutes to complete. All you do is study some computer generated faces (like those seen above) and then recognize them in a line up. Pretty simple, and kind of fun. The test is available via Faceblind.org – part of a research partnership between Harvard and UCL interested in prosopagnosia.

I haven’t been able to find online versions of the word recognition and art recognition tests given to the twins (and 1500+ non-twin volunteers). The PNAS paper does show a typical question for each however:

For both the Abstract Art Memory and Verbal Paired Association Memory tests the image/words on the left would be shown then removed before the images/words on the right would be shown.

It will be interesting to see if this research spawns a dedicated search for the genes responsible for improved (or impaired) facial recognition. Certainly growing databases of genetic samples, and the ever decreasing costs of whole genome sequencing makes that search a reasonable possibility. As we find other heritable cognitive abilities, similar work can be performed. Exhaustive studies of our genes may lead to a better understanding of the ‘genotype of cognition’.

Speaking of studies, why don’t we see how Singularity Hub readers stack up on the facial recognition scale? Take the Cambridge Face Memory Test and post your scores in the comments section. We can get a leader board started. May the best genes win!

[image credit: PNAS, Faceblind]
[sources: Faceblind, PNAS, UCL, Harvard]

Forgot to punch your time clock? Don't worry, your face did it for you.

Ditch the keys and throw away the time cards, now all you need is your face. Wholesale electronics giant Chinavision is offering a facial recognition door lock for around $465. The device uses two night vision equipped cameras to recognize up to 500 faces from 2D photographs, and transmit information via USB or ethernet port. Once it recognizes someone the facial recognition system can unlock a door and/or record the arrival of an individual (up to 150,000 such records can be stored). Chinavision is promoting the device as a replacement for time clocks at businesses, able to record access to restricted areas or keep track of employee attendance. It’s not clear how easy it is to defeat this device, so I wouldn’t wire it to your front door just yet. Still, if cheap facial recognition proves itself, we could see it replacing traditional locks in the year ahead. Check out more photos of the facial recognition door lock after the break.

Facial Recognition (FR) technology is already in use in government run security checks. While FR seems well suited to surveillance and restricting access, we’ve also recently seen it gain popularity on Facebook as a way to augment social networking. Right now, I certainly trust it more in the latter than I do the former. FR checks are only as good as the original images with which they are calibrated. London airports (and others) are often comparing against passport photos. The device from Chinavision also uses 2D images. While using two cameras and applying some careful stereography will keep FR from being tricked by a photo cut out from a magazine, the technique is far from fool proof. If facial recognition gains popularity, we may see a push for ID cards to contain 3D scans of our features to provide extra security. Still, while humans naturally focus on faces, other biometrics (finger prints, ear scans) may be a better avenue for tracking our identity. It’s still unclear which kinds of scans, or combinations of them, will form the security check of the future, but you can bet it will involve using your body as its own ID. Carrying around strangely shaped pieces of metal can’t last forever.

Two cameras compare your face to a 3D object developed from a stored 2D image.

Not a bad package for less than $500.

[photo credits: Chinavision]

What makes a robot smile?

Dr. David Franklin Hanson, Jr , the founder of Hanson Robotics, owes much of his success to Frubber. Frubber is a revolutionary material with a porous structure that allows it to move similarly to human tissue. Previously used materials were too heavy and too stiff. Using Frubber, Hanson can create a wide range of realistic facial expressions while cutting the need for powerful actuators. Typical applications using Frubber consume more than 20 times less power than with traditional materials. A realistic face and neck can run for hours using just a few AA batteries.

…Believe It or Not

The engineers at Hanson Robotics appreciate good science fiction. That’s why in 2005 they built a realistic replicant of author Phillip K. Dick (Blade Runner, A Scanner Darkly and many others). Using information about the authors life, voice recordings, and an AI program, they were able to create an interactive experience. Here’s a great video from WIRED:

After Phillip Dick, other great science luminaries were sure to come. In 2006, Hanson teamed up with Hubo robotics in Korea to form a fully articulate android… of Albert Einstein. Whereas the Phillip animatronic just sat down, this new android could move around. Since it’s creation this android, and the detached robotic head, have been big crowd favorites. It recently appeared, along with other Hanson creations, at the 2009 TED conference. Check out the original Hubo/Hanson session in the video below:

Undoubtedly the success of these replicants is largely due to Frubber and fame. We perceive realistic facial expressions on celebrities and we overlook the shortcomings. It’s the same set of conditions that makes a wax museum successful. In addition, however, Hanson robots can also maintain eye contact. Using face tracking software, the replicants know where to look and how to move to keep their audience interested. Coupled with an interactive program that can learn from past conversations, these developments push the Hanson robots closer and closer to the Uncanny Valley.

The Uncanny Valley is that point where a simulated being becomes too realistic too enjoy. It’s a classic concept in robotics: develop something vaguely human and it will be cute. Create something too life- like and we will be disturbed by its imperfections. Hanson robotics doesn’t believe in the Uncanny Valley. Or rather, they think they have a solution: the Bridge of Engaging Design.

In Hanson’s view, people will accept things in art that might disturb them in reality. Horror movies or sculptures by Ron Mueck, we appreciate these things even though they lie in the middle of the Uncanny Valley. The desire to understand and anthropomorphize can permit viewers to enjoy a piece of art at all levels of realism. In that way, Hanson’s work is less disturbing than it is intriguing.

Bringing It Back to the Singularity

One of Hanson’s latest projects is called Zeno. Zeno is an anime style robot designed for entertainment and toy sales. He can walk, dance, and interact with others. His programming adopts the developments from previous Hanson projects: he adapts in conversation, maintains eye contact, and has limited learning. As part of his purpose as merchandise, he even comes with a great background story that could be included in literature, movies, or other media. You see, Zeno is the hero of the Singularity.

That’s right, in a meta-physical field day of fact and fiction, Zeno’s backstory details how he starts off as a toy in present day. By 2029 (Kurzweil fans will recognize the date), Zeno develops a greater intelligence and starts to question his existence. After the requisite government kidnapping and installation in a secret robot academy, Zeno meets new friends and helps save humanity and robots alike. Thus Zeno is the hero of both mankind and robokind. Well, it’s cute, but like so many things from Hanson Robotics, it’s a little weird. I mean you’re playing with a toy that claims it will one day develop from a toy into a sentient being. That has to make you a little cautious. “Sure Zeno, you can have some more batteries, just remember me when the revolution comes.”

Whether in toys, movies, or interactive displays, Hanson Robotics and its replicants are pushing the boundaries of what people will accept as human or human-like. Either with Frubber, or the next great material, engineers will keep improving the facial expressions, programmers will improve conversation software, and voice generation/recognition can only get better. While the animatronic faces and voices are imperfect (and in my opinion a little creepy), Hanson Robotics may be correct: in the end there may not really be an Uncanny Valley, just an audience that is hungry for new and innovative design. To those waiting for the next form of entertainment there’s likely no such thing as “too real.”