Generating 3D Models of Your Face in Real-time with RGB

Want a 3D movie of an object? Forget scanners, forget stereoscopic cameras. All you need to capture real-time 3D movement is one camera and some colored lights. Researchers at Toshiba's Cambridge Research Lab (CRL) have developed a new process that uses red, green, and blue light to capture a 3D image of your face. Better still, the system can record all the movements and facial expressions as they happen. The result is a dynamic 3D model created in real-time. This is an excellent feat of computer vision, made more impressive because it uses fairly simple hardware. Silicon.com had a chance to interview the CRL team and take some cool new pictures. The technology may one day let us create hyper-realistic avatars to use online in real-time applications, upgrade augmented reality, or improve virtual actors.

Generating a 3D computer model of a solid object is fairly simple these days. Just place something on a turntable, take the right series of pictures, and voila! Dynamic models, however, are much more difficult. You can't rely on taking a series of pictures, you have to take just one shot, make a 3D model, and update it in real-time. That's no easy task. Luckily, Toshiba CRL has a strong history in exploring computer vision. They helped develop the gesture recognition systems that Toshiba is incorporating into its TV sets. The use of RGB lights is equally ingenious. Depending on its contours, each part of your face will reflect a different amount of each colored light, letting a computer identify the angle of the surface just by its overlapping color. Since these colors can be identified as quickly as the camera refresh rate you have highly accurate 3D mapping in real time. Very cool.

Current systems require you to be in a dark room, but ideally the system would be used under normal conditions. I wonder if CRL couldn't develop a similar system that used near IR light instead of the visible spectrum. That way, users wouldn't notice the illumination, and the technology could be incorporated into public spaces. I'm sure security checks could be improved, and maybe made less intrusive, if 3D computer models could be generated of everyone as they walked by.

James Cameron probably had a better system for generating 3D models with his movie Avatar, so I doubt we'll see a RGB system used for films. Still, for cheaper applications, we could use the CRL system to improve computer models and graphics for other forms of entertainment. The most likely place we're going to find this technology is online. A ring stand, or other harness, could hold RGB lights, some software could control your webcam, and you'd be ready to present a 3D model of yourself anywhere on the internet. MMORPGs could be swarming with fantastic creatures with our faces slapped on them complete with real time movements. An elf with my geeky facial expressions? I'm not saying it would be pretty, but it would definitely be a lot of fun.

[image credit: Natasha Lomas/Silicon.com]
[sources: Toshiba CRL, Silicon.com]