Toshiba's researchers can generate real time models of your face using just one camera and some red, green, and blue lights.
Want a 3D movie of an object? Forget scanners, forget stereoscopic cameras. All you need to capture real-time 3D movement is one camera and some colored lights. Researchers at Toshiba’s Cambridge Research Lab (CRL) have developed a new process that uses red, green, and blue light to capture a 3D image of your face. Better still, the system can record all the movements and facial expressions as they happen. The result is a dynamic 3D model created in real-time. This is an excellent feat of computer vision, made more impressive because it uses fairly simple hardware. Silicon.com had a chance to interview the CRL team and take some cool new pictures. The technology may one day let us create hyper-realistic avatars to use online in real-time applications, upgrade augmented reality, or improve virtual actors.
Generating a 3D computer model of a solid object is fairly simple these days. Just place something on a turntable, take the right series of pictures, and voila! Dynamic models, however, are much more difficult. You can’t rely on taking a series of pictures, you have to take just one shot, make a 3D model, and update it in real-time. That’s no easy task. Luckily, Toshiba CRL has a strong history in exploring computer vision. They helped develop the gesture recognition systems that Toshiba is incorporating into its TV sets. The use of RGB lights is equally ingenious. Depending on its contours, each part of your face will reflect a different amount of each colored light, letting a computer identify the angle of the surface just by its overlapping color. Since these colors can be identified as quickly as the camera refresh rate you have highly accurate 3D mapping in real time. Very cool.
As RGB lights overlap they illuminate the contours of the human face in different colors corresponding to the inclination of its surface.
Current systems require you to be in a dark room, but ideally the system would be used under normal conditions. I wonder if CRL couldn’t develop a similar system that used near IR light instead of the visible spectrum. That way, users wouldn’t notice the illumination, and the technology could be incorporated into public spaces. I’m sure security checks could be improved, and maybe made less intrusive, if 3D computer models could be generated of everyone as they walked by.
James Cameron probably had a better system for generating 3D models with his movie Avatar, so I doubt we’ll see a RGB system used for films. Still, for cheaper applications, we could use the CRL system to improve computer models and graphics for other forms of entertainment. The most likely place we’re going to find this technology is online. A ring stand, or other harness, could hold RGB lights, some software could control your webcam, and you’d be ready to present a 3D model of yourself anywhere on the internet. MMORPGs could be swarming with fantastic creatures with our faces slapped on them complete with real time movements. An elf with my geeky facial expressions? I’m not saying it would be pretty, but it would definitely be a lot of fun.
[image credit: Natasha Lomas/Silicon.com]
[sources: Toshiba CRL, Silicon.com]
Comments
So would this also make it possible to do full body mocap with just one camera? It’s pretty amazing; this approach never would have occurred to me.
So would this also make it possible to do full body mocap with just one camera? It’s pretty amazing; this approach never would have occurred to me.
So would this also make it possible to do full body mocap with just one camera? It’s pretty amazing; this approach never would have occurred to me.
the problem is it will work only for grey objects, any coloured surface would distort the ration between reflected rgb, there might be a way to compensate for that, but I wouldn’t bet on it
the problem is it will work only for grey objects, any coloured surface would distort the ration between reflected rgb, there might be a way to compensate for that, but I wouldn’t bet on it
Human faces are seldom grey
the problem is it will work only for grey objects, any coloured surface would distort the ration between reflected rgb, there might be a way to compensate for that, but I wouldn’t bet on it
Aw i would’ve loved to see a video for this one.
Aw i would’ve loved to see a video for this one.
Aw i would’ve loved to see a video for this one.
More likely that folk would use it to interpret their facial gestures rather than map their actual face.
This technology looks like they’ve turned the ‘normal’ mapping of 3D models inside out, though that’s not really an apt comparison.
More likely that folk would use it to interpret their facial gestures rather than map their actual face.
This technology looks like they’ve turned the ‘normal’ mapping of 3D models inside out, though that’s not really an apt comparison.
More likely that folk would use it to interpret their facial gestures rather than map their actual face.
This technology looks like they’ve turned the ‘normal’ mapping of 3D models inside out, though that’s not really an apt comparison.