What if you could remove all the ugliness in the world? It’s not a hypothetical question. Researchers at Ilmenau University of Technology in Germany have developed a new augmented reality technique that erases images from real time video. Called Diminished Reality, the software can take any area selected in a video feed and use photo-shop like adjustments to copy the surroundings into its place. Where once you saw an object now you see the object has been removed. A piece of your world has been erased. Diminished Reality records video from a camera and displays the modified result on a screen with only a 40ms delay. To your eyes it’s effectively instantaneous. Watch a demonstration of the augmented reality editing program in the video below. I’m blown away by how well it works in these early examples.
The power to edit reality is going to create some amazing applications for this system. You could make certain people, or objects, invisible on live television – censorship and entertainment may never be the same again. You could create new augmented reality video games that not only add in virtual objects to your environment, but subtract real objects as well. Jan Herling and Wolfgang Broll, the researchers at Ilmenau’s Department of Virtual Worlds and Digital Games that created Diminished Reality, have suggested it may help us with urban planning and interior decorating – remove the old and envision the new all without changing a thing. The developers also suggest that Diminished Reality may be a counter to invasive ads and over stimulation. If DR could be incorporated into a pair of glasses you could filter out all the unwanted visuals around you. No more billboards, no more posters, no more commercials. For that matter, no more graffiti, no more trash on the streets… The ability to subtract things out of our view of reality could be amazingly seductive.
When I first saw this demonstration video I thought the technology was a fake. Film an object, then film the same space with it missing – instant disappearing act. Yet the mirror reflection at 0:48 and the mild distortions you can see when the camera pans quickly have convinced me that DR is the real thing.
As the video shows, Diminished Reality works by applying several adjustments to an image. The picture is reduced in resolution, the selected area is erased, and various copying and blurring effects are applied as the image is restored to higher resolution and repaired. All of this takes just 40ms, allowing the process to work on streaming video. According to the Diminished Reality press release, the video quality and processing speed of mobile devices is improving such that we can expect DR to be available on smart phones soon. We’ve already seen other augmented reality applications on mobiles, I think we could have Diminished Reality in the next few years.
It’s really the speed and portability of this technology which will make it a breakthrough. After all, removing unwanted parts of images has been around for a long time, and the processes for doing so are becoming more sophisticated, automated, and faster. You may have heard of Arturo Flores, a grad student at UC San Diego, who developed a prototype program that could remove people from Google Street View. In some respects, Diminished Reality is just another step forward in this basic field of visual editing. Yet the steps that DR takes (real time application to video feeds, and the potential to become easily portable) are important ones, and could change the technology from something that’s neat to play with, to something we use all the time.
When combined with traditional augmented reality, DR could let you swap out one image for another. Not just addition or subtraction, but possibly seamless replacement. Once digital graphics become sophisticated enough, AR and DR could completely rewrite the world we see. Provided we’re looking at that world through a digital camera. But that’s probably what many of us will be using. Computer displays will likely be built into our car windshields, our windows, or the glasses we wear. Everything will have a video on it, which means that everything will have the potential to be AR and DR edited. A really smart guy I know got pretty paranoid when he described this technology to me. What happens when we all have cybernetic eyes, Aaron? Someone’s going to abuse this sort of technology, hack your vision, and make themselves invisible!
Well, I’m not sure that we need to worry about that just yet. Still, it’s kind of amazing that such a simple idea – removing objects from a video -can have such a wide range of applications. Diminished Reality is likely to be a common part of the next generation of augmented reality tools. Give it some time, and some major advances in video headgear, and DR could be erasing the ugly bits of your world too. Sorry advertisers, better stick to radio.
[image credits: Ilmenau University of Technology Department of Virtual Worlds and Digital Games]
[sources:Ilmenau VWDG Press Release (De)]
