Why is there a giant insect crawling...oh, wait, that's just augmented reality. Cool.

I don’t know if you noticed, but a parallel Earth has merged with our own, overlapping our reality no matter where we go. I’m talking about the digital realm – the confluence of internet, data, and social networks that we spend so much time with every day. As you walk around, the digital realm comes with you and augmented reality (AR) is your portal to merging that information with the world you see. With AR you can pick up your mobile phone, turn on the camera, and look as icons and virtual objects overlap with the real environment you see on your screen. Mobile AR lets you see reviews for restaurants as you glance up the street, or see new dimensions to the ads and posters around you.]

Metaio, based in Munich, is taking AR to the next level. Their free mobile app Junaio seems like just another AR interface. Fire it up, look at the world, see digital information layered on top of it. But Junaio has been downloaded 1.7 million times! Why? Junaio’s attracted 5000 third party developers creating their own personalized layers or “channels” for the app. Channels give Junaio users amazingly detailed AR tools through which they can view their world. Now Metaio is leveraging that AR marketplace in a great new way, with an AR browser. Junaio 3.0 lets you simply look through your mobile, and press scan – instantly all the channels relevant to what you’re looking at will appear. A barcode – channels about the product will come up, maybe pointing you to the nearest store where you can find it, or the best recipe that uses it. The same thing can happen with a QR code, or a famous piece of art. GPS and compass integration means you can point your mobile anywhere and the relevant channels for the area will instantly appear for you to select from. AR browsing takes thousands of independently created AR channels and turns them into one mighty mobile application. Check out Junaio 3.0 in the video below. Metaio has hit upon an incredible idea, one powerful enough to succeed no matter which company ultimately does it best.

Here’s a look at Junaio 3.0, notice how a single button press calls up a variety of different channel options in a wide assortment of AR use cases:

I had a chance to talk with Trak Lord, Metaio’s San Francisco PR and Social Marketing Director, and, perhaps more importantly, a former Junaio developer. In just three years, Metaio has attracted the attention of 5000 developers creating many thousands of channels, hundreds of which have become popular and active. Lord estimates that about 75% of channels are created by third parties (the remaining coming from Metaio). Its completely free to become a developer, and there is a tool, Ad Inject, that allows you to incorporate unobtrusive ads to make money off your channels. Many developers are like Lord was when he was developing: end-users who wanted to create cool channels for themselves and their communities. (Lord made a custom Junaio channel for San Francisco that shows you where all the good comic book stores are.) With 1.7 million users, and thousands of developers, Junaio has so much content that it recently became “harder to intuitively find a channel.”

Metaio couldn’t let that stand, and so they updated Junaio 3.0 to act as a browser. Think of the internet, if you tried to find websites you wanted just by looking through URLs, you’d be completely overwhelmed. That’s why you have a browser like Firefox and search engines like Google. And that’s why AR needs a browser like Junaio. Of course, Junaio is only looking through the channels on its own market, but it still has the right idea. If AR is to become the ubiquitous mobile tool we want it to be, you need a way to easily select which virtual objects you want to see super-imposed over reality. It’s a brilliant step.

To give you an idea of the AR channels Junaio 3.0 lets you select from, here’s their demo reel. The following video shows you several real-world channels created by various developers:

To Lord, “life should be just as browsable as the internet.” You can see that goal in Junaio. An internal web browser lets you click on a point of interest (POI) icon and jump to a website without ever leaving the Junaio app. So you can read a restaurant review quickly, and then go back to scanning up and down the street without missing a beat. Junaio also lets developers embed all kinds of great media in the POIs they create on their channels. Videos, images, panoramic shots…imagine walking through a neighborhood looking at houses for sale – you could click on a home and take a 360° tour without ever stepping foot inside. Metaio really wants you to be able to explore the digital while you explore the physical (and vice versa) and their Junaio AR browser capability, integrated with all their great AR channel features, makes that possible.

Metaio also wants to “make the digital a natural experience” as Lord says. They’ve been working on 3D shape recognition and object tracking. Basically that means that you’ll be able to look at an object through your mobile camera and Metaio will know what that object is, and follow it on your screen. Lord says they have test cases where you can look at a printer and see an AR overlay on how to insert the cartridge correctly. Pretty amazing stuff. In the following video, Metaio demonstrates on a scale model of a city how their 3D object recognition and tracking will allow for ultra accurate AR in true real-world environments:

Here’s a local news segment out of San Francisco featuring Lord as they review some of the features of Junaio 3.0.
*Note: The 3D tracking discussed is still in the prototype phase.

There are many things that make Junaio an innovative AR product – they have tons of content (the channels) and they curate it well (both by having a browser, and by providing optional certification to developers to help them create those channels). But, of course, Metaio isn’t the only AR company. I’ve discussed Dutch firm Layar with you before, and they too have a wide variety of content overlays (they call them “layars”) developed by third parties. We’ve also seen Google Goggles, which lets you perform visual search simply by pointing your mobile camera at something. Junaio has advantages over both these other options (Layar is mostly GPS/compass based while Junaio handles a wider variety of inputs and Google Goggles isn’t really AR, just visual search), but that’s not really important.

When you let third party developers free rein you get get a wide variety of applications. Junaio has many different channels handling many different sides of augmented reality.

The real impact here is that Metaio has struck upon a brilliant idea – an AR browser for all the different AR channels people produce. They may not have been the first to come up with that idea and I definitely doubt they’ll be the last. The idea is just that good. Again, imagine the internet without search engines. It just wouldn’t work – at least not as well. Allowing people to create their own augmented reality content opens up a world of possibilities. AR browsing harnesses those possibilities into an application people can actually use.

Browsing is now a must for anyone hoping to create their own mobile AR network. It’s that simple.

One could ask, of course, how important AR markets really are, and that’s a valid critique if you’re in the US. America simply hasn’t jumped on the mobile AR bandwagon in a significant way. I live in San Francisco, tech hub of the West, and I’ve never seen anyone looking through their mobile at the world to get AR content.

As we’ve discussed before, however, Europe and Asia are a different story. Lord confirms that those markets are loving AR content. QR codes on buildings, virtual butterflies you can collect for coupons – the application space is exploding and non-US markets are riding the shock wave to new technological frontiers. Honestly, I’m kind of surprised we’re not doing the same thing here.

So if you’ve never explored AR mobile, it’s probably time you check it out. Go get the Junaio app for iOS, or Android. It’s free, there’s tons to explore, and you can become a developer in minutes if you feel so inclined. As with many AR applications, the power/speed requirements are a little harsh. I’ve seen it run well on iPhone4, various Samsung devices, and iPad2, but older models…well, give it a few years and AR should run fine on everything. In the meantime, push your boundaries a bit and take a look at the digital realm around you. Metaio is just one of many companies ready to give you a portal to travel between worlds. How can you pass up an adventure like that?

[image and video credits: junaio/metaio]
[source: junaio, Trak Lord]

The US Intelligence Advanced Research Projects Activity (IARPA) has been aiming to trace criminal and terrorist activities perpetrated online. Started in 2008, their “Project Reynard” is specifically geared towards uncovering recruiting and training operations going unnoticed in online gaming environments. Check out the news story explaining Project Reynard in the video below. While the IARPA project is unlikely to produce publicly accessible results anytime soon, anxiety over the use of virtual reality for terrorism is already increasing. The recent bombing and massacre in Norway, and a Dutch best selling novel have also heightened the public awareness of terrorists benefiting from online games. Yet even if terrorists can be found in virtual reality the way they have been found in every other communication system, will kicking them out really end their operations?

That terrorists have infiltrated online games is no longer largely questioned. The US National Intelligence director acknowledged the threat in his Data Mining Report in 2008. This is a real phenomenon, a real “threat to national security”, and that blows my mind. When did hard-line killers get together and say, “you know where we can organize our attacks? The same place where nerds go to pretend to be superheroes and kill bad guys.” The irony there is palpable. What remains to be seen, however, is if governments will be able to accurately quantify the size of this threat, and neutralize it. That’s where Project Reynard comes in, as Bill Moyers explains:

Bill Moyers’ coverage of IARPA’s Reynard gets a little hyperbolic (suddenly 2008 seems like a lifetime ago) but the concerns over the breadth of the project are real. Documentation for Reynard shows that the goal isn’t so much to find specific terrorist cells using specific online platforms, it’s to develop a way of discovering any kind of suspicious behavior occurring in these virtual realities. Essentially, IARPA wants to have a reliable means of quantifying the way characters act in these games so that they can draw reasonable conclusions about the people controlling those avatars. What does the guild you join on World of Warcraft, the shops you frequent in Second Life, and the characters you chat with reveal about your nationality, your economic status, and your inclination towards real world violence? It’s a little funny (weird, not ha-ha) to think of terrorists resorting to online video games as secret hideouts, but it’s considerably less so to consider governments will now be trolling through everyone’s game statistics to find those criminals.

By the way, if connecting digital behavior to personality traits seems like a foolish endeavor, remember that scientists are finding ways to translate the smallest facial tics and physical gestures into profiles to find terrorists entering public locations like airports. With enough data, even virtual clues may be able to reliably identify groups with terrorist associations. From there, it’s just a matter of data mining and analysis to determining who is “suspicious” and when and where they are gathering. The same basic approach is already used to cull through millions of emails, phone calls, and text messages.

Yep, these are definitely terrorists.

Project Reynard launched in the last quarter of 2009 and is set to conclude sometime next year. It will join other online intelligence gathering operations, like the EU’s much larger Project Indect, aimed at combing through huge amounts of data looking to find evidence of criminal or terrorist activity. While there have been noted examples of terror organizations like Al Qaeda recruiting members through email, porn sites, and other internet forums, considerably less is known about recruitment and training in online games. That’s because so much that happens in those virtual worlds is simply unmonitored…for now. While privacy terms for most virtual worlds like Second Life make it clear that your actions in most online environments is considered public, most of us can converse freely without worrying about government eavesdropping. Reynard (and to some extent, Indect) may change that. While we’ve grown accustomed (or oblivious) to the automated sifting through our mobile calls, emails, and text messages, it’s going to be extra creepy to think that IARPA may be tracking who I talk to while role-playing a giant anthropomorphic panda.

Not the good kind of cosplay. Maniacs like Breivik will always be with us, and for now virtual games may hold an appeal for them. But we shouldn't blame games for Breivik anymore than we should blame J.D. Salinger for John Lennon's death.

Even before Reynard is ready to share its results, crime fighting organizations in the US have already taken up monitoring online gaming to catch illegal activities. According to documents captured by LulzSec, and released on Public Intelligence, New Jersey and New York authorities (in cooperation with the FBI) have monitored conversations made by gang members on the Play Station 3 and Xbox 360 consoles related to past and future crimes, some international. It seems gangs figured out that most voice over internet protocols would largely go unexamined. While clearly that is less true than they assumed, VOIP is probably still a safer bet for clandestine activities than landlines and mobile phones. In-game discussions, say in a rousing match of Halo may be even more difficult to pin down at the moment. I’m sure police and criminals are learning what Reynard already assumes – anytime you have a virtual location that enables one-to-one communication, you have the possibility of its use for facilitating illegal activity.

Of course, some people don’t even need the one-to-one communication to help them commit terrorist atrocities. Anders Behring Breivik, the gunman responsible for killing dozens of children and camp staff in the recent attack in Norway, left a massive manifesto describing his motivations and preparations. Apparently the fiend used Call of Duty: Modern Warfare 2, which has many realistic depictions of 21st Century military operations, as a training ground for his attack. Reactions to Breivik’s use of online gaming has provoked a wide range of reactions, from condemnations among gaming communities to knee-jerk calls for banning all sales of the game in Europe. No matter how you feel about violence in video games, however, Breivik’s activities suggest the possible uses terrorists may find in virtual worlds without using them for communication.

An international best seller, MMORPG raises concerns about how terrorists could easily use online gaming to recruit and train operatives. Of course, Dan Brown raised similar concerns about albinos and the offspring of Jesus so...

Dutch author Emile van Veen is far less subtle, preferring to outline exactly how terrorists can use video games to kill us all. His international best seller, MMORPG, is a Dan Brown-esque thriller wherein a duo of unlikely heroes must race against the clock to unveil a hidden terrorist plot centered around the use of a massively multiplayer online game. Schlocky? Yeah, maybe, but since its release in March of this year, van Veen’s MMORPG has raised public awareness of the links between terrorism and online gaming to new heights. Based on “two years of research” into the realities of virtual assisted terrorism, MMORPG paints a grim picture. Aided by the anonymity and accessibility of online gaming, terrorists can talk to each other almost anywhere, anytime, without the threat of being detected. At least, in van Veen’s fiction. It’s scary to contemplate that the same may be equally true in the real world.

As always with the ongoing “War on Terror”, the risks and defenses seem almost equally horrible. Clearly some lunatics will use video games as inspiration or preparation for their murder sprees. I doubt we’ll ever be able to stop that completely – such crazies will always find something to fuel their need to hurt people. What’s both more concerning, and more controllable, is the use of virtual worlds as staging grounds for organized terrorist attacks. It’s disturbing to think that the only solution proffered to fight such organizations is a reduction of everyone’s privacy while playing these MMORPGs. The loss of freedom to fight the loss of life seems like a zero sum game to me.

Yet it’s also the only game intelligence agencies seem to want to play. Before we sign over all privacy in virtual reality, we should remind our governments that while online games may temporarily provide ideal circumstances for terrorists to communicate, the advantages there won’t remain unique. Any virtual location where people can go unmonitored is going to be a problem. Even if Project Reynard effectively closes virtual games to terrorism, another such online opportunity will arise. Trolling World of Warcraft for terror cells may be necessary, but it’s a short term fix. Ultimately we’re going to have to fight the sources of these movements (both in the real and virtual worlds) if we’re to have any hope of them ending.

[image credits: Erwin Franzen, Emile van Veen, Andrew Berwick ]
[video credits: Bill Moyers/PBS]
[sources: IARPA, EmileVanVeen.com, Public Intelligence]

We need to go deeper: a virtual reality inside a virtual reality inside a virtual reality.

Locating your indie film festival in the real world? Meh, that’s so mainstream. The fourth annual Machinima Expo is coming this November to a virtual world near you. Taking place in the online game Second Life, MachinExpo features movies made by recording events in virtual realities, often dubbing over them with music and original dialogue made by the film’s creators. Real filmmakers recording virtual characters to make virtual movies which are then shown to real people via their virtual characters in a virtual world. It’s like Inception, only more pixelated. Watch trailers and highlight reels for MachinExpo in the videos below. While my mind is blown by the meta-ness of a virtual film festival for a virtual audience, I’m excited by the potential. Not only is this new genre of movie making growing, it’s demonstrating how rich and consuming virtual life can really be.

Machinima (pronounced like ‘Cinema’) is the catch all term for videos made by capturing footage from virtual realities, typically games. Common sources for this raw footage include: Second Life, Unreal, Sims 3, and World of Warcraft. Quite often the virtual characters in these games are recorded (using screen capture software or a graphics engine) doing mundane things like waving their arms, walking around, examining objects, interacting with others, etc. Once a Machinimator collects the clips they need from the virtual reality they can add in layers of their own content. Most common is replacing the audio with your own dialogue – you can make the soldiers, housewives, and dwarves of these games discuss philosophy, sing along to famous songs, or just make asses of themselves. Or, you can try to create a substantial piece of art, which is the goal of many who submit to the MachinExpo. The following is a trailer for MachinExpo featuring clips from some of the best short movies of 2010:

If you really want a taste of what these short films are like, however, here’s the 2010 highlight reel. There are close to a dozen brief Machinima movies included, so feel free to skip around. You can find a list of the sources for these segments here.

To those new to the genre, Machinima can seem pretty freaking weird. Why would I want to watch over-dubbed versions of video game sessions? Well, first off, many of these movies are actually decent pieces of art that stand on their own. Beyond that, I think one of the biggest draws is an attachment to the virtual worlds used as the raw footage for these films. When you spend hours playing Second Life, Sims, or World of Warcraft, you start to become invested in the reality of the game. You give the characters you play back-stories, interests, etc. Watching someone’s Machinima film is kind of like watching someone making a movie with the characters from your imagination. Not only can that be engrossing, it adds a fascinating new level to your virtual experience.

A level which is abundantly evident in MachinExpo. While it can be watched via their main website, the film festival mainly takes place in the virtual reality of Second Life. People log in to the game, take their avatar to a virtual screening, and then watch the movies. Movies which, again, feature characters much like the avatar that’s watching them. Virtual actors and virtual audience members: in the context of Second Life, it’s make a certain sense.

To the real world filmmakers behind these short videos, Machinima can offer some real benefits over traditional formats. While recording standard gameplay is generally considered illegal, recording non-traditional uses for game characters is generally considered fair use. In other words, you can’t make a movie of yourself playing Super Mario Bros all the way through, but you (probably) can make a movie of Mario walking back and forth as if he were searching for his lost keys. That fair use means that Machinima filmmakers need much less money to produce their works. Virtual actors work for free, and the virtual realities have already been built for you. Many of the submissions for MachinExpo were made by individuals or very small groups. If you need a place to break into the movie industry, Machinima isn’t a crazy place to start.

But maybe the point isn’t to go mainstream anyway. Hanging on the fringes of the art scene since the 80s, Machinima is slowly becoming more prevalent, if not exactly more respected by the independent film industry. Machinima.com, a site built around the community of virtual movie makers and their products, is boasting 2.3 billion views a year! Browse through YouTube, and you’ll find that there are many (SO FREAKIN’ MANY) videos made using over-dubbed clips from virtual realities and games. It’s not just an art form, it’s an art form for the virtual masses.

MachinExpo is currently taking submissions for through September. The festival itself will not only feature these short films, but also panel discussions, live filmmaker interviews, and even keynote addresses – all through Second Life. I strongly encourage you to check it out in November. It may be a bit weird, but it’s an interesting window into the art of the future. As virtual technology gets more realistic, and more widely used, these Machinima films are only going to get better. Someday, we may all be watching them…in our virtual houses…with our virtual avatars…in our virtual lives…

…my brain is starting to hurt. Here, let’s end things with one of the most popular (ongoing!) Machinima series ever made: Red vs Blue Season 1 Episode 1. Enjoy the Halo-larious hijinks.

[screen capture credit: Phil Rice]
[video credits: collections edited by Ricky Grove and Phil Rice, Red vs. Blue created by Rooster Teeth Productions]
[sources: Machinima Expo]

Tag Systems unleashes the untamed speed demon in all of us. And it's completely safe.

Daredevils have an insatiable need for speed, but they play with fire when pushing their vehicle or driving abilities too far. Tag Systems offers a unique solution by transporting the motorist and their physical car to a virtual road, a first in driving simulation technology. Sensors collect real-time performance data on an actual automobile, giving you a first-person perspective in 3D world. Basically, you can accelerate, steer, and brake inside your own car as you would on the open road, but the vehicle is driving in place on steel rollers. This one-of-a-kind VR system allows drivers to approach 150 mph on renowned Formula 1 courses, compete in Fast and Furious-style drag races, or even cruise the surface of the moon. And the best part is that you’re driving your car, not the half-cabin, video game-like simulators of the past. You can really push your driving prowess to the edge in a completely safe environment. Still unconvinced of its unparalleled realism and awesomeness? Check out the videos below, and prepare to be amazed. Virtual speed demons, start your engines!

This video showcases the TS8000, Tag System’s most sophisticated unit. It begins with an overview of the concept and provides a brief description of how it all works (0:47). The steel rollers pick up torque, power, and braking data while twin lasers measure steering. All this is fed to a central computer, which syncs this information to a fully immersive environment projected on three massive screens that envelop the driver. The most practical applications are highlighted (1:23): pre-license education, practice for advanced drivers, and training special permits. These driving exercises are highly effective, because hazards are generated at will, honing driving proficiency in high-risk situations.

Education is all fine and good, but can I pretend I’m Mario Andretti? They get to the fun stuff at 2:44, with a speedster deftly maneuvering a race course. This is followed by the Drag Tag (3:07), a racing system meant to keep Vin Diesel wannabes off the street (see the video below for a more complete demo). It concludes with the Driving Proving Centre, where the Tag System’s virtual worlds are created by CGI wizards. Check out the virtual moonscape at 4:22!

The next video shows the Drag Tag racing system in full force. Multiple units are lined up side-by-side, pitting drivers against one another in a virtual race that make video games like Need for Speed or Gran Turismo look like child’s play. With Drag Tag, you can put your racing buddies to shame without making State Farm or Geico cringe.

Tag Systems even caught the eye of perhaps the world’s most famous car aficionado, Jay Leno. Recently, the TS8000 was featured on his website, Jay Leno’s Garage, as you can see in the video below. At 4:45, Jay bravely steps into his yellow Corvette to give the Tag System a whirl. You can tell by his facial expressions and quips that he’s really digging it. The Tag System evokes a physical response in Jay at 7:08 as he navigates a parking garage. While this speaks to the realism of the system, this feeling arises because of a lack of G-forces when hugging the turns. Maybe a tilting mechanism could somewhat compensate for this? Tag System’s general manager says prior trials have shown that the body adjusts to this after a few runs (8:10). As Jay reiterates at the end, this was only the first demo on US soil, so Americans shouldn’t dust off their ’69 Charger just yet.

For me, one of the most exciting things about Tag Systems is the ability to create an endless number of new courses and terrains for your vehicle. When the whole globe is accurately mapped in stunning detail, I think one day we’ll be able to recreate driving experiences for any location on the planet. Hopefully, Tag System graphics engineers will take advantage of this trend and continue expanding their catalogue of virtual environments. Who could resist speeding across serpentine roads atop the Great Wall of China?

This technology enters the scene at an interesting time in automobile history, when people could soon let computers take the wheel or link with other cars to generate collective, synchronous driving patterns. However, technology like the Tag System could help restore the visceral thrill of untamed driving by letting users drive how they want, wherever they want. Even the best drivers in the real world can’t drive like that on today’s crowded roads and highways.

But how long will the Tag System be relevant? If computers become better motorists than we are, it might be unnecessary for most people to sharpen their driving abilities on the Tag System. Also, virtual reality technology is improving all the time, and VR engineers of the future could replicate the realism seen in these videos without needing an actual car. Capturing the subtle differences between make and models will be challenging, so it might be awhile before this comes to fruition. In the meantime, Tag Systems technology will likely remain on the leading edge of driving simulators. You can bet I’ll be the first in line if a Drag Tag demo comes to my town. I wonder if my mom will let me borrow her car. :-/

<Image Credit: Tag Systems>

<Video Credits:  Tag Systems, Jay Leno’s Garage>

<Sources:  Tag Systems, Jay Leno’s Garage>

Tron Legacy Looks Pretty Sweet

Like most of the big films being released recently, Tron will be available in 3D.  With Tron’s signature high speed lightcycle chases, glowing disk projectile weaponry, and a simulated digital universe the film is ripe for some 3D awesomeness.

Will Neon Suits Be New Fashion Trend?

When the original Tron movie was released in 1982 it was a revolutionary, daring production seen as high risk from its producer Walt Disney.  Computer programs were a barely emerging concept in a society of Pong, mainframes, and a sizable dose of skepticism and fear about the potential for computers.  In a sign of how the times have changed, today’s Tron sequel will fit naturally into our world of fast converging digital and biological technologies.  Tron’s virtual world of humans and programs projecting themselves as digital avatars is a common theme in movies (Avatar) and TV shows (Caprica), and truly exists in the real world to a certain degree (Second Life, Augmented Reality, Foursquare, Video Conferencing).  It is increasingly hard these days to think of futuristic advances that aren’t already being worked on in labs and corporations around the world.  Enjoy the futuristic (but not that futuristic!) trailers below:

Latest trailer:

Previous teaser trailer:

Here is a bonus treat for you, the trailer for the original Tron from 1982:

[image credits: Walt Disney Pictures]
[sources: Wikipedia, The Internet Movie Database]

Avatar Movie an animation marvel?

The Avatar Movie from renowned director James Cameron is set to open in theaters worldwide this week.  Cameron, creator of movie greats including Terminator, Aliens, and The Abyss, not to mention Titanic, the highest grossing film of all time, has not released a movie in 11 years.  Touting a more than $300 million budget, reportedly the most advanced special effects ever seen in movie making, and the prowess of Cameron himself, the hype for the Avatar movie would seem impossible to live up to.  Yet indications from last week’s premier of the film in London indicate that the movie just might live up to viewers’ lofty expectations after all.

Those who attended the London premier were reportedly raving about the stunning animation and overall wonder of Avatar’s fictional planet Pandora and its inhabitants.  Several viewers have noted a weak plot fraught with overused cliches,  yet the overall consensus seems to be that the impressive animation and vision of this film make it a must see movie destined for box office success.  For what its worth, Rottentomatoes has given the movie a respectable 84% rating.

Movie reporting isn’t a staple for us here at the Hub, but movies like Avatar loaded with futuristic visions of telepresence, alien civilizations, and medical marvels are right up our alley.  From a technical standpoint the Avatar movie gains our attention because it represents a radical shift to more realistic, more immersive movie animation.  The Avatar movie has amassed the budget, the credibility, and the talent required to create a new standard for movie animation techniques that will set the pace for the rest of the industry in the years to come.  Just as with the Star Wars films before it, the Avatar movie is an animated marvel today, but in only 5 years will look dated and normal as it propels the rest of the industry to a new level of greatness.  After seeing the Avatar movie viewers will expect more, and we will all benefit from a new generation of more realistic movies that entertain our imaginations.

Avatar's Pandora Planet

With its wide release in a format allowing viewers to wear glasses for full 3D immersion, Avatar joins an impressive host of 3D films that have been released this year.  As we wrote earlier, although 3D movie technology has been available for well over 50 years, 2009 will mark the year that the technology has overcome political, financial, and technical hurdles to finally break out into mass production in theaters across the globe.  This is just the tip of the iceberg of a trend in virtual realism that is bridging the gap between the physical and virtual worlds.

As long as we enter the movie with the right expectations of stunning animation supported by a mediocre plot, the most anticipated movie of the year looks to be a winner.  In a toast to the future (and supposedly a better viewing experience) be sure to check out the movie in 3D if a theater near you supports the format.  Pandora, here we come!

Below is the trailer in case you live under a rock and haven’t seen it yet, or if you understandably just want to see it again:

Brother Industries, Ltd. has just announced that next year it plans to produce Retinal Imaging Display (RID) glasses that augment regular vision.  Just as in the popular Terminator movies, these glasses could project supplemental information into your field of vision to augment your normal vision.  Although Brother and most of the blogosphere is hyping that the product will be ready by 2010, I am skeptical.  Even if they do appear in 2010, will the quality and form factor of the glasses be reasonable?  I doubt it.  Augmented reality sent directly into our retinas is certainly on the horizon, and its going to be awesome…but the technology still needs several more years of innovation before it is ready to break out.  In the meantime, we can wet our imaginations with what is to come.

So how do the RID glasses work anyway?  For those that want to get more depth on how the whole RID thing works, a decent place to start seems to be available from the US Navy.  Sadly the information from Brother is pretty thin, only adding to my skepticism of their product plan.  The glasses are equipped with an attachment piece that literally projects light onto your retina.  The light source for the eyepiece comes from a hard drive sized laser generator that you would have to carry on your hip or somewhere on your body.  Brother claims that the images are transparent enough that they don’t interfere with the real visual field of what you are seeing.

It sounds great in theory, but does it really work as advertised and without complications?  What about the images causing headaches due to temporal distortion, image stabilization, and other factors?  How is the augmented information correlated in realtime with the real images streaming into your field of vision?  Will the images being projected into your retina be of high enough resolution to see writing and other informative data crisply?  All of these issues and more will need to be addressed before Brother is able to offer a viable product.

Don’t get me wrong – I certainly commend Brother for working to develop such a neat product and I am excited to see where they can go with this.  I just think we need to set reasonable expectations for the timeline of their product development.  The Hub has reviewed several augmented reality applications that already exist on iphones and elsewhere and even a contact lens concept, and now Brother introduces us to the possibility of wearable awesomeness.  The real and virtual worlds continue to collide, but it will still be several years before this collision starts to make some really big waves.  In the meantime maybe its time to watch that Terminator movie again – I’m pretty sure augmented reality is not the only part of our future that the movie foretells…

Oh my goodness, this world looks vaguely interesting!

With the amazing video recording and production systems commercially available, how hard can it be to create a realistic virtual reality environment? I mean, all we really need are some cameras, some computers, and a video screen, right? Well, Virtualization Gate, a new project from the INRIA and Grenoble University in France, debuted at SIGGRAPH in early August. Apparently realistic VR is harder than it looks. Or maybe everything else we encounter, cutting edge video games, CGI films, and high definition projectors, just make VR seem less real than we would like. Check out the Virtualization Gate demo video after the break and marvel at their big green room of goodness.

What Virtualization Gate does well is track the user’s motion. A system of multiple cameras, a PC cluster, and a head mounted display (HMD) the size of Sputnik are used in conjunction to place the user precisely in the virtual world. With 20 frames per second, the VR graphics actually appear to move reasonably well in real time. This allows our demonstration gamer to kick over some urns, push around a virtual copy of himself, and even stare at his VR avatar in the mirror. Not bad, VGate, but not mind-blowing.

So credit to INRIA and Grenoble for putting together or creating the best technologies at hand to get what appears to be a fairly acceptable VR system. But I’m sorely disappointed. Not in this group, who I applaud for making the attempt, but in the whole VR concept. I mean, do any of us even really understand what we hope to accomplish with VR?

With augmented reality the purpose is much clearer. We add digital information into the real-world view to make it more useful or even just cooler. Totally immersive VR, however, is not so easily defined. Do we want to touch as well as see and hear? If so, we’re going to need a system of haptics, maybe some version of the tactile holograms system also seen at SIGGRAPH. Do we need to smell? Taste? Should one’s sense of gravity, pain, body position, or time also be controllable?

One of the problems is that virtual worlds are out-pacing virtual interfaces. When we watch a VR environment on a movie screen, it follows a single view. This allows for the maximum amount of complexity in the image. The same for sound. Adding an interface, the HMD and cameras, and you’ve jumped the single view track. Now your VR environment has to really work its physics engine, and obey a set of interface rules that gobble up processing speed and limit the reality of the images. Movies and games are poised to cross the uncanny valley, VR seems destined to be mired within it.

I just don't think we're going to accomplish VR in a place like this.

I look at our earlier story about Braingate, a device which monitors motor neurons to control cursors or mechanical devices, and I see a much greater possibility for VR than when I look at VGate. Cameras and big green rooms seem almost childish when compared to directly reading brain signals. Add in the analysis of mental states provided by fMRI scans of the brain, and it seems like that avenue is a lot closer to creating a VR interface than anything else on the market.

Why not harness the brain for processing as well as for interfacing? Every night most of us have dreams that put VR to shame. Devices that could take advantage of the brain’s inherent ability to generate virtual worlds with changeable physics would be ideal. The concept may sound far-fetched, and I don’t really want to jump into some crazy discussion about lucid dreaming, but is it that much nuttier than thinking we’ll ever get a believable VR from PCs and HMDs? Developing a truly immersive experience, one that allows the user to experience a virtual world as a real one, is going to be an epic journey. My guess is that traditional methods of virtual reality are on the wrong path. What do you think?

Say goodbye to boring buildings with art installations from Urbanscreen

If this trend catches on, we could see entire cities changed at night into evolving art spaces. With the sagging economy, artists are clamoring for grants and jobs…so why not combine that need with a little urban renewal? The Kunsthalle installation was based on the question: “What would a building dream?” That sounds esoteric, but the project came out very crowd-pleasing. A city-wide project might have the same great combination of artistic merit and public appeal. And it would certainly boost tourism.

But maybe that’s thinking too small. We’ve talked about Total Immersion’s Augmented Reality technology, and how it can combine recorded and synthetic images in real time. Mix AR with building-scale projectors and you could have a city with one foot in virtual reality. The creative potential is staggering. Of course we’ll probably see most buildings turned into billboards, but that might be a small price to pay to live in an interactive urban playground.

As far as I can tell, the technology is pretty basic: mammoth projectors, cameras, and hours upon hours of work on a computer. For all that effort, I’m surprised that the installation only lasted a short time. I might have left the thing up permanently. I should point out that Urbanscreen and the Kunsthalle installation are far from unique. Many artists have projected images onto 3D shapes before. Pablo Valbuena actually did a very similar installation in the Netherlands. As a general trend though, this is pretty cutting edge. And it looks awesome. What more do you want? I look forward to the possibility of traveling down my street and watching the buildings dream.

]]> http://singularityhub.com/2009/08/10/forget-paint-turn-the-side-of-your-building-into-a-video-screen/feed/ 11 http://singularityhub.com/2009/07/31/haptics-unleashes-virtual-reality-and-telepresence-revolution-awesome-vids/ http://singularityhub.com/2009/07/31/haptics-unleashes-virtual-reality-and-telepresence-revolution-awesome-vids/#comments Fri, 31 Jul 2009 18:00:01 +0000 Keith Kleiner http://singularityhub.com/?p=5629

Exohand - Its Haptic Baby!

Prepare to be blown away with this post!  Here at the Hub one of the things that we are all about is the coming merger of the physical world with the virtual world.  Programmable matter, moving beyond the outdated mouse and keyboard interface model, immersive environments, and so on.  Now we are here to tell you about a seriously game changing technology that is absolutely busting apart the traditional barrier between the physical and virtual worlds.  Its called haptics, and if you haven’t heard about it yet then read on and prepare to be thrilled.

Dramatic opening paragraph, I know, but I am just so excited about the possibilities of haptics that it is hard for me to hold back.  Haptics is the set of technologies that causes a person operating a device to engage in a sensory feedback loop between what the device is feeling and seeing and what the person is feeling and seeing.  Sound confusing?  Keep reading, it will make sense!

Today most graphics designers draw things on a computer the old fashioned way using the mouse and keyboard to interface with their design software.  Haptics changes all of this.  Imagine instead of a mouse that you have a wand or joystick that can serve as your paintbrush, eraser, scalpel, etc as you design an object on the computer.  As you push down on the virtual object you are designing on the computer with your joystick you can actually feel pressure push back on you from the joystick.  Its a completely different way of graphics design that will rewrite the rules of the game.  Speaking of games, haptics also has the potential to revolutionize the gaming industry. Check out this video to see what I am talking about:

After watching the video hopefully you agree that this is some pretty cool stuff!  To fully grasp the potential of this disruptive technology, you have to keep in mind that the haptic device you are controlling does not have to be in the same physical location as the object, game, or target you are manipulating.

Imagine a surgeon performing surgery on a patient half way across the world using a haptic device.  The surgeon would be able to feel, see, and even smell the organs and tissue of the remote patient with a sensitivity and realness that is indistinguishable from physically being there.  Of course the technology still has a ways to go to get to that point, but not as much as you might think.  Remember our coverage of the da vinci surgical robot?  This robot is currently taking the practice of surgery by storm as it allows surgeons to operate a suite of scalpels, cameras, pincers, and other devices to perform surgery with a level of precision and agility that exceeds natural, unaided human ability.  Several efforts are underway to augment the Da Vinci robot with haptic capabilities. And as you will see in a video at the end of this post, haptic based remote surgery has already been achieved several years ago. Yikes!

Graphics design and gaming seem to be the first killer apps that will be amenable to haptic enhancement.  The gaming industry in particular has the financial incentive and the platform to make this happen soon.  As seen in the video above, products are already on the market for the gaming industry and the real problem is not the technology, but rather the integration of device and game.  Once the gaming industry can work with the haptic device industry to create an open standard that will allow any haptic device to interface seemlessly with any game we might see an explosion in haptically enabled gaming.

Aside from the Falcon device from Novint, there are some other serious players in this field.  Two others you should check out if you want to learn more are Sensable with their Phantom line of products and Quanser with their large suite of products.  Here is an illustrative (hah!) example of the Quanser device in action:

The potential for haptics is just out of control. We’ll be keeping an eye on this emerging field continually here at the Hub, so stay tuned for updates in the future. In the meantime, haptics is a technology that lends itself very well to video demonstrations, so lets end this post with yet another video demo that is pretty cool as long as you can tune out the cheesy host:

The dome from outside, looking like a scene from one of the Myst games. Photo courtesy of UCSB

In the early days of science, checking out your data was easy: there wasn’t very much of it. Collecting data was the hard part; analyzing and cataloging it was a relatively straightforward affair.  But the times, they are a-changin’. Nowadays, complex research can spit out mountains of data, making analysis a confounding and time-consuming process. Plus, it can be pretty boring. That is, unless you have an AlloSphere handy.

The AlloSphere is a data presentation instrument – environment, really – that allows researchers to explore complex datasets in a fully immersive audiovisual environment. It is composed of two 5-meter radius hemispheres with a catwalk along the middle for researchers to stand on. From this perspective, they can watch visual representations of their data surround them, whether they are interested in neural activity or quantum fields. At the same time, other data can be represented in audio form, making the data exploration a multimedia affair. And to top it off, it’s real-time responsive, letting users manipulate the data they look at.

To get an idea for its broad applications (and to see it in action) check out this amazing video from this year’s TED conference:

Developed over the process of twenty four years, the AlloSphere is housed at UC Santa Barbara’s California NanoSystems Institute building. Its unique construction provides a 360 degree visual field for data to be presented, projected onto the hemispheres by video projectors. About 500 speakers are placed around the sphere, providing 3D sound to accompany the visual display. It takes up a 3-story cube, surrounded with sound absorption material so that audio doesn’t echo. The catwalk can fit from 20-30 people inside at a time, so your whole chemistry class can take a field trip.

So what kinds of data does the AlloSphere accommodate? Physicists seemed to have jumped right on board, using the system to visualize hydrogen bonding and quantum mechanical wavefunctions. Another project is working to provide a 3D visualization of nanoscale structures (yes, with the silly glasses) that a user can interact with and manipulate. My favorite is the AlloBrain, which lets the user navigate macroscopic fMRI datasets and hear blood flow in the brain as audio. Heck, they’re even trying to get Google Earth running on this thing.

The AlloSphere in action. Photo courtesy of UCSB

Got some ideas for the AlloSphere? They’re open to suggestions. The hardware is all up and running, and they’re looking for new software ideas to explore the possibilities of how their new toy can be applied. Their doors are open to art projects, green technology, nanotech, national defense applications, medical breakthroughs, psychologists, physicists, educators, you name it. If you’ve got a cool idea that could use a little audiovisual immersion, the Allosphere is listening. I personally promote a Pink Floyd laser light show, but that’s just me.

The AlloSphere gives scientists an innovative way to look at their data, hopefully inspiring a few “eureka” moments along the way. It also makes science really cool, engaging young students by bringing lofty concepts back down to earth and into an IMAX style presentation. We’re getting a glimpse into the future of how technology can help us to experience and explore scientific data, making it beautiful and engaging (not to mention educational).

And now, back to writing my own proposal.  Prepping next year’s tax return is gonna be a blast. Hopefully.