Word Lens Translates the Text You See In Real Time. Amazing to Behold (Video)

4,914 8 Loading
Word Lens translates what you see in real time.

Word Lens is augmented reality text translation. Welcome to the future.

Another part of the universal translator has fallen into place. Quest Visual recently released Word Lens, a killer iPhone app that lets you see the world around you translated into a new language. It's a great example of augmented reality: images from the camera are screened for words; Word Lens then translates those words into a different language, and displays the translation on your screen replacing the original text. It's like looking through a window into another world where everyone writes in the language you choose. The Word Lens App works for iPhone 4, iPhone 3Gs, iPod Touch 4 and is free to download. Language packets, however, will cost you $5, and currently Spanish to English and English to Spanish are the only ones available. Still, Word Lens is amazing to watch in action, as you can see in the announcement video from Quest Visual below. I recently downloaded Word Lens and tried out the free demo "languages" (text reversal and removal). The app is fairly limited right now, but it holds enormous promise.

Once you download the free demo onto your iPhone, it's really simple to get Word Lens started. Just tap on the icon and you're instantly in a window that is sampling from your camera and changing the letters you see. You can pause the video feed, select a region, even zoom in or turn on the light if you want (iPhone 4 only). Unlock the rotation icon and you can easily view things in landscape mode. With a little practice you can get Word Lens to find and change text all around. Billboards, gum wrappers, computer screens, you name it. When it works well this application is an absolute killer.

Quest Visual is the brainchild of Octavio Good and John DeWeese, who have been working on the Word Lens application for more than two years. Word Lens uses optical character recognition to find words in your smart phone's video feed. OCR works best with clear legible fonts, and really can't handle script or many words written by hand. It also can be hit or miss in even detecting which figure is a letter and which is part of a background design. Working with the free demo I constantly had to move the phone back and forth, or slightly rotate it in order to get Word Lens to find the words I wanted it to recognize. Octavio Good admits that this early form of the App has its limitations in the following video, but he promises that there will be more languages and improvements ahead.

When I see an idea like Word Lens I am both thrilled by the possibility and slightly disappointed by the reality. Word Lens absolutely is an amazing App when it works well. It crops out text, and replaces it with an easy to read font, and it does it all in real time. That's awesome. But the OCR has its limits, as I just said, and the dictionaries they use are far from perfect as well. I'm not sure how I feel about Quest Visual deciding to use downloaded dictionaries rather than a Google API or other cloud-based service. Having all the necessary content on your phone probably makes things quicker, and it will definitely help in all those locations where AT&T reception is poor (pretty much everywhere). Still, I'd rather have a startup like Quest Visual focus on one task and perform it very well. The OCR based augmented reality is enough to deal with and needs lots of fine tuning. Does Quest Visual really need to split their focus by worrying about their incorporated translation software as well? I'm not sure how I feel about the $5 per translation packet either. Prices will go up to $10 on January 1st. That's still a fairly good deal, but only if Quest Visual can continue to improve the quality of their product.

Word Lens Isn't Perfect

+10 points for good OCR, -100 points for bad translation. **UPDATE: As pointed out in the comments, this is actually an OCR mistake: confusing the "lt" for an 'n'. "Apunta muy ano" would mean "points very anus."**

For now, Word Lens is fairly unique among iPhone Apps for what it can do. (There is a related feature in Google Goggles that let's you translate a photo, but that's not real time like Word Lens.) That means that Quest Visual is pioneering new ground. We should expect some more bumps along their journey even as they make more amazing progress. In the end great technology like this is bound for one of three destinies. It could get bought by a larger developer (as Google did with BlindType), it could continue to grow on its own like FourSquare, or it could be replaced by competitors who improve upon their design (MySpace). I wish Good, DeWeese, and the rest of the Quest Visual team the best, and I'm really impressed by their work. That being said, I'm not sure which of the potential destinies for Word Lens will actually be most beneficial for end users like you and I.

Over the last two years we've seen the parts of a universal translator slowly arrive. Audio translators let you speak in one language and have your smart phone speak in another. Automatic text translations are becoming more ubiquitous online. Apps like Word Lens would be the next piece in the puzzle. Some time soon we should have the means to make real time translations from any form of input. That's very exciting. It would be wonderful if Quest Visual was part of that process. Even if they are not, now that they've shown us what their piece of the universal translator puzzle looks like we are sure to have it the linguistic arsenal of the future. We're getting closer to a world without language borders. I can't wait.

[image credits: Jacqui Cheng/ArsTechnica]

[sources: Quest Visual, ArsTechnica]

Discussion — 8 Responses

  • Nicolas December 19, 2010 on 2:23 am

    Ray Kurzweil should have done that.

  • Simon December 19, 2010 on 3:06 am

    This app is awesome for travelling overseas. That’s when having the offline dictionary is valuable. My mate just got stung hundreds of dollars because he updated a 40mb application while on global roaming. Ridiculous. (And as I’m an ipod touch user, I appreciate applications that work offline.)

  • Jimmyt December 19, 2010 on 2:12 pm

    Applications that aren’t tied to a 24/7 connection make perfect sense– especially an application like this. I don’t understand why everyone is so enamored with “the cloud” as opposed to actual applications. Cloud-based apps are “always up to date?” Fine– make your application automatically update itself when a new version is available! Translation could be better handled by Google’s servers? Howso? And what’s more important– having a 100% accurate translation or having the software sometimes work, sometimes not, depending on where you’re traveling, what roaming agreements are in place or what the connection happens to be like in the location where you need the translation.

    In general I think it’s better to carry water with you rather than hope that everywhere you go there is a waterfountain available to you. And the great thing about software is it doesn’t weigh anything (unless you count the memory in a device that it takes up).

  • James P. Wack December 19, 2010 on 8:09 pm

    Apunta muy alto -> Apunta muy ano (OCR mistaking the lt as a n) -> Points very anus

    the OCR and the translator are not perfect, but it’s a clear step in the right direction

    • Adsaenz James P. Wack December 19, 2010 on 11:36 pm

      @James P. Wack —You’re absolutely right, I totally missed that. That is definitely an OCR mistake, not a translational one. I’m updating the caption now.

  • wealthychef December 25, 2010 on 2:45 pm

    Note: The app does not seem to be on the store at this time.
    Another note: Reviewers are complaining that it is not free — you can download an app that does nothing for free, and you must add “language packs” to get any functionality from it.

  • Ekahan1 December 27, 2010 on 7:12 am