Word Lens is augmented reality text translation. Welcome to the future.

Another part of the universal translator has fallen into place. Quest Visual recently released Word Lens, a killer iPhone app that lets you see the world around you translated into a new language. It’s a great example of augmented reality: images from the camera are screened for words; Word Lens then translates those words into a different language, and displays the translation on your screen replacing the original text. It’s like looking through a window into another world where everyone writes in the language you choose. The Word Lens App works for iPhone 4, iPhone 3Gs, iPod Touch 4 and is free to download. Language packets, however, will cost you $5, and currently Spanish to English and English to Spanish are the only ones available. Still, Word Lens is amazing to watch in action, as you can see in the announcement video from Quest Visual below. I recently downloaded Word Lens and tried out the free demo “languages” (text reversal and removal). The app is fairly limited right now, but it holds enormous promise.

Once you download the free demo onto your iPhone, it’s really simple to get Word Lens started. Just tap on the icon and you’re instantly in a window that is sampling from your camera and changing the letters you see. You can pause the video feed, select a region, even zoom in or turn on the light if you want (iPhone 4 only). Unlock the rotation icon and you can easily view things in landscape mode. With a little practice you can get Word Lens to find and change text all around. Billboards, gum wrappers, computer screens, you name it. When it works well this application is an absolute killer.

Quest Visual is the brainchild of Octavio Good and John DeWeese, who have been working on the Word Lens application for more than two years. Word Lens uses optical character recognition to find words in your smart phone’s video feed. OCR works best with clear legible fonts, and really can’t handle script or many words written by hand. It also can be hit or miss in even detecting which figure is a letter and which is part of a background design. Working with the free demo I constantly had to move the phone back and forth, or slightly rotate it in order to get Word Lens to find the words I wanted it to recognize. Octavio Good admits that this early form of the App has its limitations in the following video, but he promises that there will be more languages and improvements ahead.

When I see an idea like Word Lens I am both thrilled by the possibility and slightly disappointed by the reality. Word Lens absolutely is an amazing App when it works well. It crops out text, and replaces it with an easy to read font, and it does it all in real time. That’s awesome. But the OCR has its limits, as I just said, and the dictionaries they use are far from perfect as well. I’m not sure how I feel about Quest Visual deciding to use downloaded dictionaries rather than a Google API or other cloud-based service. Having all the necessary content on your phone probably makes things quicker, and it will definitely help in all those locations where AT&T reception is poor (pretty much everywhere). Still, I’d rather have a startup like Quest Visual focus on one task and perform it very well. The OCR based augmented reality is enough to deal with and needs lots of fine tuning. Does Quest Visual really need to split their focus by worrying about their incorporated translation software as well? I’m not sure how I feel about the $5 per translation packet either. Prices will go up to $10 on January 1st. That’s still a fairly good deal, but only if Quest Visual can continue to improve the quality of their product.

+10 points for good OCR, -100 points for bad translation. **UPDATE: As pointed out in the comments, this is actually an OCR mistake: confusing the "lt" for an 'n'. "Apunta muy ano" would mean "points very anus."**

For now, Word Lens is fairly unique among iPhone Apps for what it can do. (There is a related feature in Google Goggles that let’s you translate a photo, but that’s not real time like Word Lens.) That means that Quest Visual is pioneering new ground. We should expect some more bumps along their journey even as they make more amazing progress. In the end great technology like this is bound for one of three destinies. It could get bought by a larger developer (as Google did with BlindType), it could continue to grow on its own like FourSquare, or it could be replaced by competitors who improve upon their design (MySpace). I wish Good, DeWeese, and the rest of the Quest Visual team the best, and I’m really impressed by their work. That being said, I’m not sure which of the potential destinies for Word Lens will actually be most beneficial for end users like you and I.

Over the last two years we’ve seen the parts of a universal translator slowly arrive. Audio translators let you speak in one language and have your smart phone speak in another. Automatic text translations are becoming more ubiquitous online. Apps like Word Lens would be the next piece in the puzzle. Some time soon we should have the means to make real time translations from any form of input. That’s very exciting. It would be wonderful if Quest Visual was part of that process. Even if they are not, now that they’ve shown us what their piece of the universal translator puzzle looks like we are sure to have it the linguistic arsenal of the future. We’re getting closer to a world without language borders. I can’t wait.

[image credits: Jacqui Cheng/ArsTechnica]
[video credits: Quest Visual]
[sources: Quest Visual, ArsTechnica]

Ignore the catch line. Intelius has far bigger goals than just a dating App.

Maybe you’ve dated someone you shouldn’t have. Sometimes that means getting bored or a little inconvenienced. For a friend of mine that meant being lied to by a con-artist for months in his elaborate ruse to get money. It’s a tough world out there so information snooping website Intelius is offering Date Check, a new sort of free App for your iPhone, Android, or (soon) BlackBerry. Using just an email address or phone number, Date Check lets you search for someone’s criminal record, investigate their living situation, scan through their social networking, and browse other public information. It’s the online equivalent of a full body frisk. Maybe not the most romantic thing to do to someone you’re hoping to have a nice meal with, but it could be helpful to people hoping to avoid being conned. Check out the video from Intelius after the break.

This sort of public information interrogation twists the idea of privacy in interesting ways. Intelius only gets a hold of files that are public: (unsealed) criminal records, registered phone numbers, registered leases/titles on homes, twitter feeds, facebook profiles, etc. Still, it puts all of that information in one access point (the App) and doesn’t mediate it. Yes, John Smith has a DUI, but was it a one time mistake or a habitual crime? Jane Doe may have a really scandalous facebook profile, but is that a sign of a personality flaw?

Part of me can’t help but think that Date Check is itself trying to pass off a deception. Yes Intelius has marketed this as a free dating App. Yes it contains a horoscope comparison feature. Yes their booth at CES was populated by scantily clad ladies spouting concerns about modern dating dangers. But let’s look beyond that, shall we? Once you get a few tidbits of information about a date, the Date Check App will route you to the Intelius website to see if you want to learn more. For around $5-$50 (depending on the type of info) you can see the really juicy public records. In short, Date Check is just a gateway drug to get people into the Intelius website and pay to research other people. Heck, the biggest market for this App may not be women on dates, it’s probably landlords and employers looking to investigate new people.

But, you know, I’m not sure I have anything against that. These are public records. Even credit reports, which are a mine field of privacy issues, are open to be viewed by almost anyone with the funds. I don’t particularly like people browsing through my information, but they can do that anyway. Date Check and Intelius just make it easier.

Or they should. After working with Date Check’s booth girls for a while I couldn’t get any down and dirty information about any of my friends. I tried phone numbers and email addresses and nothing juicy popped up. Not even for my really sketchy acquaintances (you know who you are). Now, that’s partly an artifact of my demographics. I’m young enough that most of my friends move around a lot, few own homes, and many are still in school. That’s not going to produce many records for Intelius to browse through. Still, it suggests that a positive result on Date Check is not a guarantee that the guy you’re trying to date isn’t a sleazebag. Maybe he’s just under the radar.

[image and video credit: Intelius]

Dragon Dictation lets you record speech, edit , and send via SMS, email, or clipboard.

Dragon Dictation has most of the features you’d want in a speech to text App. It allows you to edit converted text easily. It let’s you cut and paste text into other Apps. It can’t directly interact with other applications, but that’s Apple’s fault (damn the ‘no Apps running in the background’ restriction). DD is also fairly quick. Wifi connections allow for text conversions to happen in a second or less. 3G connections bring that time up to around three seconds or so (depending on connection). As speech recognition technology improves further it’s going to become even more universal. The new 3GS iPhone has voice command capabilities and Nuance provides the voice command routines for the Samsung Rogue (see the second video below). We’re likely to see the smart phones of the future run on a combination of touch, talk, and physical movement.

While many are impressed with Dragon Dictation’s capabilities, there’s some big concerns about the way it processes those dictations. Each spoken recording is sent to Nuance’s servers, converted to text, and then returned to the user. Although all the translation is handled by computers, no information is saved, and the servers are secure, reviews at the App store have repeatedly complained about the possible infringement on privacy. Even more concerns have arisen over Dragon sending your contact list to the server during translation. This information is only used to help recognize which names you’ll be using in speech, but it still worries some. Luckily, Nuance has promised to make the upload of contact optional in the next version.

I’m not suspicious of Nuance using servers to handle speech recognition, but I’m a little surprised it’s still necessary. Translation applications like Jibbigo have speech recognition, translation, and text to talk code and still don’t require connecting to a central server. Granted, the speech recognition isn’t as advanced as Nuance’s…but still, one wonders. Perhaps the Dragon Dictation umbilical cord is in place as a result of it being free software. When Nuance starts charging for the App (and I’m sure they will at some point) they may also provide a version that can run without connectivity.

Right now even the future of the free version of Dragon Dictation is a little uncertain. Despite it’s technical achievements, the program is slightly below a two star rating (as the time of writing) at the App store. Again, most complaints are not in regards to the actual work of the program but rather focus on the relay of information to the Nuance servers. Privacy is a strange thing. Google and other search engines and web email providers don’t apologize for their rampant (and automated) pillaging of your online activity to provide you with ads. Maybe Nuance would get better reviews for Dragon Dictation if they simply buried the privacy concerns in a mountain of fine print instead of being upfront about it.

No matter. Even if Nuance doesn’t find success on the iPhone, dictation programs certainly will. They’re just too useful. Hopefully whichever App eventually becomes the virtual stenographer of the future will be as capable as Dragon but without all the controversy. Oh, and while I’m wishing, please make this perfect software permanently free as well. Thank you.

[photo and video credit: Nuance Mobile]

Will the iPhone be the Universal Translator of the future?

When watching Star Trek, I was always amazed that the aliens all spoke English. Writers explained this happy coincidence by the presence of a universal translator (UT), a portable device which could take any language and translate it into any other language as needed. Our nonfictional modern world has more than 20,000 different languages (if you include dialects) in its history. With the rise of globalization we desperately need an universal translator to help us speak to one another. Luckily, there are several different companies that are taking the first steps to creating an UT. Sahkr and Jibbigo have developed iPhone Apps which transform the smart phone into a handheld speech to speech translator. Check out some of their demo videos after the break.

Computer aided speech to speech translation isn’t easy. You need speech recognition, language analysis, machine translation, language generation, and speech synthesis. We’ve had limited handheld translators for some time. Devices like Voxtec‘s Phraselator allow you to select a range of sentences and have them translated into many different languages. We’ve also had more complex non-portable systems (like IBM’s MASTOR) which can do nearly full translation. But the products created by Sahkr and Jibbigo represent a new paradigm in speech to speech: commonly used hardware devices adapted solely through software. By integrating into an existing hardware platform, these next generation programs are taking us one step closer to having a universal translator that anyone can carry in their hands.

Sahkr is one of the premier English/Arabic translation companies in the field. The translation software, Arabic Language Buddy, went live in September and is now available for download on either the iPhone or BlackBerry. It is limited to conversions between English and Modern Standard Arabic (MSA), but it works amazingly well:

Sahkr takes great pride in the realistic synthetic voices used for their translation software. I don’t speak MSA but the flow and tone of the translation is much better than I would have expected. The company press release also claims that their translations are ranked #1 by the US government. Users can even adjust the text of their translation (via keypad) to better communicate their words precisely before it is spoken. As you can imagine, this high quality comes at a high price. The software is for lease only (which isn’t that unusual for premium smart phone Apps) and the first month is $10. Subsequent months, however, can cost you either $50 or $150, depending on whether or not you qualify as an academic user. That’s no small change for the average user, but for the international business traveler it likely falls within budget.

If you’re looking for a cheaper option, and you favor the iPhone over the BlackBerry, look no further than Jibbigo. Developed by Alex Waibel at Carnegie Mellon University, and released in October, Jibbigo does most everything that Shakr does, just in Spanish and only on the iPhone. You can edit via keypad, shake the iPhone to restart, and there’s even a back translation check. All that, and you make just one payment of $25.

The difference in price between the two applications may reflect a relative scarcity of Arabic to English translators, but I’m guessing it also has to do with Shakr’s higher status as a translation service. Jibbigo is a fairly new company with hopes of expanding from Spanish and English into other major languages. Recently however, they had to deal with an audio glitch in their first released version (it’s fixed with an update).

The tele-scouter projects texts and images directly into your peripheral vision and could easily be used as a speech to text translator.

Of course, a portable UT doesn’t have to fit in your hand. NEC’s Tele Scouter is a head mounted optical display which projects words and images into your peripheral vision. Engineers claim that the use of the peripheral allows you to wear Tele Scouter for hours without headaches or other discomfort. It is very light weight, and essentially looks like a small projector on a pair of glasses. A digital eyepiece reader is connected to a portable computer which interfaces with a remote server giving the user access to a local (or perhaps even global) network. Currently, NEC plans to market Tele Scouter as a means for engineering and technician teams to remotely share data and expertise in vivid detail. For a team of 30, the Tele Scouter system would cost around $8 million. NEC will launch in 2010 and hopes to sell 1000 such systems in three years.

A trimmed down version of Tele Scouter may be adapted to serve as a universal translator. With rapid access to a remote server, the eyepiece has much more resources at its disposal than a smart phone and could serve to provide closed captioning during a conversation, as well as an augmented reality experience. There has been some buzz that NEC plans on following that route.

Yet even if Tele Scouter stays a high-level engineering network tool, the concept behind a visual universal translator could be adapted into another system quite easily. Today, all it takes is a web browser and Google to translate text from one language to another. In fact, judging from our servers, many of you actively use Google to read this blog in a non-English language. It’s only a matter of time before someone combines a head mounted display, a mobile internet connection, and speech to text software to create a do-it-yourself UT.

If you’re looking to start one of those DIY projects, you should do so quickly. As Shakr and Jibbigo find success in their Apps, we’re likely to see similar products spring up for many more languages. Eventually smart phones will become powerful enough to support translators that can identify and convert anything they hear. Many of the technologies we review on Singularity Hub are expected to take many years or even decades before they come to full fruition. Speech to speech hand held devices, however, are already here. Universal translators may appear much sooner than anyone believed possible. So get excited, Star Trek fans, the 23rd century is arriving early.

[photo credits: Shakr, NEC]
[video credits: Shakr, Jibbigo]

Need a ride? iPhone has an App for that.

Like peanut butter and chocolate, Batman and Robin, or babies and puppies, some things are just more powerful when combined. So it is with the iPhone and Zipcar. For those of you who don’t live in a major US metropolitan area, Zipcar is a car sharing company that allows users to locate, drive, and return a car without a traditional rental facility. iPhones, of course, are all powerful hand held devices that can do practically anything if you write a software application for them. Now, those Zipcar members that have an iPhone can use it to make a reservation, find their car, honk its horn, and even open its doors. It’s a cool upgrade for an already cool system. Check out USA Today’s video with Luke Schneider, CTO of Zipcar, after the break.

Car sharing is an eco-friendly, traffic cutting, and democratic approach to transportation, and Zipcar is the US leader in the field. For around $7 per hour, you can pick up a car in your neighborhood knowing that all your gas, parking, and insurance will be paid for. The inclusion of an iPhone App is taking this great idea and propelling it into the 21st century. You now have a mobile connection to a semi-public resource. Smart phones are becoming a type of ‘single key access’ to all your living necessities. Instead of a ring of keys, a wallet of credit cards, or a host of user IDs and passwords, you may one day only need to carry around your phone. It’s just another way in which smart phones, and the iPhone in particular, are demonstrating their power as digital tool kits for the 21st century.


Typical access to a Zipcar involves making a reservation by telephone or online portal. You get a time and a location to find your car and you’re off. Keys are left in the car, members use a plastic card for entry. The iPhone App helps you make the reservation, and adds the ability to honk the horn of the car to help you locate it, and to open the doors of the vehicle. For a member on the go, using the iPhone App will provide unparalleled rapid access.

The App is free to Zipcar members, of whom about 30% also use an iPhone. Integrating a smart phone application into the Zipcar fleet was relatively easy because the cars were connected wirelessly anyway. Each Zipcar contains the M200, a black box type device that controls membership card access to each vehicle. The M200 is what allows members to leave keys in the car. Of course, combining two technologies does allow for twice as many chances for things to go wrong. If you lose or damage your iPhone while renting a Zipcar, chances are that the M200 won’t let you start the vehicle. However, Zipcar says that a quick call is all it takes to fix that problem should it occur.

A new company formed from students of Singularity University are trying to out zip the Zipcar. Gettaround Car Sharing will not only work on the demand side, providing cars to those who need it, they will also help people make money by renting out their cars (~$1000/month). The project is in beta, and you can sign up on their website. What’s more, Gettaround already has its own iPhone App (no idea which company got there first) which is as good if not better than Zipcar’s. Check out the navigation arrows (1:17) and the lights-on feature (1:35) in the video below.

It’s a pretty standard science fiction conceit that in the future there will be tiny hand held devices that allow you to do just about anything. The iPhone is just a credit card and laser gun away from bringing that conceit to life. The diversity of applications, and the work of third party developers have transformed smart phones from gadget novelties to incredible digital toolkits. It will be interesting to see how future-minded technologies, like Zipcar’s car sharing, will shape and be shaped by the new smart phone paradigm. My guess is that we will see many more examples of technological dynamic duos in the days ahead.

[photo credit: Zipcar]

[video credit: USA Today]

Need to count calories? DailyBurn and iPhone have an App for that.

Your iPhone can play so many roles in your life: tour guide, military commander, camerman, restaurant reviewer, bank teller, DJ, and now… nutritionist. The newly renamed DailyBurn (formerly Gyminee) has developed a calorie counting iPhone App, called FoodScanner, that lets you scan barcodes on packaged food products to keep track of your daily intake. With over 200,000 registered items in their database, DailyBurn’s App also lets you view nutritional labels post-scan. While the retail price is $3, you can purchase the App now for just $1. Check out the demonstration video after the break.

DailyBurn’s App is just the latest example of how the physical world and the digital information world are merging. Just by pointing the iPhone camera at a scannable object, users can connect themselves to a large database that is directly relevant to their needs. In the case of health and nutrition, this database is DailyBurn’s extensive online community. We’ve seen other such connections like the Layars augmented reality functions on other smart phones. In the future every item in the real world will be enhanced by an extraordinary depth of data we will be able to connect to quickly and easily.


Scanning barcodes isn’t easy, and the curves and folds in packaging often makes it more difficult. DailyBurn has conquered the challenges of scanning by partnering with Occipital’s Red Laser APP, easily reading barcodes on reflective, curved, and uneven surfaces. The Red Laser App is worth a post just by itself, check out the video below.

I’m not sure how many iPhone users will pay $3 (or even $1) for a scanning calorie counter, it could be quite a few, but that’s only part of the promise of FoodScanner. iPhone users can opt to sync up their counting with DailyBurn’s more elaborate website. Much like Fat Secret, DailyBurn has created a network of like minded dieters looking for weight loss tips, shared workout experiences, and online support. Users can plot graphs of their progress, and share them with groups of friends. It’s a good model for business (social networking site meets locker room) and the App will probably help DailyBurn increase its share of the market. We’ll let you know how the competition between Fat Secret and DailyBurn (and other such sites) pans out.

Of course, DailyBurn’s App does have one big drawback. As they discussed in the video, you can’t scan an apple, though you can keep track of whole food consumption manually. It’s a little ironic that an App designed to help you eat healthier doesn’t like to track healthy foods. Maybe the best sign that you’re eating right is if you can’t use the App at all.

Still, DailyBurn and other iPhone App creators are onto a powerful trend. Linking portable devices to vast databases of information is going to change the way we view our world. Smart phones are starting to act more and more like external memory devices. You don’t have to remember or learn something when your phone can keep track of it for you. iPhone, what should I eat? iPhone, where’s the nearest bathroom? iPhone, why is everyone looking at me like I’m crazy?

[photo credit: DailyBurn]

[video credit: DailyBurn, Occipital]

Could the iPhone replace the bulky controls of the Raven Drone?

UAVs and other military drones are an important part of the modernization of the US Army. The Raven drone is a light-weight surveillance platform that can be launched by hand. Unfortunately, it’s operating controls are a heavy briefcase like enclosure that has to be lugged around. That sort of discontinuity irks HAL team leader Prof. Missy Cummings. Cummings was once a F/A-18 Hornet fighter pilot for the Navy and saw the UAVs as a great way to prove that drone control could be light-weight, simple, and elegant. Her students were the ones that pointed out they could use iPhones. (They would all have to receive iPhones as part of their research, of course).

The proof of concept tests were done with a commercially available four-rotor spy drone called a micro aerial vehicel (MAV). The drone only cost about $5000 and flew well inside the MIT robotics range. Watching the ease at which the operator controls the MAV it is hard to believe that HAL got the project from conception to demonstration in just six weeks!

It’s kind of crazy what a simple iPhone app can do. Besides manual control, the iPhone operator can cue in locations that he or she wants the drone to travel to called waypoints. There are single touch keys for launch and landing, and streaming video appears right inside the app. All this and the phone can still receive calls, texts, etc.

If the military decides to pick up HAL’s work, it will drastically lighten the load of soldiers in the field and allow them an easier way to manage their drones. Certainly the application could be modified so that a single iPhone could serve as a control for several different drones on collaborative missions.

Outside of the military, commercial drones could find more popularity with the ease of use promised by an iPhone application. In the final video, we see how the MAV can explore a new region with a high level of autonomy using a webcam and laser range finder. That’s good news for everyone from cave explorers, to security guards, to rescue workers. We’re bound to see more people using light weight and simple drones in important roles in the years to come. After all, iPhone now has an App for that.

USAA bank allows iPhone users to send in checks via photo.

While I haven’t used the USAA iPhone App, I do have some experience with their previous generation of online deposits. My old roommate used to borrow my scanner all the time to send checks to USAA. The scanning software (akin to a Java applet) was hard to use at first, but became much easier with practice. USAA has had that technology in place for 3 years. A fact I often consider while trudging uphill both ways in the snow to my local bank’s ATM.

USAA isn’t even in the top 20 of US financial institutions, but with only one branch in the country (San Antonio) it has to focus on banking anytime, anywhere. A good philosophy when a large part of your customer base is overseas military personnel. About one million of USAA’s 7.2 million clients access their accounts via their phone by text or online access. They require you to pass a credit check, and have some form of insurance with the company. About 60% of their total clientele qualify. While less than 3% of the million mobile users may own an iPhone, USAA is confident that just having the option will draw customers and raise brand awareness.

Larger banks have been slower to allow scanned or photographed check deposits, but most have iPhone Apps. Bank of America, whose smart ATMs seem to be everywhere these days, say that 43% of their 3 million mobile banking customers use iPhones. That’s more than 1.2 million people who could save themselves a trip to the bank if BOA took a page from USAA’s book and started accepting scanned checks.

In the meantime, USAA continues to blaze new trails in the world of iPhone Apps and may start to enjoy the benefits. Certainly the overly friendly gentleman in the video seem to think so. In general, with the range of abilities available to smart phone users, they are on the path to becoming universal tools of trade and communication. When you are able to pay for something by bumping your iPhone against a cashier’s terminal we’ll finally hit pay dirt. For the moment, though, there’s not an App for that.