The Google Translate mobile app is a hint at the awesomeness that is to come.

The Universal Translator is one step closer to reality. One awesome and frustrating step. The Google Translate mobile app, now available on iOS as well as Android platforms is the sort of technology we’ve all been waiting for since we first saw Star Trek. Simply by speaking into your phone you can have your words translated and then spoken aloud in a different language to someone else. The person you are talking to can do the same in reverse. And what do you have to pay for this proto-universal translator? That’s the best part, it’s completely free. Amazing. Google Translate currently works in 57 languages of which 15 are available for speech to text and 23 languages available for text to speech. Go and download the app right now. Seriously, go do it now. Just be prepared: as wonderfully freeing as this technology could be, it is very far from perfect. This app hints at, but does not completely deliver on, the universal translator we will want in the future.

The following videos demonstrate the use of Google Translate for Android and iPhone respectively. You can see the original presentation of the ‘conversation mode’ feature by Google in our previous discussion.

For the past 24 hours I’ve been doing little more than playing with Google Translate on my iPhone. I may have kept my wife up at night. I may have apologized in translated Hungarian, Japanese, and Haitian Creole. I may have received glaring looks of death. It’s hard to remember.

What I can tell you from all this testing is that the translations on the app are amazing. Simply wonderful. You can give the program fairly high level words (‘adumbrate’, ‘evanescent’, etc, etc) or ambiguous phrases (“This thing is not the thing I ordered from the other thing”) and get the right translations. As Google has proven time and time again in its online tools, the search engine’s statistical approach to the task is both powerful and flexible.

Yet it’s the voice to voice capability that really drives our desire for a universal translator and it is here that things begin to break down for the mobile app. The program takes your speech, converts it to text, translates, and then converts the translated text to speech. The latter three parts work well. The first is a frustrating quagmire of pronunciation fueled guilt. No matter how much I articulate, no matter how carefully I space my words, it’s rare for the app to capture what I say perfectly. Working with Google Translate can feel like talking to your aged aunt who is deaf in one ear. Having to repeat the same simple phrase four times to be understood can be damnably frustrating.

Google makes up for this with several useful features. You can edit the text of your speech in case the program didn’t understand what you wanted to say. Phrases can be starred so that they are saved and easily accessible to be used later. The text translation can be displayed in a full screen mode to make it easier for others to read. As it is, I could rely on the Google Translate app while traveling, but I would still need some of my own language skills in order to get by. The app is great, but it’s not a universal translator.

Perhaps more patience is needed. As we’ve seen with Watson and other forays into real language comprehension, understanding human intent is still a difficult task for machines. Speech to text will get better as we move ahead, and as it does Google will undoubtedly improve upon its mobile app. At the same time they will likely expand their list of available languages and features. (Hopefully we’ll get ‘conversation mode’ on iOS soon.) The Google Translate app has irked me these past hours with its failings, but overall I am still absolutely delighted by its presence. A good, not perfect, proto-universal translator is available for free on your mobile devices. 2011 is shaping up to be an extraordinary time in history. Who knows how quickly this technology will take off in the years ahead. Google Translate gives me hope that one day the language barrier will no longer divide humanity. Beyond that, who knows what might happen…

[image credit: Google]
[video credits: AppVee, ABCMSAJ, Google Channel UK]
[sources: Google Blog]

Will 2011 be the year of the universal translator?

Hundreds of companies are working to hammer down the language barrier, but Google’s about to bring in a wrecking ball. For the past year, the giant amongst search engines has been touting a new feature they will be bringing to their smart phones – it’s called ‘Conversation Mode’. Combining the high speed text translations of Google Translate and the text to speech/speech to text skills of Google Voice Actions, Conversation Mode allows users to speak in one language and have their phone speak in another. Google hasn’t announced any firm launch dates (it’s been ‘months away’ for months now) but they demonstrated the technology during IFA 2010 in September to great effect. You can watch the demo in the video clip below. We’ve seen smart phone based applications like this before, but not with the power or versatility of Google Translate behind them. For those who are keeping track, GT added its 57th language this year. Conversation Mode still has its bugs to work out, but it looks like a nearly universal translator could be coming to your phone in 2011.


This video clip is from Eric Schmidt’s hour-long keynote address at IFA 2010 in September. You can watch the entire presentation here.

We’ve been talking about universal translators a lot recently because there have been so many wonderful demonstrations of the technology in the past 18 months. Smart phone applications capable of translating a single language to English have been available for a while, with great examples for Spanish and Arabic appearing in late 2009. (Prototype software, of course, has been around for much longer.) In those same 18 months we’ve also seen advancements in text to speech/speech to text programs, and the number of applications using that capability continues to increase. I think it’s clear that we’re approaching a perfect storm moment in the development of universal translators. We have the necessary pieces to start cobbling a UT together, and in my mind that’s what Conversation Mode really is. Glue Voice Actions to Google Translate and you get a system that can handle 50+ languages with lag times of just a few seconds. And there’s every reason to believe that Conversation Mode will be free. How incredible is that?

Of course, not everything about a Google created voice translator will be perfect. Google Translate is definitely a cloud-based service, meaning that Conversation Mode will likely require connectivity to work. Not very useful if you’re in the middle of nowhere and can’t get a signal. Also, as you could clearly see in the video above, the Conversation Mode demo took three attempts to properly translate “color”. The first generation of this service is likely to require some major amounts of patience.

Yet one of the most promising things about Conversation Mode is that it will get better with time. Possibly much better. Google Translate learns how to convert between languages by examining millions of documents and developing rules for translation that constantly evolve. Because it is cloud-based, you can access improvements in Google Translate almost as soon as they are made. Conversation Mode (and its successors) won’t just be a great semi-universal translator, it will be a tool that changes as the languages of the world change. A UT not just for today, but for the future in perpetuity. In five years I expect I will be able to travel around the world and never need a translator besides my phone. In ten I think the language barrier will be so crumbled that we will take it for granted that we are crossing over it many times each day. It’s going to be amazing.

[video credits: Google]
[sources: Google at IFA (video), The Times]

"No hablo espanol."

The universal translator is a sci-fi staple: Star Trek made it infamous. Star Wars had C3PO. Hitchhiker’s Guide had the babel fish. Stargate and Dr. Who both had some variation of a voice-to-voice translating device. In some ways, the future is already here: Google Translate can turn around a workable text translation almost instantly (automatically in Chrome), and it’s letting the multilingual web talk to itself. Word Lens will even translate text you see in real time as augmented reality on your smartphone. Text translation is all well and good, but when will the holy grail arrive? When will voice-to-voice translation become a reality? When can you finally toss your Rosetta Stone software?

Actually, it’s already here – it’s just not as smooth as you might have hoped (yet). All the basic pieces of software necessary to a universal translator have already arrived: speech recognition (voice-to-text), language translation (text-to-text), and speech synthesis (text-to-voice). In fact, it’s already being employed in a number of sectors using current technology. Granted, the process is pretty clunky, but it’s here and it works.

The Army has been using a system developed by DARPA under the Spoken Language Communication and Translation System for Tactical Use (TRANSTAC) to help soldier speak in foreign countries. One such system, IraqComm, was developed in conjunction with SRI International and translates back and forth between English and colloquial Iraqi Arabic. Check out the system in action:

I also found this video of Ray Kurzweil demoing a basic version of a translator a few years ago:

From these videos alone, you can get a good idea of what needs to improve. First, the translation process isn’t nearly fast enough to hold a fluid, natural conversation. This particular hurdle shouldn’t be a difficult one to overcome; faster computers will be able to run the recognition, translation, and synthesis algorithms much more fluidly. However, there is an upper limit to the speed that these systems could acquire: the recognition software needs to hear most of the sentence before it can pass a text copy on to the translator. Languages don’t correspond to one another word-for-word, so a real-time translator isn’t really possible. At best, we should expect news-correspondent delays.

Second, the translation is a bit rough, and doesn’t always catch the finer points of what was said. Again, this is technology that has been improving over time (and nowadays, text translators can almost always capture the general idea in their translation). The newer era of translators – Google Translate included – are a significant leap from the generations that came before them. It wouldn’t surprise me to see the newer algorithms picking up slang, idioms, etc. as they refine their algorithms.

Finally, the automated voice sounds mechanical and awkward. I find this to be true of all the speech software I’ve encountered, and it tends to bother me (however, I have friends who listen to PDFs this way and don’t mind it). Certainly this kind of software is improving as well, but I have yet to hear speech software that sounded completely natural. This might actually be the last hurdle to be overcome. It reminds me of how you can’t lock eyes with someone over a webcam because the cameras aren’t behind the monitor: we’re always looking slightly to the side. The ideal speech translator would reproduce your own voice, as if you spoke that language, but needless to say this is a long way off. There might also be an uncanny valley along the way.

These three pieces are now being integrated more seamlessly, and the hardware is already here to support the improving software. Imagine using your smartphone and a Bluetooth to translate in real time in a foreign country. I doubt it’ll be absolutely perfect in the foreseeable future, but there are already some early versions coming. Earlier this year Google told The Times it was working on such a package, and hopes to have something that will “work reasonably in a few years time.”

We can add one more job to the robot-replacement endangered list: translators.

Automatic language translators are taking your tweets to a global audience.

The universal translator was once a convenient fiction from Star Trek that let aliens break through language barriers, but now it’s an awe inspiring reality. Automatic language converters are on the internet, in your iPhone, and they’re ready to take on Twitter.  Companies like Twanslate, Twinslator, and Tweet Translate will take your tweet and change it into any one of a myriad of languages you choose. Lost in a foreign land? Now you can let your tweets speak for you. Twitter itself recently began translating its pages into non-English languages. Soon visitors to Twitter will be able to read the main text of the site in any of the world’s languages from Arabic to Urdu. It’s probably only a matter of time before Twitter adopts a universal translator into the site itself. Think about that! You tweet in one language and Twitter would automatically broadcast it in another. Just click on a twitter feed and you can read it, no matter where it’s from. Universal translators are going to take the tweet from a fad to the forerunner of global society.

Automatic language translation lets artificial intelligence connect you to anyone no matter the language. The Apps on your smart phone allow you to translate between languages simply by speaking into the device and pressing a button. That’s just a mind blowing technology. Unfortunately, it’s not available in all languages. But guess what is? Text translators. They’re fast, fairly accurate, and usually free. Those are the key ingredients that can bring a technology to explode onto the global scene.

As text based tweet translators become firmly entrenched, they’ll let your tweet reach a wider audience, a global one that doesn’t have to learn new languages to communicate. AI will do it for us. As Twanslate and other third-party-developed programs become more popular, they’ll also struggle to get faster and better at what they do. How long before tweet translators are smart enough to automatically convert any twitter feed you want to follow without you having to ask? At that point, Twitter users won’t be divided by language barriers, they’ll be united by common interests.

The technology for a global tweet is here and different companies give you different ways to make it happen. You can use Google Language Tools to translate text and then cut and paste it into Twitter. Twanslate makes it a little simpler: just direct message them on Twitter and they’ll direct message you back with the text translated into a different language of your choosing. Twinslator and Tweet Translate are the most direct: you just go to their website, input your text and they’ll tweet it for you.  Of course, you’ll also have to give them your User ID and password. All of these, however, are  precursors to Twitter integrating translation software directly into its site. Give it a few years, and tweet translations will happen automatically, without you having to do anything at all.

When Twitter includes translation software into its site the 140 character message will become the world standard.

That’s becoming true wherever you look. Universal translation software has reached a point were it can be adapted into almost any other technology. It still has its kinks. Nuance and subtlety will confuse translators most of the time and programs that access a central server can take a few seconds to process the conversion. Yet that’s why universal translators fit so well with twitter. There’s only ever 140 characters to deal with, and users are intentionally making their messages as clear and precise as possible. Twitter and universal translators were meant for each other. It’s fate, baby.

Tweet translators are likely to impact other Twitter phenomenon. Massively tweeted messages, the so-called tweetbombs, are going to spread wider and more easily when they can be automatically translated into many languages. Twitter accounts with huge numbers of followers (>1 million) may not effect the tweet-sphere all on their own, but think of what power they might have when they can reach users all over the world in their native tongues.

We’re already under a constant bombardment of online information. News feeds, Facebook updates, Google Alerts, emails, and text messages…there’s too much in the pipeline for anyone to really absorb and understand it all. It looks like it’s going to get worse. As universal translation software matures we will have access to people of diverse cultures and languages as easily as we have access to our neighbors. Information will flow freely in many different directions. Looking at Singularity Hub’s stats, we already enjoy readers in different countries, speaking dozens of languages, all without much work on our part. Universal translators allow common interest and curiosity to shape our information flows even as we struggle to find direction under their torrents.

With so much to process, we’re going to want to learn more in less time – that’s the essence of Twitter! The tweet, the small encapsulated thought bubble, could carry universal translation into the mainstream. Automatic language conversion software may give ride to the flood of interest that breaks down the language barrier once and for all. Global culture based on just 140 characters of text? Awesome and terrifying at the same time.

[image credit: Twinslator, Twitter]

Will the iPhone be the Universal Translator of the future?

When watching Star Trek, I was always amazed that the aliens all spoke English. Writers explained this happy coincidence by the presence of a universal translator (UT), a portable device which could take any language and translate it into any other language as needed. Our nonfictional modern world has more than 20,000 different languages (if you include dialects) in its history. With the rise of globalization we desperately need an universal translator to help us speak to one another. Luckily, there are several different companies that are taking the first steps to creating an UT. Sahkr and Jibbigo have developed iPhone Apps which transform the smart phone into a handheld speech to speech translator. Check out some of their demo videos after the break.

Computer aided speech to speech translation isn’t easy. You need speech recognition, language analysis, machine translation, language generation, and speech synthesis. We’ve had limited handheld translators for some time. Devices like Voxtec‘s Phraselator allow you to select a range of sentences and have them translated into many different languages. We’ve also had more complex non-portable systems (like IBM’s MASTOR) which can do nearly full translation. But the products created by Sahkr and Jibbigo represent a new paradigm in speech to speech: commonly used hardware devices adapted solely through software. By integrating into an existing hardware platform, these next generation programs are taking us one step closer to having a universal translator that anyone can carry in their hands.

Sahkr is one of the premier English/Arabic translation companies in the field. The translation software, Arabic Language Buddy, went live in September and is now available for download on either the iPhone or BlackBerry. It is limited to conversions between English and Modern Standard Arabic (MSA), but it works amazingly well:

Sahkr takes great pride in the realistic synthetic voices used for their translation software. I don’t speak MSA but the flow and tone of the translation is much better than I would have expected. The company press release also claims that their translations are ranked #1 by the US government. Users can even adjust the text of their translation (via keypad) to better communicate their words precisely before it is spoken. As you can imagine, this high quality comes at a high price. The software is for lease only (which isn’t that unusual for premium smart phone Apps) and the first month is $10. Subsequent months, however, can cost you either $50 or $150, depending on whether or not you qualify as an academic user. That’s no small change for the average user, but for the international business traveler it likely falls within budget.

If you’re looking for a cheaper option, and you favor the iPhone over the BlackBerry, look no further than Jibbigo. Developed by Alex Waibel at Carnegie Mellon University, and released in October, Jibbigo does most everything that Shakr does, just in Spanish and only on the iPhone. You can edit via keypad, shake the iPhone to restart, and there’s even a back translation check. All that, and you make just one payment of $25.

The difference in price between the two applications may reflect a relative scarcity of Arabic to English translators, but I’m guessing it also has to do with Shakr’s higher status as a translation service. Jibbigo is a fairly new company with hopes of expanding from Spanish and English into other major languages. Recently however, they had to deal with an audio glitch in their first released version (it’s fixed with an update).

The tele-scouter projects texts and images directly into your peripheral vision and could easily be used as a speech to text translator.

Of course, a portable UT doesn’t have to fit in your hand. NEC’s Tele Scouter is a head mounted optical display which projects words and images into your peripheral vision. Engineers claim that the use of the peripheral allows you to wear Tele Scouter for hours without headaches or other discomfort. It is very light weight, and essentially looks like a small projector on a pair of glasses. A digital eyepiece reader is connected to a portable computer which interfaces with a remote server giving the user access to a local (or perhaps even global) network. Currently, NEC plans to market Tele Scouter as a means for engineering and technician teams to remotely share data and expertise in vivid detail. For a team of 30, the Tele Scouter system would cost around $8 million. NEC will launch in 2010 and hopes to sell 1000 such systems in three years.

A trimmed down version of Tele Scouter may be adapted to serve as a universal translator. With rapid access to a remote server, the eyepiece has much more resources at its disposal than a smart phone and could serve to provide closed captioning during a conversation, as well as an augmented reality experience. There has been some buzz that NEC plans on following that route.

Yet even if Tele Scouter stays a high-level engineering network tool, the concept behind a visual universal translator could be adapted into another system quite easily. Today, all it takes is a web browser and Google to translate text from one language to another. In fact, judging from our servers, many of you actively use Google to read this blog in a non-English language. It’s only a matter of time before someone combines a head mounted display, a mobile internet connection, and speech to text software to create a do-it-yourself UT.

If you’re looking to start one of those DIY projects, you should do so quickly. As Shakr and Jibbigo find success in their Apps, we’re likely to see similar products spring up for many more languages. Eventually smart phones will become powerful enough to support translators that can identify and convert anything they hear. Many of the technologies we review on Singularity Hub are expected to take many years or even decades before they come to full fruition. Speech to speech hand held devices, however, are already here. Universal translators may appear much sooner than anyone believed possible. So get excited, Star Trek fans, the 23rd century is arriving early.

[photo credits: Shakr, NEC]
[video credits: Shakr, Jibbigo]