Individually none of us seem that great at predicting the future, but collectively we could have some valuable insights. Recorded Future harnesses that potential by searching through public commentary and speculations on the net and offering them through a search-engine type interface. It’s sort of Google meets Gallup Poll meets Nostradamus. The Boston based startup scrapes real time data from the web and uses key words about dates to place that information in a time line. By analyzing sentiment, Recorded Future can classify data as either supporting or opposing that an event will happen at the time referenced. The result is a detailed, and some hope accurate, portrayal of the past, present, and future. Among Recorded Future’s small number of clients are people who analyze stock markets, media trends, and terrorism. According to MIT’s Technology Review, the company has received funding from Google and the CIA. Check out the introductory and sample videos for Recorded Future below. With a tool like this, the field of futurism could get a lot more crowded.
Whereas search engines typically have one entry box, Recorded Future has three. You explain What you are looking to happen, Who or Where you are looking for it to effect, and When you think it matters. For instance, you could search for major mergers in US banks in the next ten years, and Recorded Future’s results will give you a timeline for what has been predicted in that field. You can click on any listing to see the original source, and view the connectivity between subjects as well. The graphic interface for the results lets you easily vary your search parameters to find the information you need. The demonstration below gives a crash course in using the system:
To give you a better idea of how the actual search process occurs, I’m including two sample videos below. The first walks you through researching a person (Israeli Prime Minister Benjamin Netanyahu), while the second explores a topic (pharmaceuticals). In both cases, you’ll notice that Recorded Future isn’t giving you a single definitive prediction, it’s just collecting a wide realm of relevant predictions and helping you explore them quickly. Notice how the sentiments of sources (explored in the second video around 1:34) are colored as positive or negative to help you make sense of the jumbled mass of opinions that your Recorded Future search will produce.
How is Recorded Future weighting its results? Most search engines used the connectivity of web pages to determine their importance. The more people who link to your work the more important you’re judged to be and the higher up you appear in search results (generally speaking). Recorded Future takes things to another level. It explores the content of news feeds, government reports, tweets, press releases, and speech transcripts in order to extract pertinent dates and future looking statements, as well as judging the sentiment of the content itself. It can rank sources according to how accurate their outlooks tend to be. In other words, Recorded Future doesn’t simply let you know where to find predictions, it lets you know which predictions you should pay attention to.
As you can imagine, the credibility of each source waxes and wanes over time. In general, the vision of the future evolves quickly, and Recorded Future has to evolve right along with it. Never is this clearer than when you take a behind the scenes look at how the system tracks relationships between topics. In the video below, Recorded Future shows the network analysis surrounding an important public figure in the media (US President Barack Obama) through the year 2009. The spiderweb of events, places, and people surrounding Obama shows the complex interplay that Recorded Future is tracking in order to understand the predictive success of its sources. This graphic display of information is awesome to watch.
Along with hands-on exploration of data, Recorded Future also lets you passively keep track of important topics through email alerts. Much like Google Alerts, you can receive daily or weekly updates from Recorded Future when new predictions or information is released. I’ve already signed up for half a dozen and they’re pointing me towards some interesting stories I may write later. It will be interesting to see if these “futures”, as the alerts are called, prove to be more valuable than standard updates from Google, Twitter, etc. If you’d like to receive your own custom selected “futures” you can sign up for them easily on the Recorded Future website. Go ahead and subscribe to a bunch, they’re totally free.
Speaking of which, Recorded Future looks to be rather expensive. While the email alerts won’t cost you a penny, the use of the predictive search engine starts around $150 per month…per user. If you want to develop your own Recorded Future application through one of their APIs, you’re looking to shell out $2500. That’s relatively cheap for some SDKs, but considerably more expensive than access to other APIs (which often are free to develop). I’ve failed to mention Recorded Future’s ‘pay to use’ status up to now because I find it utterly disappointing. Yes they have a unique approach to exploring and weighting digital data, but I’m not paying $150 a month to access it. Can you imagine if Google charged you for their search results? Hopefully Recorded Future’s business model will change soon.
In the short term, however, Recorded Future will probably do just fine. What they offer is too powerful for many to ignore. Through sentiment and network analysis of predictive content, the company provides an added dimension to search. I’m not just talking ‘time’ here. Recorded Future does more than simply show you what’s on the web, it shows you how that content relates, and gives you a hint at what people are really thinking. As we’ve seen with analysis of Twitter, understanding the mentality of the online crowd can help you make startlingly accurate predictions in some cases. In fact, Recorded Future claims it has a good record of predicting the volume of activity in financial markets. This is only going to become more important as we increase the amount of sentiment ladened data we collect. There are people working on ways to collect data by directly analyzing overheard comments made in public. Imagine how that added information could enhance or reshape the results from an engine like Recorded Future. We are moving from a society where individual intelligence is dominant to one where collective knowledge acts as a great equalizer. Recorded Future is part of that trend. Whatever power we have as a civilization to predict the future is channeled through a single lens. There’s no guarantee that will give us a clearer picture of what lies ahead, but if it’s a choice between believing an individual futurist or the collective conscious of the world, I’m going with the world every time.
[sources: Technology Review, Recorded Future]
