Forecasting the Future: Ray Kurzweil, Nate Silver, and—the Market?

Image credit: Sean McGrath

If you’re a futurist, chances are you have a favorite oracle or uncanny prediction. Maybe it was John Paulson’s bet against the housing market in 2007-08, or Nate Silver’s spot on election call in 2012, or Ray Kurzweil—just in general.

But if you haven’t already, you should consider adding another prophetic voice to the list: the market. In the last decade, creators of “prediction markets” have applied market principles to answer very specific questions about the future.

Who will win the presidential election in 2012? Will the US debt limit be raised by the end of the year? Will the US government announce the existence of extraterrestrial life by December 31, 2015? (As of this writing, 20% chance that last one happens. Yikes!)

There are a number of different prediction markets—Betfair, Iowa Electronic Markets, and InTrade. InTrade is perhaps the best known example in the US, and has made headlines with a recent Commodity Futures Trading Commission (CFTC) ban. (InTrade says they’re working hard to bring their markets back to the US legally. We’ll reserve whether the ban should have happened in the first place for another post.)

So, how do prediction markets work? Participants buy and sell shares at the prevailing price, say $6 on an Obama victory in the presidential election. The outcome is binary—settling at either $10 if the event happens or $0 if it doesn’t.

If you bought at $6 and Obama wins, you pocket $4 less transaction fees. If Obama loses you get $0, foregoing your $6 plus transaction fees. (Go here for a more detailed explanation, including short selling and pre-settlement trading.)

Share price equates to the market’s best guess at the probability of an outcome—$6 is said to be a 60% chance, $3 is a 30% chance, and so forth. If an outcome is a long shot, demand falls and so do share prices. For very likely events the opposite is true.

Prediction markets in action are potentially very accurate. They often beat polls and pundits alike. InTrade, for example, has correctly predicted every presidential election since 2004, the first such contest it covered.

In 2004, the market nailed every state and the total electoral count. In 2008, it missed the electoral count by one. And in 2012, the market predicted an Obama victory, missing only Florida. It got the Senate and House right too.

Presidential election prediction market. Source: InTrade

Stephen Dubner of Freakonomics explains it like this, “If you ask a lot of people a question about politics or sports or Hollywood movies, and those people are motivated to answer it correctly [by the prospect of profit or loss], their collective judgment turns out to be fairly accurate.”

However, prediction markets, like other markets, aren’t perfect. They can be traded too thinly to efficiently find an accurate price. They may be subject to manipulation. And most importantly, prediction markets are dependent on widely available public information. They can miss big if information simply isn’t there.

InTrade, for example, showed upwards of 75% odds the Supreme Court would overturn the individual insurance mandate in the recent health law. It was wrong. The market missed this call in large part because the necessary information was contained within a tiny private circle of nine justices. (And of course, we should remember a 75% likelihood leaves a 25% chance it’ll go the other way.)

That’s not to say markets in general, and prediction markets specifically, aren’t the powerful predictive force many believe them to be—just that market conditions (liquidity, size, rules, etc.) influence the accuracy and range of market predictions. Either way, those of us wondering what’s just around the bend would be remiss to ignore the wisdom of crowds (and naive to forget their occasional madness).

Jason is managing editor of Singularity Hub. He did research and wrote about finance and economics before moving on to science and technology. He is curious about pretty much everything, and sad he'll only ever know a tiny fraction of it all.