For the last few months, the political pundit class has been at war with NYT/FiveThirtyEight blogger Nate Silver. Joe Scarborough of MSNBC called him a “joke,” while an op-ed in the LA Times accused him of running a “numbers racket.” The Examiner dismissed him as a “thin and effeminate man with a soft-sounding voice.” Even the legendary David Brooks claimed that his work was “getting into silly land.”
His crime? Standing by the projections of his proprietary statistical model that have predicted the results of the last two elections with stunning accuracy.
A former baseball statistician, Silver has made his name by aggregating state level polling data, and predicting the likelihood of victory by adjusting for different factors that have historically influenced electoral performance. He consistently rejected the conventional wisdom that the race was tied, and for that, he was cast as a left-wing nut peddling bad science.
But last night, Silver triumphed: every one of his state-level presidential predictions proved true.
And with this victory, he may have changed the way elections are covered for good. By disregarding the industry wisdom that gut feelings, day-to-day poll movements, and talking head commentary are what matter, he has decimated the value of these superficial judgments.
This year, according to the New Republic, “somewhere between 10 and 20 percent of politics visits to the New York Times website included a stop at FiveThirtyEight;” but last week, “that figure was 71 percent” – on the sixth most trafficked site on the web. Over the last 24 hours, there has been an 800% increase in sales for Silver’s latest book. It won’t be long before every major news outlet has their own Silver-clone spewing out statistics and probabilities.
Beyond just personal vindication, Silver has proven to the public the power of Big Data in transforming our electoral process. We already rely on statistical models to do everything from flying our airplanes to predicting the weather. This serves as yet another example of computers showing their ability to be better at handling the unknown than loud-talking experts.
By winning “the nerdiest election in the history of the American Republic,” Barack Obama has cemented the role of Big Data in every aspect of the campaigning process. His ultimate success came from the work of historic get-out-the-vote efforts dominated by targeted messaging and digital behavioral tracking. Future campaigns will build off this success and spread these tactics to every part of the electoral process, continually accumulating whatever new types of data are available in the future.
In an interview with BuzzFeed, Silver admitted that he had a lot riding on the outcome of the election – he recognized he would “get too much credit if the prediction is right and too much blame if it is wrong.”
A statistical model can’t predict a leaked fundraiser video, a lousy debate performance, or an enormous storm – but it can provide valuable insights into how things actually look, without the noise. You can’t “moneyball” a winning team of candidates, but you can see where things are likely to go.
The field of “political data science” is now more than just a concept – it’s a proven, election-winning approach that will continue to revolutionize the way campaigns are run for decades to come.
