Kurzweil Defends His Predictions Again: Was He 86% Correct?

13,603 9 Loading

"I figure lots of predictions is best. People will forget the ones I get wrong and marvel over the rest." --Alan Cox

How My Predictions Are Faring

Kurzweil gives a 147 page review of his predictions. Should you read it?

How would you grade yourself if you had a chance to write your own report card? Ray Kurzweil is giving himself a high B. With his recent essay "How My Predictions Are Faring" the noted futurist reviews forecasts he made more than a decade ago for our current times. His predicted future is now the present, so it's time to see how he did. The report looks at 147 predictions that Kurzweil made in his 1999 book The Age of Spiritual Machines, and goes on to briefly explore comments in The Age of Intelligent Machines (1990) as well as his more recent The Singularity is Near (2005). Of the 147 predictions Kurzweil maintains that 115 were 'entirely correct', 12 'essentially correct', and 17 'partially correct, with just 3 being outright wrong. Counting only the entirely and essentially correct, Kurzweil claims an accuracy of 86%. The report, dated October 2010 and now widely available on the web, has sparked heated discussions on various tech forums. I've read "How My Predictions Are Faring" and have drawn my own conclusions about Kurzweil's prowess: If you're pouring over each of the 147 statements and debating their accuracy then you're probably wasting your time. When it comes to evaluating predictions, I'm not sure we're doing it right.

Before I Cop Out, A Review

For years, Ray Kurzweil has been one of the foremost futurists in the public arena. He predicted the fall of the Soviet Union, and the rise of the internet. He's also predicted the rise of artificial intelligence and the eventual union of man and machine to occur in the 21st century. While he didn't coin the term 'Singularity', he has brought the concept of an exponential growth in technology to the masses through books, documentaries, and countless speeches around the world. In many ways, Kurzweil has become the face of the Technological Singularity, and that has made him a target to techno-optimists and Luddites alike.

Perhaps rightfully so - many world leaders in industry and government listen to his opinions. For instance, he's an advisor to the U.S. Army for developing a rapid response to biological threats. If Kurzweil's beliefs about the future affect public policy it's important we understand why he's given such clout, and that we challenge him about his errors.

In his book The Age of Spiritual Machines (written in the late 90s and published in 1999), Kurzweil made definitive statements about the state of technology in the year 2009. Actually, he was talking about that period in time in general, so we can add some error bars of plus or minus a few years around 2009. Now that we're entering 2011, though, we can get a good idea about whether or not Kurzweil's 2009 predictions were correct.

Hence the "How My Predictions are Faring" report. It's a 147 page review of his 147 predictions (can't be coincidence) from ASM and it goes into each statement in full detail. Kurzweil steps you through numbered lists of these predictions as divided by topic and gives you his reasoning for why each was correct, essentially correct, partially correct, or wrong. For someone looking to review Kurzweil's precognition the report is easy to access.

With access has come the opportunity for controversy. Critics are able to go through and evaluate not only the original predictions, but Kurzweil's explanations of those predictions. We now have 147 more pages of text to debate on forums and in comments. You could, if you like, do exactly that yourself. The essay is freely available on Kurzweil's site, and mirrored on Scribd. You could spend hours looking through that document and analyze every sentence for accuracy and then share your thoughts with the internet-capable world.

But please don't waste your time.

And Now the Cop Out

Look, I encourage people to read things for themselves and I encourage healthy debate. These are essential parts of the scientific process, crucial skills for society as a whole, and the only way we can shape the internet into a tool for enlightenment. Yet I'm deeply worried when I see the techno-optimistic community putting efforts into these kinds of debates, because I believe that they are largely unnecessary. Why?

Judging the accuracy of predictions like these is simply too subjective. Predictions are drastically influenced by interpretation, as you can see in Kurzweil's report. When he reviews each statement, Kurzweil is selecting those parts he finds relevant and commenting on them. You and I might choose to make judgments based on completely different sections. Take #13 in Education: "Many children learn to read on their own using their personal computers before entering grade school." Kurzweil deems this as correct by focusing on the developmental aid young children experience after using computers habitually. He also points to a report that 9% of children who enter kindergarten already know how to read. A critic, on the other hand, might have called this a fail because developmental skills are not the same thing as actually reading, and increases in pre-K reading skills aren't necessarily due to computer usage. By focusing on different parts of a prediction you can draw drastically different conclusions.

Well, duh, Aaron.

Ok, so maybe it's obvious that judging predictions is going to be subjective. But the pervasiveness of this kind of selection bias isn't always clear. After all, in the paragraph above you might have evaluated prediction #13 and drawn your own conclusions about whether or not Kurzweil was right. But did you consider whether or not #13 in Education was the best example for me to use in that paragraph? Why not #19 in Communication or #5 in Politics and Science?

Everyone who comments on Kurzweil's work is selecting both the predictions they want to focus on as well as the parts of that prediction they want to focus on. That doubles the bias and it means that it will be very hard for anyone to come to an agreement about Kurzweil's predictive prowess as a whole. This is a problem I came across when reviewing Michael Anissimov's critique of Kurweil a year ago. When there is this much picking and choosing to be done I think that the debate simply devolves into arguments over what to talk about. Or what we think someone else was talking about. Critics and counter-critics will simply talk around each other rather than coming to a consensus.

The way I see it, there are only two ways out of this mess.

First, we can set objective standards for predictions. We can require futurists to make statements that are as clear and closed to subjective interpretation as possible: "By January 2015 (plus or minus 12 months) the gross domestic product of the US, as measured by the Bureau of Economic Analysis, will surpass 30 trillion dollars (not adjusted for inflation)." We can then take those predictions and assign them a score that's either binary (did it come true or not) or graded (how many years difference was there between the predicted and actual date for an event). Collect all the predictions for a futurist in one place and then give them a general score for their efforts.

To some degree the Long Bets website is already trying to do this. They even added the element of making wagers. If we expanded this concept to every single prediction someone like Kurzweil makes, then maybe we'd have an objective score for their skills.

The second idea would be to focus on the broad picture a futurist paints. Instead of evaluating each of the 147 predictions we could try to summarize what Kurzweil is saying in general. This will be highly subjective, but it might help us come to an agreement we could act upon. Is Kurzweil basically telling us that "information technology will continue to accelerate indefinitely." If we accept such a premise we can start trying to find ways to take advantage of that general trend for the betterment of everyone. I think Singularity University operates on such an assumption.

The first solution would require us to throw out many predictions as unclear, the second solution would have us abandon objectivity to a great extent. So yeah...neither really seems like a great choice.

Yet I think the current trend of evaluating subjective statements objectively is an even bigger mistake. As objective as Kurzweil may have been trying to make his predictions in The Age of Spiritual Machines, I think most simply aren't precisely worded enough to evaluate as such. I've spent hours looking over this document, reading the original statements and Kurzweil's evaluations, and I find myself being able to argue for and against each prediction. It's all about selection. I have pages of detailed notes on this document, and I'm looking at them and asking myself, what use are these?

We don't have a baseline comparison for evaluating the statements of futurists. If Kurzweil makes a prediction that anyone could have guessed, it doesn't speak much to his credit.

#20 in Education: "Learning is becoming a significant portion of most jobs."
#6 in Business : "China has also emerged as a powerful economic player."
#19 in Arts: "Users also experience fantasy environments with no counterpart in the physical world."

These all should have been expected, I hope.

#6 in War: "Military conflicts between nations are rare, and most conflicts are between nations and smaller bands of terrorists," that might have been less obvious to the public ten years ago but experts would have taken it as a given.

And #15 in Business: "There is a strong trend towards the geographic separation of workgroups. People are successfully working together despite living and working in different places." Even if we didn't understand the depth of how powerfully the internet would affect work, we probably could have made a similar statement in 1999, right?

(But there I go being selective. I've chosen five statements to prove a point, but you could probably find another five that demonstrate how insightful Kurzweil was.)

If we don't have a baseline measurement about our ability to predict the future then we can't establish a reasonable expectation about how accurate prominent futurists will need to be in order to trust them more than average man on the street.

Even if we do establish that some experts have been better at predicting the future than others, how much do we invest in the predictions they make next? If Kurzweil has been 86% accurate about 2009, can we say he'll be 86% accurate about 2019, 2029, or 2039 ? Does it just mean that he's accurate 10 years out? By the time we could say for sure the world would have aged decades.

I'm indebted to Kurzweil for writing books like The Singularity is Near and The Age of Spiritual Machines. Without them I may not have discovered a passion for discussing accelerating technologies, and I probably wouldn't have this job. Yet I'm rapidly losing interest in evaluating Kurzweil's predictions point by point. I don't see the use. Predictions aren't the means to judge a futurist. The impact they make on the world is.

So here's my summary for how Kurzweil's predictions are faring: Kurzweil got many things right. He also got some things wrong. It's unclear how many of his predictions would have been reasonably anticipated by the general public at the time had they been asked to make similar forward looking statements about 2009.

Now, if you'll excuse me, I think I'll go back to spreading the word about emerging science and technologies one development at a time. It's not as grand as discussing luminaries in the field of futurism, but it seems much more useful.

"Prediction is very difficult, especially if it's about the future." --Niels Bohr (disputed)

[image credit: Rennio Maifredi via Wired]
[source: KurzweilAI.net]

Discussion — 9 Responses

  • Terry Donaghe January 4, 2011 on 4:39 pm

    What the thinker thinks, the prover proves. – Robert Anton Wilson

  • DragonAss January 4, 2011 on 6:41 pm

    There is but one Singularity and Kurzweil is His Prophet…

  • Ben Reierson January 4, 2011 on 7:27 pm

    I think being willing to take a close look at your own past predictions in the harsh light of the present is generally admirable, but you make some excellent points. I think Ray is smart enough to know that making extremely specific predictions is a fool’s game, and he freely admits it’s easier to just make lots of more general predictions because you’ll look smarter.

    If you haven’t read it, I’d recommend Taleb’s ‘The Black Swan’ for insight into how best to think about the future, and more importantly, how not to. One of his biggest complaints is the lack of accountability in disciplines which rely on prediction. So at least Ray is trying here, and I feel like he avoids Taleb’s fallacies better than most.

  • Rev. Joe Zarro January 4, 2011 on 7:57 pm

    I agree with Aaron that it’s less useful to examine the outcomes of specific predictions and rather look at the paradigm as a whole. Kurzweil takes premises that few people hold but empirical evidence suggest is true (exponential technological growth) and then uses fairly solid logic to arrive at novel conclusions. If the premises and logic is sound, than the predictions will be relatively accurate. But when we look at Ray’s predictions 10 years ago, instead of arguing about the definition of “learning to read using computers” we should focus instead on the evidence that either supports or undermines his premises. Obviously educational software has made tremendous strides the past decade, on an exponential scale not a linear one, and toddlers understand how to use iPhones. While most kids still learn to read as they did 20 years ago in school — and his prediction isn’t quite “right” — you can see that the /paradigm/, the structure in which he is forming conclusions, is proving itself to be relatively sound.

    The singularity is not going mainstream because of Ray’s books. Very few people read or paid attention to his books before 2005. It’s because what Ray saw 20 years ago other intelligent people can easily see today — technology is accelerating and our generation is going to see a profound scientific and cultural revolution unlike any other in human history. Exponential growth was evident before the digital age, but now it is difficult to miss and easier to accept.

    Far more important than whether he is right on specific predictions is whether he is write in his general worldview, which is profoundly different from the average person off the street. His paradigm is obviously prevailing, so who cares if his timeline is exact?

  • Kurzweilian January 4, 2011 on 8:57 pm

    A person is allowed to think as much as they want. Some people don’t share thoughts because it might make them feel like an outcast or think that’s there’s some sort of voodoo that goes along with making an accurate “guess.” Biased towards Ray as an great individual because I own one of his awesome keyboards; I say more power to him! Seems to me the ones who ridicule and point out others mistakes instead of coming up with a better or different point, don’t have one…all they can do is point and find mistakes. Great thinkers have been dealing with these kind of imbesols (spelling?) for centuries and I will have a chuckle when something in the future comes out about one of the “scientists” they look up to isn’t actually true, or disproven; but that’s in the future and no one can “really” predict the future! Kudos Kurzweil!

  • David Wood January 5, 2011 on 2:10 pm

    Although I love the idea of a state of objective standards for futurist predictions, I just think it is generally too difficult to get most people to agree on any set of objective standards. I think what we all need to do is read the predictions ourselves, examine what actually happens and make our own subjective judgment and use that judgment to decide how much credence we give individually to a given futurist.

    As for kids reading before kindergarten, my oldest son is three and he’s there. That’s from a combination of reading physical books, and playing spelling and reading games on his Leapfrog hand-held console. Same thing with addition and subtraction. So I’d judge that as a partially correct prediction. Some others might argue that it is ‘essentially correct’ or completely correct.

  • Robert Redford January 8, 2011 on 12:41 am

    I predict that Kurzweil will be about 80% correct in all his future preditions.

  • Simon January 11, 2011 on 3:10 pm

    It is very easy to say “That was obvious” after the fact. Perhaps you can tell us what predictions are obvious for the next 10 years. Write it down, sign your name and stand by it.

  • Anonymous January 17, 2011 on 3:56 pm

    Edward Casey made 10’s of 1000’s of predictions that were all true. He had devine power of being able to go to the place where all things are. Back in his day dignetaries and regular people asked him their relavent questions. His futuristic questions were few. Some are yet to be realised. Non the less he had the power of the exact answer.. He said we all have the ability to do what he did. Basically, the information is already there you just need to know how to find it.