“I figure lots of predictions is best. People will forget the ones I get wrong and marvel over the rest.” –Alan Cox
How would you grade yourself if you had a chance to write your own report card? Ray Kurzweil is giving himself a high B. With his recent essay “How My Predictions Are Faring” the noted futurist reviews forecasts he made more than a decade ago for our current times. His predicted future is now the present, so it’s time to see how he did. The report looks at 147 predictions that Kurzweil made in his 1999 book The Age of Spiritual Machines, and goes on to briefly explore comments in The Age of Intelligent Machines (1990) as well as his more recent The Singularity is Near (2005). Of the 147 predictions Kurzweil maintains that 115 were ‘entirely correct’, 12 ‘essentially correct’, and 17 ‘partially correct, with just 3 being outright wrong. Counting only the entirely and essentially correct, Kurzweil claims an accuracy of 86%. The report, dated October 2010 and now widely available on the web, has sparked heated discussions on various tech forums. I’ve read “How My Predictions Are Faring” and have drawn my own conclusions about Kurzweil’s prowess: If you’re pouring over each of the 147 statements and debating their accuracy then you’re probably wasting your time. When it comes to evaluating predictions, I’m not sure we’re doing it right.
Before I Cop Out, A Review
For years, Ray Kurzweil has been one of the foremost futurists in the public arena. He predicted the fall of the Soviet Union, and the rise of the internet. He’s also predicted the rise of artificial intelligence and the eventual union of man and machine to occur in the 21st century. While he didn’t coin the term ‘Singularity’, he has brought the concept of an exponential growth in technology to the masses through books, documentaries, and countless speeches around the world. In many ways, Kurzweil has become the face of the Technological Singularity, and that has made him a target to techno-optimists and Luddites alike.
Perhaps rightfully so – many world leaders in industry and government listen to his opinions. For instance, he’s an advisor to the U.S. Army for developing a rapid response to biological threats. If Kurzweil’s beliefs about the future affect public policy it’s important we understand why he’s given such clout, and that we challenge him about his errors.
In his book The Age of Spiritual Machines (written in the late 90s and published in 1999), Kurzweil made definitive statements about the state of technology in the year 2009. Actually, he was talking about that period in time in general, so we can add some error bars of plus or minus a few years around 2009. Now that we’re entering 2011, though, we can get a good idea about whether or not Kurzweil’s 2009 predictions were correct.
Hence the “How My Predictions are Faring” report. It’s a 147 page review of his 147 predictions (can’t be coincidence) from ASM and it goes into each statement in full detail. Kurzweil steps you through numbered lists of these predictions as divided by topic and gives you his reasoning for why each was correct, essentially correct, partially correct, or wrong. For someone looking to review Kurzweil’s precognition the report is easy to access.
With access has come the opportunity for controversy. Critics are able to go through and evaluate not only the original predictions, but Kurzweil’s explanations of those predictions. We now have 147 more pages of text to debate on forums and in comments. You could, if you like, do exactly that yourself. The essay is freely available on Kurzweil’s site, and mirrored on Scribd. You could spend hours looking through that document and analyze every sentence for accuracy and then share your thoughts with the internet-capable world.
But please don’t waste your time.
And Now the Cop Out
Look, I encourage people to read things for themselves and I encourage healthy debate. These are essential parts of the scientific process, crucial skills for society as a whole, and the only way we can shape the internet into a tool for enlightenment. Yet I’m deeply worried when I see the techno-optimistic community putting efforts into these kinds of debates, because I believe that they are largely unnecessary. Why?
Judging the accuracy of predictions like these is simply too subjective. Predictions are drastically influenced by interpretation, as you can see in Kurzweil’s report. When he reviews each statement, Kurzweil is selecting those parts he finds relevant and commenting on them. You and I might choose to make judgments based on completely different sections. Take #13 in Education: “Many children learn to read on their own using their personal computers before entering grade school.” Kurzweil deems this as correct by focusing on the developmental aid young children experience after using computers habitually. He also points to a report that 9% of children who enter kindergarten already know how to read. A critic, on the other hand, might have called this a fail because developmental skills are not the same thing as actually reading, and increases in pre-K reading skills aren’t necessarily due to computer usage. By focusing on different parts of a prediction you can draw drastically different conclusions.
Well, duh, Aaron.
Ok, so maybe it’s obvious that judging predictions is going to be subjective. But the pervasiveness of this kind of selection bias isn’t always clear. After all, in the paragraph above you might have evaluated prediction #13 and drawn your own conclusions about whether or not Kurzweil was right. But did you consider whether or not #13 in Education was the best example for me to use in that paragraph? Why not #19 in Communication or #5 in Politics and Science?
Everyone who comments on Kurzweil’s work is selecting both the predictions they want to focus on as well as the parts of that prediction they want to focus on. That doubles the bias and it means that it will be very hard for anyone to come to an agreement about Kurzweil’s predictive prowess as a whole. This is a problem I came across when reviewing Michael Anissimov’s critique of Kurweil a year ago. When there is this much picking and choosing to be done I think that the debate simply devolves into arguments over what to talk about. Or what we think someone else was talking about. Critics and counter-critics will simply talk around each other rather than coming to a consensus.
The way I see it, there are only two ways out of this mess.
First, we can set objective standards for predictions. We can require futurists to make statements that are as clear and closed to subjective interpretation as possible: “By January 2015 (plus or minus 12 months) the gross domestic product of the US, as measured by the Bureau of Economic Analysis, will surpass 30 trillion dollars (not adjusted for inflation).” We can then take those predictions and assign them a score that’s either binary (did it come true or not) or graded (how many years difference was there between the predicted and actual date for an event). Collect all the predictions for a futurist in one place and then give them a general score for their efforts.
To some degree the Long Bets website is already trying to do this. They even added the element of making wagers. If we expanded this concept to every single prediction someone like Kurzweil makes, then maybe we’d have an objective score for their skills.
The second idea would be to focus on the broad picture a futurist paints. Instead of evaluating each of the 147 predictions we could try to summarize what Kurzweil is saying in general. This will be highly subjective, but it might help us come to an agreement we could act upon. Is Kurzweil basically telling us that “information technology will continue to accelerate indefinitely.” If we accept such a premise we can start trying to find ways to take advantage of that general trend for the betterment of everyone. I think Singularity University operates on such an assumption.
The first solution would require us to throw out many predictions as unclear, the second solution would have us abandon objectivity to a great extent. So yeah…neither really seems like a great choice.
Yet I think the current trend of evaluating subjective statements objectively is an even bigger mistake. As objective as Kurzweil may have been trying to make his predictions in The Age of Spiritual Machines, I think most simply aren’t precisely worded enough to evaluate as such. I’ve spent hours looking over this document, reading the original statements and Kurzweil’s evaluations, and I find myself being able to argue for and against each prediction. It’s all about selection. I have pages of detailed notes on this document, and I’m looking at them and asking myself, what use are these?
We don’t have a baseline comparison for evaluating the statements of futurists. If Kurzweil makes a prediction that anyone could have guessed, it doesn’t speak much to his credit.
#20 in Education: “Learning is becoming a significant portion of most jobs.”
#6 in Business : “China has also emerged as a powerful economic player.”
#19 in Arts: “Users also experience fantasy environments with no counterpart in the physical world.”
These all should have been expected, I hope.
#6 in War: “Military conflicts between nations are rare, and most conflicts are between nations and smaller bands of terrorists,” that might have been less obvious to the public ten years ago but experts would have taken it as a given.
And #15 in Business: “There is a strong trend towards the geographic separation of workgroups. People are successfully working together despite living and working in different places.” Even if we didn’t understand the depth of how powerfully the internet would affect work, we probably could have made a similar statement in 1999, right?
(But there I go being selective. I’ve chosen five statements to prove a point, but you could probably find another five that demonstrate how insightful Kurzweil was.)
If we don’t have a baseline measurement about our ability to predict the future then we can’t establish a reasonable expectation about how accurate prominent futurists will need to be in order to trust them more than average man on the street.
Even if we do establish that some experts have been better at predicting the future than others, how much do we invest in the predictions they make next? If Kurzweil has been 86% accurate about 2009, can we say he’ll be 86% accurate about 2019, 2029, or 2039 ? Does it just mean that he’s accurate 10 years out? By the time we could say for sure the world would have aged decades.
I’m indebted to Kurzweil for writing books like The Singularity is Near and The Age of Spiritual Machines. Without them I may not have discovered a passion for discussing accelerating technologies, and I probably wouldn’t have this job. Yet I’m rapidly losing interest in evaluating Kurzweil’s predictions point by point. I don’t see the use. Predictions aren’t the means to judge a futurist. The impact they make on the world is.
So here’s my summary for how Kurzweil’s predictions are faring: Kurzweil got many things right. He also got some things wrong. It’s unclear how many of his predictions would have been reasonably anticipated by the general public at the time had they been asked to make similar forward looking statements about 2009.
Now, if you’ll excuse me, I think I’ll go back to spreading the word about emerging science and technologies one development at a time. It’s not as grand as discussing luminaries in the field of futurism, but it seems much more useful.
“Prediction is very difficult, especially if it’s about the future.” –Niels Bohr (disputed)
[image credit: Rennio Maifredi via Wired]
[source: KurzweilAI.net]
