Wednesday 14 March 2018

Superforecasting: The Art and Science of Prediction by Philip E Tetlock and Dan Gardner



Philip Tetlock runs a forecasting tournament / competition / information gathering exercise so he can author books like this one. If you are selected to be a forecaster, you will need to predict all sorts of information - the political, economic and key events to come. This book, co-authored by Dan Gardner, summarizes the findings of the tournament, and attempts to answer some questions, like - Is anyone good at prediction? Who? And why?

There are a lot of anecdotal stories about good and bad forecasting and forecasters. The first thing we learn is that the experts are not. Experts at forecasting, that is. Well, that is useful information, but we mostly knew that before. They're also not accountable - never explaining why predictions don't pan out, or admitting errors.

There are also some good stories about history repeating itself, although not often enough, or in the same way as to make forecasting it easier. The other point that stories and narratives explaining predictions anchor us, and make us slow to change our minds was interesting.

The key insights seemed to be that in order to be a good forecaster you need to a) be intelligent and b) update your forecasts continuously.

Some memorable quotes - 

Keynes is always ready to contradict not only his colleagues but also himself whenever circumstances make this seem appropriate,” reported a 1945 profile of the “consistently inconsistent” economist. “So far from feeling guilty about such reversals of position, he utilizes them as pretexts for rebukes to those he saw as less nimble-minded. Legend says that while conferring with Roosevelt at Quebec, Churchill sent Keynes a cable reading, ‘Am coming around to your point of view.’ His Lordship replied, ‘Sorry to hear it. Have started to change my mind.’ 

“It’s very hard to master and if you’re not learning all the time, you will fail. That being said, humility in the face of the game is extremely different than humility in the face of your opponents.” 

“It was the absence of doubt—and scientific rigor—that made medicine unscientific and caused it to stagnate for so long.”

“All who drink of this treatment recover in a short time, except those whom it does not help, who all die,” he wrote. “It is obvious, therefore, that it fails only in incurable cases.” 

“It is wise to take admissions of uncertainty seriously,” Daniel Kahneman noted, “but declarations of high confidence mainly tell you that an individual has constructed a coherent story in his mind, not necessarily that the story is true.” 

“So finding meaning in events is positively correlated with well-being but negatively correlated with foresight. That sets up a depressing possibility: Is misery the price of accuracy?”

“there is no such thing as failure. Failure is just life trying to move us in another direction….Learn from every mistake because every experience, encounter, and particularly your mistakes are there to teach you and force you into being who you are.” Everything happens for a reason. Everything has a purpose.” 

“I have been struck by how important measurement is to improving the human condition,” Bill Gates wrote. “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal….This may seem basic, but it is amazing how often it is not done and how hard it is to get right.” 

“It’s a rare day when a journalist says, “The market rose today for any one of a hundred different reasons, or a mix of them, so no one knows.”

“In one of history’s great ironies, scientists today know vastly more than their colleagues a century ago, and possess vastly more data-crunching power, but they are much less confident in the prospects for perfect predictability.” 


This was a most enjoyable read while I was at it, but considering the key takeaway - keep updating your predictions, that's the best way to ensure that they are the closest to the truth that emerges  - it seems to be a bit redundant and obvious, and not all that helpful. Aren't we looking for super forecasting that can predict outcomes long before the events, instead of just the day, or hour before?

4 stars

ISBN: 9780804136693

You may also enjoy Richard Thaler's Misbehaving. Or The Undoing Project by Michael Lewis.

No comments: