Experts Simply Aren't That Expert

I recently read Philip Tetlock's book Expert Political Judgment, which presents his 20-year study about experts' ability to predict the future. The insights he gleaned about the so-called experts who make prediction their business are certainly worth sharing, especially as he found they're no better than the proverbial chimps throwing darts.

Tetlock divided forecasters into two general categories: Hedgehogs and Foxes.

• Foxes draw on a wide variety of experiences and don't boil the world down to a single idea.
• Hedgehogs view the world through the lens of a single defining idea.

• Hedgehogs are more confident than foxes, but they're also wrong more often.
• Foxes rarely see things as bad as they appear at the trough or as good as they look at the peak.
• Optimists tend to be more accurate than pessimists.
• Backgrounds and experience made virtually no difference. The only predictor of accuracy was fame, which was negatively correlated with accuracy: The most famous -- those more likely feted by the media -- made the worst forecasts.
• Beyond a stark minimum, subject matter expertise translates more into overconfidence than forecasting accuracy -- and the ability to spin elaborate tapestries of reasons for expecting "favorite" outcomes.
• Experts fall prey to the hindsight effect, claiming they knew more about what was going to happen than they actually knew before the fact. "This systematic misremembering of past positions may look strategic, but the evidence indicates that people sometimes truly convince themselves that they 'knew it all along.'" Hindsight bias causes overconfidence.
• The marketplace of ideas may fail because consumers simply listen to experts who share their points of view, rather than listening to all ideas.

Sadly, Tetlock concluded: "No matter how unequivocal the evidence that experts cannot outpredict chimps or extrapolation algorithms, we should expect business to unfold as usual: pundits will continue to warn us on talk shows and op-ed pages of what will happen unless we dutifully follow their policy prescriptions. We, the consumers of expert pronouncements, are in thrall to experts for the same reasons that our ancestors submitted to shamans and oracles: our uncontrollable need to believe in a controllable world and our flawed understanding of the laws of chance. We lack the will power and good sense to resist the snake oil products on offer. Who wants to believe that, on the big questions, we could do as well tossing a coin as by consulting accredited experts."

However, he noted that all is not lost: "Even fierce resistance can be overcome. Low-transaction-cost index funds have benefited -- very substantially -- from slowly spreading knowledge of how hard-pressed stock-pickers are to best dart-throwing chimps and other mindless algorithms."

The lesson Tetlock's research provides is that as much as we would like to believe there are those who can predict the future, prognosticating is the occupation of charlatans. What investors should focus on are the things they can control -- the amount of risk they take, costs, tax efficiency and not spending too much.