Ever since he took the world of political punditry by storm in 2008 with his electoral prediction website, FiveThirtyEight, Nate Silver has been one of the more high profile stats-gurus going around. Having been an avid follower of his blog (before and after its move to The New York Times), I was keenly interested to read his book debut, the bestselling The Signal and the Noise.
And having utilised my summer to work through his pretty lengthy read, it’s safe to say I have a better appreciation of what we can and can’t do when making predictions and forecasts, particularly in an economic context.
Silver goes through a variety of fields that utilise forecasting and predictive techniques heavily with the practitioners within these areas enjoying varied successes. Some of the fields, such as weather forecasting and betting in sports (with a heavy focus on Bayesian probability theory) have utilised forecasting with increasing skill and accuracy, whereas our fellow economists have become notorious for their failure to work through an array of data and decipher what it’s telling us; to truly perceive the signal through all the noise that comes with the world we live in.
This failure of economic forecasting is particularly highlighted by Silver in his opening chapter, an indictment on those who failed to predict the onset of the Global Financial Crisis in 2007-8 either through an incomplete understanding of the context they worked in, or self-interest, as has been demonstrated in the actions of credit agencies in the years leading up to the crisis.
Silver explains that in this ‘catastrophic failure of prediction’ the recurring failure when it came to an array of predictions (the trajectory of housing prices, the capacity for agencies to predict the risk of a complex security like a derivative, the capacity for the housing crisis to infect the entire financial system and the capacity for it affect the whole economy) was that the participants involved weren’t acknowledging that these events were occurring out of sample. By failing to recognise this context they were operating in, in turn bringing about a false sense of confidence, policymakers, consumers and traders alike continued on a path to chaos which eventually came with the bursting of the housing bubble.
In a later chapter Silver elaborates on the challenges of forecasting changes in economic data, and how a failure to properly explain the uncertainty in the data can create misleading reports to the general public. Such failures can have immense effects, particularly in a political context that feeds off such minute-to-minute data.
So what can we learn from a book like Silver’s when it comes time for us to take up the challenging task of prediction; especially in an economic context? For me, it is to embrace and accept that there is uncertainty in the world, even in the era of Big Data. And whilst we have made significant progress when it comes to explaining how the world works and doesn’t work we must accept that there are limits to our understanding, and that they can only begun to be overcome through objective analysis of the data and an imaginative mind to think about what it means.
As Silver explains, “The more eagerly we commit to scrutinising and testing our theories, the more readily we accept that our knowledge of the world is uncertain, the more willingly we acknowledge that perfect prediction is impossible, the less we will live in fear of our failures, and the more freedom we will have to let our minds flow freely. By knowing more about what we don’t know, we may get a few more predictions right.”
If we, as wannabe-economists, can commit ourselves to abiding by this statement then surely we will be contributing something of value in our careers to come.
You can follow me on Twitter @CRJWeinberg.