The Reverend Thomas Bayes, RA Fisher, Johann Gauss: your boys took one hell of a beating. That’s right, apparently it wasn’t only the Brazilian national team that suffered ignominious defeat on Tuesday night, as Germany tore them apart in a brutal, tragicomic home-turf humiliation. It was also the abstract concept of statistical probability.
Nate Silver, the stats-nerd pundit who gained worldwide fame by successfully foretelling how almost every state would vote in the 2008 and 2012 US presidential elections, whiles away the time between polling nights by predicting the outcome of sporting contests. That was, in fact, how he got started – he built a statistical model of baseball games which allowed him to evaluate the performance of players – and now he has trained his Nostradamus spreadsheet on the World Cup.
Except the poor guy said that Brazil had a 65 per cent chance of going through to the final, which made him look a little silly when Germany didn’t just sneak through but humbled the hosts 7-1.
The history of sport punditry – and, more specifically, sport forecasting – is long and rich and, generally speaking, ignominious. Sport is unpredictable for the same reasons that the future in general is unpredictable: it is complex and chaotic. But we love to try to predict it anyway.
At the last World Cup, an octopus called Paul gained a lot of attention when he correctly “predicted” the results of all seven Germany matches and the final between Spain and the Netherlands. The odds of that happening are one in 256 (the octopus couldn’t predict draws), which sounds impressive until you think that there might have been quite a lot of other purportedly psychic animals which didn’t make the headlines: this year, the BBC’s World Cup-predicting dog Mr Whippy got his very first one wrong, backing England over Italy, and no one really paid him much attention after that.
Human pundits don’t tend to do all that much better than Mr Whippy, and almost none do as well as Paul. They – we – are fundamentally bad at probability and statistics. We over-extrapolate from small samples, so if a player has a few bad games we assume they’re a bad player; we misinterpret randomness as patterns, so if a team has won three in a row, we tend to assume they’ll win the fourth, even though the evidence is that there is no such thing as momentum in sport. And we’re taken in by our fallible perception. For instance, according to Simon Kuper and Stefan Szymanski’s book Soccernomics, pundits tend to rate blond players more highly than their dark-haired team-mates – not because blonds are better but because they stand out, so the positive things they do stick in the memory. In-built blind spots and hidden prejudices like that make reliable predictions hard.
This doesn’t just apply to sport: human predictions are generally far worse than we think they are, and usually can be beaten by simple algorithms. But we know some people can beat the odds, because there are a few who make a living betting on sport. In his book The Signal and the Noise, Silver describes one such man, Bob Voulgaris, who makes between one and three million dollars a year betting on American sports.
The difference between the shiny-suited pundit on the Match of the Day sofa and the millionaire sports gambler is not some trick or magical insight. The difference is, largely, humility. The psychologist Philip Tetlock ran a 20-year study into predictions by foreign affairs pundits. He found that on average, the pundits were as accurate as “a dart-throwing chimpanzee” when it came to predicting specific events. But some pundits were a bit more accurate (meaning that most were actually less accurate than the chimpanzee). The ones who did better were the ones who accepted that the world is complex and frequently unpredictable, acknowledged when they had got their predictions wrong, and revised their beliefs accordingly. The ones who did badly were the ones who had a big, all-encompassing theory for how things work, and explained away failures as “near misses”. Tetlock called the first kind “foxes” and the second “hedgehogs”, after a line in a Greek poem: “The fox knows many things; the hedgehog one great thing.”
Silver, who is a fan of Tetlock and would no doubt think of himself as a fox, does not pretend that his Brazil prediction was a near miss: “Time to eat some crow. That prediction stunk” was his response. But nor does it undermine his entire prediction system, which is based on an algorithm incorporating a team’s success rate, home advantage, and so on. The question is not whether he got this one “wrong”, but whether his 65 per cent guesses come in 65 per cent of the time; that is the mark of a good predictor.
Of course, Silver was not the only person to get it slightly wrong over the Brazil-Germany game. “Brazil are going to win this World Cup at a canter. They’ve got the talent, the passion, the fans,” said one Piers Morgan. I imagine he’s a hedgehog.