The Wisdom of Crowds

Since the beginning of the “age of the common man” in the nineteenth century, there has been a widely held belief that, as Henry Thoreau put it, ‘The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest.’ The French sociologist Gustave le Bon articulated this belief most succinctly when he said, ‘Crowds can never accomplish acts demanding a high degree of intelligence,’ because they are ‘always intellectually inferior to the isolated individual.’

Not surprisingly, this kind of thinking has most often been found in people who consider themselves to be members of society’s intellectual elite; Francis Galton – founder of Eugenics – H.G. Wells, George Bernard Shaw, D. H. Lawrence, Friedrick Nietzsche and Henrik Ibsen.  More recently the stupidity of the masses has been remarked on by politicians like Tony Blair who has warned against the rise of ‘populism’ by which Blair means the electorate falling to the lowest common denominator of the crowd’s intelligence.

This matter has, of course, been put under the spotlight most recently by the result of the Brexit referendum in the UK and the Presidential election in the US. In both cases it has been strongly alleged – by the losers – that the winning vote arose from group stupidity, gullibility, lack of information, susceptibility to fake news, outside interference and the manipulation of voters through targeted Facebook advertising.

It’s rather surprising that the ‘stupidity of the masses’ idea should have gained traction again at all, because it was comprehensively demolished with empirical fact in 2004 by James Surowiecki in his book “The Wisdom of Crowds”.

The number of examples of group decisions excelling those of expert individuals is so great that I can only advise you to read Surowiecki’s book yourself. It is an enjoyable as well as an eye-opening experience.  But I’ll pull out a couple of outstanding examples here because they show, as the author says, that the evidence for the superior wisdom of crowds is all around us, every day, yet we scarcely notice it.

Most people have watched, or know of, the TV show “Who wants to be a millionaire”. Contestants have the chance of winning £1 million ($1 million in the US) if they can answer 15 questions correctly. If they are unsure of the answer they have the option of phoning a friend- someone knowledgeable in the area of the question – or asking the audience who vote via computer.

You might expect that phoning an expert in the field would be a better bet than asking a random audience. Yet the ‘experts’ were right 65 per cent of the time, while the audiences were right 91 per cent.

The space shuttle Challenger exploded shortly after takeoff at 11:39 on 28 January 1986. The first news story was on the Dow Jones News Wire service 8 minutes later. Immediately, investors on the stock market started dumping their shares in the four major contractors to the launch: Rockwell who built the shuttle and its main engines; Lockheed who managed ground support; Martin Marietta who made the external fuel tank; and Morton Thiokol who built the solid booster rocket.  Twenty one minutes after the explosion Lockheed shares were down 5 per cent, Martin Marietta’s 3 per cent and Rockwell down 6 per cent. The hardest hit was Morton Thiokol – so many people tried to dump its shares that trading in them was suspended. When it was resumed, they were down 12 per cent. The shares in the other three crept back up and by the close of business were all down only 3 per cent.

There was no speculation on TV or in the papers about the cause which was a mystery that took six months to unravel. When the cause was identified – the O-ring seals on the booster rockets – it was indeed Morton Thiokol to blame. The stock market had identified them as the culprit within minutes.

One possibility that was explored was some kind of insider trading although no evidence of this was found. Indeed the logical thing for an insider to do would be to sell Morton Thiokol shares and buy the other three companies stock – but that didn’t happen.  How thousands of shareholders were able to suspect a component that took Nobel laureate Richard Feynman and his expert team six months to determine remains a mystery.

Surowiecki also examines ‘experts’ versus the crowd in science with some surprising results. He points out that, ‘Since scientific results should be replicable, you don’t in theory have to trust anyone’s judgement [as to whether a theory is valid]. If an experiment works, it will work whether or not the vast majority of scientists say it does or not. But the picture is more complicated than this. Most scientists are never going to replicate other experiments. They are going to trust that the data is correct and that the experiments worked as the scientists who performed them said they did. A successful hypothesis is a hypothesis that most scientists find credible, not a hypothesis that most scientists have tested for themselves and found to be true.’