The ox that made a statistician change his mind

In the autumn of 1906, Francis Galton walked through the West of England Fat Stock and Poultry Exhibition in Plymouth. He was 84, a half-cousin of Charles Darwin, and one of the most prolific scientists of his age: he had pioneered weather maps, the statistical idea of correlation and the use of fingerprints to identify people. He was also, by temperament, a man with little faith in the judgement of ordinary people.

At the fair, a competition caught his eye. For sixpence, visitors could buy a ticket and guess what an ox on display would weigh once it had been slaughtered and dressed. The best guesses won prizes. Galton saw a small, tidy experiment: hundreds of people, one question, a single right answer that would soon be known. So after the contest he borrowed the tickets.

What the tickets said

There were 800 of them. Thirteen were defective or illegible, which left 787. A few came from butchers and farmers who had spent their lives judging cattle. Many more, Galton noted, came from people with no special knowledge at all — clerks, families, people who simply liked a flutter. The guesses ranged widely, some absurdly high, some absurdly low.

Galton arranged the estimates in order and took the one in the middle — the median, or what he called the “middlemost” estimate, the value that half the crowd guessed above and half below. It was 1,207 pounds. The ox, once dressed, weighed 1,198 pounds. The crowd, taken together, was off by less than one per cent.

He published the analysis in Nature in March 1907, under the title “Vox Populi” — the voice of the people. In a short exchange of letters that followed, he also reported the plain average of the guesses, which turned out to be closer still: 1,197 pounds.

Why it worked

The trick was not that the crowd was secretly clever. Each person was wrong — but wrong in their own way. One guesser was misled by the animal’s size, another by its coat, another by the weight of the last ox they had seen. Some errors pushed the estimate up, others pushed it down. Put enough independent mistakes side by side and many of them cancel out, leaving something close to the truth in the middle.

Nearly a century later, the journalist James Surowiecki opened his book The Wisdom of Crowds (2004) with Galton’s ox and asked what made crowds like this one work. His answer was a short list of conditions:

• Diversity — people bring different information and different ways of looking at the problem.
• Independence — each person forms a view without simply copying the others.
• Decentralisation — people draw on their own local knowledge rather than one shared source.
• Aggregation — there is some way to combine all the private judgements into one answer.

The Plymouth fair happened to tick every box. Nobody could see anyone else’s ticket, the visitors came from very different walks of life, and the competition itself — tickets in, numbers out — was a ready-made way to collect them.

Jelly beans and classrooms

The effect has been reproduced many times in far humbler settings. In a well-known version, the finance professor Jack Treynor asked his class of 56 students to estimate the number of jelly beans in a jar. The jar held 850. The group’s average was 871 — and only one student in the room guessed better than the average of everyone. Treynor published the result in the Financial Analysts Journal in 1987, as a way of thinking about why markets that pool many opinions can be hard to beat.

Teachers have run the bean jar so often since that it has become a classroom ritual. The details change; the pattern is stubborn. A single expert can beat the crowd, but it is surprisingly hard to know in advance which single person that will be.

Why it breaks so easily

The same logic explains why crowds can also be spectacularly wrong. Remove independence, and the errors stop cancelling. If people can hear each other’s guesses, they start to lean the same way — towards the loudest voice, the first number said aloud, or the most senior person in the room. The mistakes line up instead of offsetting each other.

A 2011 study in the Proceedings of the National Academy of Sciences, led by Jan Lorenz with colleagues at ETH Zurich, tested this directly. Participants answered factual questions, such as estimates about Swiss geography and crime statistics, for small cash rewards. Some groups worked alone. Others were shown what the rest of the group had answered before trying again.

Seeing others’ answers made the groups more alike, but not more accurate. The range of guesses shrank, and people grew more confident — even though the collective answer did not get closer to the truth. In some cases it drifted further away.

A crowd inside one head

If a crowd works because independent errors cancel out, could one person get some of the same benefit by guessing twice? In 2008, the psychologists Edward Vul and Harold Pashler tested exactly that, in a study published in Psychological Science under the title “Measuring the crowd within”.

They asked people general-knowledge questions with numerical answers — the kind of thing nobody knows exactly, such as what percentage of the world’s airports are in the United States. Later, without warning, they asked the same people to answer again. The average of a person’s two guesses was more accurate than either guess on its own.

The benefit was much smaller than averaging across different people, because our second guess is not truly independent of our first. But it grew when the second guess came weeks later rather than immediately. Time, it seems, lets us forget some of our own reasoning and look at the question with slightly fresher eyes.

Crowds by design

Long before the term “wisdom of crowds” became popular, people tried to build the conditions Galton stumbled on into formal methods. In the 1950s, researchers at the RAND Corporation, among them Olaf Helmer and Norman Dalkey, developed what became known as the Delphi method for forecasting.

In a Delphi exercise, experts answer a question privately and anonymously. A facilitator shares a summary of the answers — often the range and the middle — and the experts are invited to reconsider, again in private. After a few rounds the estimates usually converge. The design keeps the useful part of hearing others, while protecting against the loudest or most senior voice in the room taking over.

What this means in an ordinary meeting

Most of us are not in the business of weighing oxen, but we sit in the modern version of Galton’s fair all the time: a team estimating how long a project will take, a family guessing what a holiday will cost, a group of friends deciding which restaurant will be quieter on a Friday.

The order in which people speak quietly shapes the answer. The first number said aloud becomes an anchor. The most confident person sets the tone. By the time the quietest person in the room speaks, their honest guess has often been rounded towards everyone else’s. Some organisations deal with this by asking people to write their estimates down before any discussion — an echo of the sealed tickets at Plymouth.

The part that is easy to miss

The most interesting person in the story may be Galton himself. He went to the fair half expecting to show how poorly ordinary people reason. When the numbers said otherwise, he did not bury them. He wrote them up carefully, with his methods, and sent them to one of the most respected science journals in the world.

It would be wrong to turn Galton into a simple hero. He also founded the movement he named eugenics, built on ideas about heredity and human worth that later fuelled grave harm, and that science has thoroughly rejected. His story is a reminder that brilliant people can be badly wrong about some things and admirably honest about others.

But on that autumn day in Plymouth, faced with evidence against what he believed, he let the evidence win. Changing your mind in public, at 84, about something you had argued for years, is rarer than any crowd effect. It is also a skill — and like most skills, it gets easier with practice.