The Economist:

In a recent paper Linda Chang of the Toyota Research Institute and her co-authors identify a cognitive bias that they call “quantification fixation”. The risk of depending on data alone to make decisions is familiar: it is sometimes referred to as the McNamara fallacy, after the emphasis that an American secretary of defence put on misleading quantitative measures in assessing the Vietnam war. But Ms Chang and her co-authors help explain why people put disproportionate weight on numbers.

The reason seems to be that data are particularly suited to making comparisons. In one experiment, participants were asked to imagine choosing between two software engineers for a promotion. One engineer had been assessed as more likely to climb the ladder but less likely to stay at the firm; the other, by contrast, had a higher probability of retention but a lower chance of advancement. The researchers varied the way that this information was presented. They found that participants were more likely to choose on the basis of future promotion prospects when only that criterion was quantified, and to select on retention probability when that was the thing with a number attached.

One answer to this bias is to quantify everything. But, as the authors point out, some things are mushier than others. A firm’s culture is harder to express as a number for job-seekers than its salary levels. Data can tell an early-stage investor more about a startup’s financials than a founder’s resilience. Numbers allow for easy comparisons. The problem is that they do not always tell the whole story.