Noise: A Flaw in Human Judgment

Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein | Judgment

Why we love it: A genuinely new perspective about how we err when we make judgments. It turns out that bias—systematically getting things wrong—isn’t the only challenge. Noise in human judgment is just as problematic. From three masters of the science of judgment, learn why and what to do about it. 

From pages 40-41:
Although accuracy is the goal, perfection in achieving this goal is never achieved even in scientific measurement, much less in judgment. There is always some error, some of which is bias, and some of which is noise. 

To experience how noise and bias contribute to error, we invite you to play a game that will take you less than one minute. If you have a smartphone with a stopwatch, it probably has a “lap” function, which enables you to measure consecutive time intervals without stopping the stopwatch or even looking at the display. Your goal is to produce five consecutive laps of exactly 10 seconds, without looking at the phone. You may want to observe a 10-second interval a few times before you begin. Go. 

Now look at the lap durations recorded on your phone. (The phone itself was not free from noise, but there was very little of it.) You will see that the laps are not all exactly 10.00 seconds, and that they vary over a substantial range. You tried to reproduce the same timing exactly, but you were not able to do so. The variability you could not control is an instance of noise…

Now look at the five numbers on your phone. Do you see a pattern? For instance, are all five laps shorter than 10 seconds, suggesting that your internal clock is running “fast”? Bias in this simple task is the difference, positive or negative, between the mean of your laps and 10 seconds. Noise is the variability of your results….