You will be able to recognise when an extreme result was always likely to be followed by a more ordinary one.
In March, the customer service team at Daniel's company has its worst month on record for complaint numbers. A new team lead is brought in at the start of April, introduces a fresh script and daily huddles, and by May complaints are back to normal. At the quarterly town hall, the new lead gets a round of applause. Daniel claps too, and then wonders whether April and May would have looked the same without her.
The question behind his doubt has a name, regression to the mean, and once you know it you will see it in performance reviews, sales reports and almost every story about a turnaround.
Any result you measure at work, whether monthly sales, complaints, error rates or a person's score in a review, comes from two things mixed together. One is the underlying level: how good the team or person usually is. The other is luck: the particular customers who called that month, a system outage, a big order that happened to land in March rather than April, a flu going round the office.
When a result is extreme, very good or very bad, it is likely that luck pushed it in that direction. A team with an ordinary underlying level has a terrible month because several bad things happened to coincide. Next month, those particular bad things are unlikely to all happen again. So the next result tends to be closer to the team's usual level, whatever anyone does.
That is regression to the mean: after an extreme result that was partly due to chance, the next result is usually closer to the average. It is not a force that pulls things back. It is simply what happens when you measure something that has luck in it, and you pick the moment to look because the result was extreme.
This creates a trap. People usually act when results are at their worst. A manager is replaced after a bad quarter, a new process is introduced after a spike in errors, a consultant is hired after sales slump. Because action follows the extreme, the next measurement will usually look better even if the action did nothing. The fix gets the credit for what regression to the mean would have delivered anyway.
The customer service team is a likely case. March was the worst month on record, which means luck was probably working against the team. April and May would probably have improved under any team lead. That does not mean the new lead did nothing. Her script might genuinely help. But the improvement alone cannot show it, because some improvement was likely either way.
The same trap catches people when they decide what works for themselves. You see a doctor when your back pain is at its worst, and it eases a week later. You start a new diet after your highest weight reading of the year. You change your study method after your worst exam. Each change may help, but each was made at the moment when things were most likely to improve anyway.
Regression to the mean also works in the other direction, and it can teach managers exactly the wrong lesson.
Daniel Kahneman describes this in Thinking, Fast and Slow. While teaching Israeli Air Force flight instructors about the psychology of training, he was told by one instructor that praise did not work. Whenever he praised a cadet for an excellent manoeuvre, the next attempt was usually worse. When he shouted at a cadet for a bad one, the next attempt was usually better. The instructor concluded that criticism works and praise does not.
Kahneman's explanation was regression to the mean. An excellent manoeuvre was partly luck, so the next one was likely to be closer to the cadet's usual standard, whatever the instructor said. The same applied to a terrible one, in the other direction. The instructor was seeing a real pattern and drawing the wrong cause from it.
At work, this can push a manager to praise less and criticise more, based on what looks like solid experience. A salesperson who has a record month and is congratulated will probably have a more ordinary month next. That is not complacency caused by praise. It is the record month being partly luck.
The defence is a comparison group. Before crediting a change, ask what happened to similar teams, people or months that did not get the change.
If Daniel's company has three customer service teams and only one got the new lead, how did the other two do in April and May after their own bad Marches? If all three improved by similar amounts, the new lead's script is not what drove the change. If her team improved much more than the others, that is better evidence. Where possible, look at a longer history too. If March was an unusual low after a year of steady numbers, a return to steady numbers is what you would expect.
When no comparison group exists, the honest conclusion is "results improved, and we cannot yet tell how much was the change". That is a reason to keep measuring. Lesson 4.2, Why experiments beat observation, showed how to build a comparison in from the start.
Most workplaces have a story like the customer service turnaround, where a change was celebrated after a bad patch. In the activity below you will find one and write down what a fair comparison would have needed.
Find one case at work where a change was credited after an unusually bad period, and write what a fair comparison would need.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).