Three other reasons two things move together

You will be able to list what else, besides causation, could explain two things being linked.

Next time you see a claim that two things are linked, write it at the top of a page. Under it, write four headings: confounder, reverse causation, chance, selection. Before you act on the claim, fill in at least one line under each. A finding can be accurate while the conclusion drawn from it still does not follow. Two things going together says nothing yet about whether one causes the other. Whenever two things are linked, there are at least four explanations other than direct causation, and each of them can be checked.

Lesson 2.3, Sunk costs and other traps in your own head, introduced post hoc reasoning: B follows A, and A gets the credit. The four alternatives below take that idea further and give you a question to ask for each.

Daniel's report on app users

Daniel works in marketing for an online home goods retailer. His team's monthly report finds that app users spend about twice as much a year as website-only customers (an invented example, like the other figures in his case). His manager reads it and concludes, "We need to push everyone to download the app." The budget for an app download campaign is approved the same week.

The figure in the report may be correct. What is still open is whether downloading the app is what makes people spend more.

Confounders and reverse causation

A confounder is a third factor that drives both things, so they rise and fall together while neither causes the other. It is the first alternative to check and the most common. People with gym memberships tend to be healthier. The gym may help, but higher-income people are more likely to afford a membership and also more likely to be healthier for other reasons: better housing, more time to cook, work that is less physically damaging. Income sits behind both, and the link would still appear even if the gym did nothing.

To find confounders, ask what kind of person, team or situation would be more likely to have both of these things. Any answer is a candidate confounder. In Daniel's case one answer is loyalty, meaning how much a customer already likes the shop. Loyal customers who buy often are more likely to download the app, and their loyalty drives both the download and the spending. If that is what is happening, getting casual customers to install the app leaves them as casual as before.

With reverse causation, the arrow points the other way: what was taken for the effect is actually producing the supposed cause. It is common in workplace data. Teams that hold more meetings have more problems, and it may be the problems that cause the meetings. Employees who use the company wellness programme report more stress, possibly because stressed employees seek the programme out.

The question to ask is whether the second thing could have caused the first. Picture a customer who has just collected the keys to a new HDB flat. She will spend heavily on furniture and fittings over the next few months, and because she is ordering so much, she installs the app to track deliveries and use vouchers. Her spending came first and led to the download.

Chance and how the group was picked

Chance is the third alternative: look at enough pairs of numbers and some will line up by accident. Over any few years, a count of something in one country can track a count of something unrelated elsewhere simply because both happened to rise, and collections of these absurd matches exist online. At work, chance matters most when someone searched through many possibilities to find a pattern. If an analyst checks fifty customer characteristics against spending, a few will appear linked by luck alone. The most interesting of those is the one that reaches the slide, and the forty-nine that showed nothing go unmentioned. So ask how many other things were checked before this one was found, and ask how large the sample is. Small samples make chance worse: a link across twelve customers is far more likely to be luck than the same link across twelve thousand.

The fourth alternative is selection, where the way a group was picked creates a link that does not exist across everyone. It is a close cousin of survivorship bias, covered in lesson 3.4, Survivorship bias and missing data. Selection is easy to miss because it happens before any analysis starts, in the decision about which rows to pull. Suppose Daniel's report includes only customers with at least one purchase this year. Website users who browsed and bought nothing are then missing, and that can change how app users compare with website users. Ask who was included, who was excluded, and whether the exclusion could be related to both things being compared.

What Daniel writes down before the campaign runs

Daniel does not argue that the app is useless. He writes down the four alternatives, each with a question that would check it:

Confounder and reverse causation: were app users already heavier buyers before they downloaded, and what did spending look like before and after the download for the same customers? Chance: how many other customer features were compared before this one stood out? Selection: does the comparison include visitors who never bought anything?

The first question is the most useful in his case, because it addresses the confounder and reverse causation directly. If spending was already high before the download and did not change much afterwards, the campaign is unlikely to work. Instead of a full campaign, Daniel could test the idea properly with a small trial, and lesson 4.2, Why experiments beat observation, shows how.

Claims of the form "people who do X have better Y" are common in health articles, management books and company reports. In the activity below you will take three of them and write one confounder and one reverse explanation for each.

Pick three claims of the form "people who do X have better Y". For each one, write down one confounder and one reverse explanation.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).