Good decisions can have bad outcomes

You will be able to judge a decision separately from how it turned out.

Back in Module 1, Wei Ling mapped out a proposal from her manager, Kumar, for a new warehouse scanning system. She advised holding the installation until the year-end peak season had passed, and the company took her advice.

In January, Kumar learns that a competitor put in the same system in October and went on to have its smoothest peak season ever. He sends the news on to Wei Ling with a single line: "Should have listened to the vendor." He has looked at what happened to another company and used it to mark her advice as wrong. That way of looking back is the commonest error people make when they review a choice. This module deals with separating a bad decision from a bad outcome, and it starts with that error.

Where the word resulting comes from

You met Annie Duke and her book Thinking in Bets in Lesson 7.1, Decisions are bets. Duke takes a word poker players use, resulting, for the habit of rating a decision by how its outcome turned out.

The first example in her book comes from American football. In the last seconds of the 2015 Super Bowl, Pete Carroll, coach of the Seattle Seahawks, had his team pass the ball near the goal line, where everyone expected a running play. The other side intercepted the pass and Seattle lost the game. By the next day, commentators were widely calling it one of the worst calls the game had ever seen. Duke's view is that Carroll had sound reasons for the call, and that the commentators were reacting to the ending. If the receiver had caught the ball, the same call would have been described as clever.

People slip into resulting easily. After the event, the outcome is in plain sight and hard to forget. The reasoning behind the choice, and what the person could see when they made it, have left no visible trace, so the outcome ends up standing in as the judgment.

Luck pulls decisions and outcomes apart

Lesson 7.1 made the point that every decision is a bet, with luck playing a part. A decision and its outcome can therefore disagree, and they can disagree in either direction.

Good decisions sometimes end badly because luck went against them. Say you look over a car with care, have it inspected and pay a fair price for it, and two months later the gearbox fails. Or you run a careful hiring process, pick the strongest candidate, and she resigns after six months because her spouse has been posted overseas.

Careless decisions sometimes end well because luck went their way. A driver has several drinks, drives home and gets there safely. A colleague acts on a friend's tip and puts most of his savings into one stock, which then doubles. No one would defend either choice as sound, yet if you looked only at the result you would have to call both of them good decisions.

Cross the two kinds of decision with the two kinds of outcome and you get the four boxes: good decision with good outcome, good decision with bad outcome, bad decision with good outcome, and bad decision with bad outcome. Resulting squeezes these down to two, treating every good outcome as a good decision and every bad outcome as a bad decision.

When you grade by outcome only, both kinds of luck teach you the wrong thing. Bad luck after a good decision can push you to drop a process that was working. Good luck after a careless decision can encourage you to repeat the careless process. The colleague with the stock tip may well try it a second time with a bigger sum, having drawn the reverse of the lesson his process earned.

How organisations learn to gamble

Teams and companies do the same. When a project succeeds, others copy its methods, even when a lucky market was what carried it. When a manager's risky call pays off, the manager is promoted and goes on making risky calls. Across years, this rewards luck and penalises caution, and the organisation gradually gets into the habit of gambling.

The stories that travel make it worse, since successes tend to be passed around and failures tend not to be. This is the survivorship problem from Lesson 3.4, Survivorship bias and missing data. Kumar heard about the competitor because its October install went well. A different company might also have installed in October and had a disastrous peak, and no one would have thought to forward that to him.

Three questions for reviewing a decision

To avoid resulting, grade the process behind a decision, and judge it only against what was known, or could have been known, at the moment it was made. Work through three questions.

What did I know at the time? Write out the information that was in front of you when you decided. Keep out anything you found out later. What could I have known? Look for information you could have got at a reasonable cost but did not. Real mistakes are often found in this gap. How did I reason? With the information you had, was your thinking sound? Check whether you weighed the options, the probabilities and the worst case, and whether you fell into any of the traps on your bias checklist from Lesson 5.4.

What you do next depends on the answers. A good process followed by a bad outcome points mostly to bad luck, and you keep the process. A gap in the process needs fixing whether the outcome was good or bad.

Wei Ling answers Kumar

Apply the questions to Wei Ling. When she gave her advice, she knew the vendor had no evidence of installations going smoothly close to a peak, and she knew the company's own error data covered only one busy month. On what she could have known, she might have asked the vendor for a list of clients who had installed just before a peak. Her reasoning turned on the risk of an install failing during peak season.

The danger in Kumar's note is that she takes the wrong lesson from it, deciding she was too cautious and should go along with vendor timelines in future. Her reply makes four points in her own words: the decision was based on the risk of a failed install at peak; the competitor accepted that risk and it worked out for them; with the information they had, she would make the same call again; and next time she will ask the vendor about installs near peak, so the company has better data to work from.

For the activity below, you need two decisions of your own, one that turned out badly and one that turned out well. Choose ones where you still remember what you knew at the time.

Write down one past decision that turned out badly and one that turned out well. For each, note what you knew, what you could have known and how you reasoned, then grade the process from A to E, separately from the result.

Course

Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).