You will be able to run a short review that turns past entries into lessons.
Many people start a decision journal. Fewer go back and reread it. A decision journal that is written but never reviewed is a diary: a record of what you thought at the time, and it will not change how you decide. The value comes later, when you read past entries whose outcomes are now known and ask what they teach you. That review takes about an hour, four times a year.
Every three months, block an hour and open the decision journal you set up in lesson 8.2, What goes in a decision journal. A quarter suits most working adults. It is long enough for a reasonable number of outcomes to arrive, and short enough that you still remember the context of each decision. If you make very few decisions worth journaling, you can review twice a year instead. Reviewing much less often than twice a year means you forget too much to learn from.
Put the four review dates in your calendar now, as recurring appointments with yourself, and treat them like any other meeting. The review is easy to skip when work gets busy, and busy periods are exactly when decisions most need checking.
At each review, pick out every entry whose review date has passed or whose outcome is now clear. Leave the rest for the next review. Then take each resolved entry in turn and ask three questions.
First, what happened? Write one factual line beside the original entry. Record the outcome, not how you feel about it.
Second, how did it compare with what you expected, and with the probability you gave? Say you wrote "70 percent the new hire is running the outlet within three months" (an example figure). If she was running it, note that the call came true. If she was not, note that too. One prediction on its own tells you little, because a 70 percent call is supposed to miss three times in ten.
Third, judging by what you knew at the time, was the process sound? This question comes from lesson 8.1, Good decisions can have bad outcomes. Re-read your reasons, the options you considered and your notes on your state of mind. Separate the parts that were reasonable from the parts that were careless, whatever the outcome turned out to be.
Single entries are mostly noise. Learning happens in the patterns across many entries, and those only show up when you read the entries side by side.
The most useful pattern to look for is calibration: how closely the share of predictions that come true matches the probability you gave them. Group your predictions by the probability you stated and count how many came true at each level. If you are well calibrated, things you called at 80 percent happen about 80 percent of the time.
Mei, an invented example, made ten predictions over two quarters that she rated at 80 percent. If she were well calibrated, about eight would have come true. Only five did. When Mei feels 80 percent sure, she is right about half the time, so her "80 percent" should be treated as closer to 50 or 60. This is overconfidence: things called at a given probability happen less often than that probability. The fix is to keep predicting, recognise that your stated probability is too high, and adjust your future estimates.
Calibration needs a fair number of predictions. Ten at the same probability level is a reasonable start. Two or three at a level is not enough, because a couple of lucky or unlucky results can mislead you. Philip Tetlock's forecasting work, covered in lesson 7.4, Update your beliefs with new evidence, relied on large numbers of predictions for this reason.
Other patterns are worth checking at the same time:
whether the biases from Module 5 keep turning up in your reasons whether your project estimates still run long, even after lesson 5.3 whether you keep missing the same kind of risk whether decisions made under time pressure, according to your state-of-mind notes, turn out worse than the others whether one area of your life, such as hiring or spending, goes wrong more often than the rest
A review that produces insight but no change is wasted. End each one by adding exactly one change to the bias checklist you built in lesson 5.4, Build a bias checklist for your decisions, or to the way you make decisions. Keep it to one. Five resolutions made after one review are hard to remember by the next, while one specific change is easier to keep.
Changes other people have made include never making a decision over a certain amount after 9pm, and starting every project estimate with the outside view, meaning how long similar projects have actually taken. After her calibration review, Mei's change was this: before she writes 80 percent or higher for any prediction, she must first write one reason it might not happen.
Keep a short running list of these changes at the front of the journal. After a year you will have four, and you can check whether your calibration and your outcomes have moved since you made them.
Mei's quarterly review shows how the hour divides. She spends 10 minutes picking out the resolved entries, eight this quarter. She spends 30 minutes answering the three questions for each one, 10 minutes counting predictions by probability and looking for patterns, and 10 minutes deciding her one change and adding it to her checklist. She finishes in an hour, having learned something specific about how she decides.
The hardest part is getting the first review onto your calendar before the quarter fills up. Open your calendar now and book the four review dates as recurring appointments.
Pick one past journal entry whose outcome you now know, answer the three review questions for it, then add one change to your bias checklist. Book four quarterly review dates in your calendar.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).