You will be able to run a short review that turns past entries into lessons.
Plenty of people start a journal of some kind. Fewer read it again. A decision journal that is only written and never reviewed is a diary: pleasant, maybe, but it will not change how you decide. The value comes when you sit down with past entries whose outcomes are now known and ask what they teach you.
This lesson sets up that review. It takes about an hour, four times a year.
Every three months, block an hour and open your journal from lesson 8.2, What goes in a decision journal. Pick out every entry whose review date has passed or whose outcome is now clear. Leave the rest for next time.
A quarter is a useful rhythm for most working adults. It is long enough for a reasonable number of outcomes to come in, and short enough that you still remember the context around each decision. If you make very few decisions worth journalling, twice a year can work. Much less often than that and you will have forgotten too much to learn from.
Put the four dates in your calendar now, as recurring appointments with yourself, and treat them like any other meeting. The review is the first thing to be skipped when work gets busy, which is exactly when your decisions most need checking.
For each resolved entry, ask three questions.
First, what happened? Write it in a line next to the original entry. Keep it factual: the outcome, not your feelings about it.
Second, how did it compare with what you expected, and with your stated probability? If you wrote "70 percent the new hire is running the outlet within three months" and she was, note that the 70 percent call came true. If she was not, note that too. One prediction tells you little on its own. A 70 percent call is supposed to miss three times in ten.
Third, judging by what you knew at the time, was the process sound? This is the question from lesson 8.1, Good decisions can have bad outcomes. Re-read your reasons, the options you considered and your state of mind. Separate the parts that were reasonable from the parts that were careless, whatever the outcome was.
Single entries are mostly noise. Patterns across many entries are where you learn something, and they only become visible when you read entries side by side.
The most useful pattern to look for is calibration. Group your predictions by the probability you gave them, and count how many came true. If you are well calibrated, things you called at 80 percent should happen about 80 percent of the time.
Here is a made-up example. Over two quarters, Mei made ten predictions that she rated at 80 percent. If she were well calibrated, about eight would have come true. Only five did. That is a sign of overconfidence: when Mei feels 80 percent sure, she is right about half the time. The fix is not to stop predicting. It is to notice that her "80 percent" should really be closer to 50 or 60, and adjust in future.
You need a fair number of predictions before calibration means much. Ten at the same level is a reasonable start. Two or three is not enough, and with so few, a couple of lucky or unlucky results can mislead you. Philip Tetlock's work on forecasting, from lesson 7.4, Update your beliefs with new evidence, relied on large numbers of predictions for exactly this reason.
Other patterns to look for are the biases from Module 5 showing up again and again. Do your project estimates keep running long, even after lesson 5.3? Do you keep missing the same kind of risk? Are decisions made under time pressure, according to your state of mind notes, worse than the others? Do decisions in one area of your life, such as hiring or spending, go wrong more than others?
A review that ends in insight but no change is wasted. Finish each one by adding exactly one change to your bias checklist from lesson 5.4, Build a bias checklist for your decisions, or to how you make decisions.
One change, not five. Five resolutions after one review tend to be forgotten by the next. One change, made specific, tends to stick. Mei's change after her calibration review is simple: whenever she is about to write 80 percent or higher, she has to write one reason it might not happen first. Another person might add a rule that no decision over a certain amount gets made after 9pm, or that every project estimate starts with the outside view.
Keep a short running list of the changes at the front of your journal. After a year, you will have four, and you will be able to see whether your calibration and outcomes have moved since you made them.
Mei's review in practice goes like this. She spends ten minutes picking out the resolved entries, eight this quarter. She spends thirty minutes on the three questions for each. She spends ten minutes counting her predictions by probability and looking for patterns, and ten minutes deciding on her one change and adding it to her checklist. Done in an hour, and she has learned something specific about how she decides.
The hardest part is getting the first review onto your calendar before the quarter fills up, so the activity below starts there.
Book four quarterly review dates and write the three questions you will ask of each journal entry.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).