Pre-mortems and the Day After Game

The "pre-mortem" is a technique proposed by Gary Klein and promoted by Daniel Kahneman to legitimize dissent and identify previously unconsidered risks in projects or plans. Kahneman is a psychologist who is essentially the father of behavioral economics, and who won the Nobel Prize in Economics in 2002 for developing prospect theory with Amos Tversky, so when he gets interested in a decision-making tool it's a good sign that there is some real value to it. In an interview, Kahneman described it like this:

'I learnt a great idea from a friend of mine, Gary Klein. He recommends what he calls a 'pre-mortem'. It's a very clever idea. You get a group of people who have made a plan, it's not completely final, but they have a broad plan. And then you bring them together for a special session, 45 minutes is usually enough. You tell them, take a sheet of paper and now imagine the following: a year has passed, we have implemented the plan and it is a disaster.

'Now, write down on that sheet of paper what happened. Why did the plan turn out a disaster?

'It's a brilliant idea. Because you have a group of people who are now encouraged to think of difficulties and problems with the plan. And that solves the problem of legitimising dissent very elegantly, and it's easy to implement.

Simple and elegant. So much so that there isn't much written about it online. How much more is there to say? Kahneman himself discusses the pre-mortem in this video interview (the whole discussion is worthwhile, but the pre-mortem section starts with about 12:38 remaining). As one commenter paraphrased part of Kahneman's argument:

In organizations where the members are competitive, you expect people to think quite hard about the flaws in the idea and what could go wrong. In a room of twenty people you might expect three or four new ideas that can be used to readjust and improve the proposed plan of action.

The whole concept of the pre-mortem reminds me of a type of game that RAND developed called the Day After Game. This is a much more structured exercise than a pre-mortem, but a pre-mortem seems limited to considering specific plans that are already well-developed, while the Day After Game can be applied to more general problems or issues. The Day After Game is divided into three periods. In the first period, participants are confronted with an unfolding crisis and asked for their response. In the second, the "day after" (hence the name of the technique) the crisis hits is explored, with the consequences mapped out and the effectiveness (or more likely ineffectiveness) of the measures proposed in the first period considered. But it's the third phase that really separates the Day After Game from other crisis exercises, because then participants are asked to step back to before the crisis event and consider what could have been done between the present day and the hypothetical future date of the crisis in the first period to improve the likelihood of mounting an effective response. By considering the severity of the day after, it becomes a mechanism for focusing attention on the "day before." In this respect, the Day After Game is a means of generating "prospective hindsight" in a similar fashion as the pre-mortem exercise.

This methodology was originally developed as a means of surveying the Washington policy community by running a series of games with many repetitions, in an attempt to identify the major schools of thought about policy alternatives that would likely dominate discussion in Washington in the near term. Over time it seems to have evolved into something more focused on generating "prospective hindsight." The RAND report on the first use of this technique (examining nuclear proliferation) is available for free in .pdf format in three volumes. One other Day After Game report (on cybersecurity) is publicly available as well. There is a good description of the methodology followed in each report. RAND has apparently used the technique in a number of other contexts, as well.

I'm a little surprised that this type of game hasn't made many inroads (to my knowledge) into academic or classroom settings. It's a simple enough concept. There isn't any strategic interaction, since Day After Games are run with all the participants responding as a group to the scenario, but that's part of what makes it easier to execute.