Pirates, yellowcake, and the press at Patterson

Rob Farley has a post up about the recent crisis game at the Patterson School of Diplomacy and International Commerce at the University of Kentucky. He links to the in-game press reports about the action, which give a pretty good idea of how things worked (posts are listed in blog-style reverse chronological order, so start at the bottom of the page). What's most interesting to me about this setup is that they brought in journalism students to play the world media and produce their press reports within the game, interviewing the participants as things moved along. It's an interesting concept. The media are frequently represented in this sort of exercise, but in the examples I've seen (particularly Simulex at the Fletcher School, which I will have to describe in more detail sometime), the "media" is a tool of the control team, used for distributing information (and disinformation) about developments in the scenario to the participants, rarely if ever producing reporting ON the participants. Bringing in journalism students to do so seems like a great way to kill two birds with one stone. The other participants get a taste of operating in a global media environment, while the journalists have an international crisis staged for them to practice reporting on. And as a bonus, it resulted in an easily readable account of the weekend. This is not something that would work for every crisis game, perhaps not even many of them, but it's a nice feature for this one.

Pre-mortems and the Day After Game

The "pre-mortem" is a technique proposed by Gary Klein and promoted by Daniel Kahneman to legitimize dissent and identify previously unconsidered risks in projects or plans. Kahneman is a psychologist who is essentially the father of behavioral economics, and who won the Nobel Prize in Economics in 2002 for developing prospect theory with Amos Tversky, so when he gets interested in a decision-making tool it's a good sign that there is some real value to it. In an interview, Kahneman described it like this:

'I learnt a great idea from a friend of mine, Gary Klein. He recommends what he calls a 'pre-mortem'. It's a very clever idea. You get a group of people who have made a plan, it's not completely final, but they have a broad plan. And then you bring them together for a special session, 45 minutes is usually enough. You tell them, take a sheet of paper and now imagine the following: a year has passed, we have implemented the plan and it is a disaster.

'Now, write down on that sheet of paper what happened. Why did the plan turn out a disaster?

'It's a brilliant idea. Because you have a group of people who are now encouraged to think of difficulties and problems with the plan. And that solves the problem of legitimising dissent very elegantly, and it's easy to implement.

Simple and elegant. So much so that there isn't much written about it online. How much more is there to say? Kahneman himself discusses the pre-mortem in this video interview (the whole discussion is worthwhile, but the pre-mortem section starts with about 12:38 remaining). As one commenter paraphrased part of Kahneman's argument:

In organizations where the members are competitive, you expect people to think quite hard about the flaws in the idea and what could go wrong. In a room of twenty people you might expect three or four new ideas that can be used to readjust and improve the proposed plan of action.

The whole concept of the pre-mortem reminds me of a type of game that RAND developed called the Day After Game. This is a much more structured exercise than a pre-mortem, but a pre-mortem seems limited to considering specific plans that are already well-developed, while the Day After Game can be applied to more general problems or issues. The Day After Game is divided into three periods. In the first period, participants are confronted with an unfolding crisis and asked for their response. In the second, the "day after" (hence the name of the technique) the crisis hits is explored, with the consequences mapped out and the effectiveness (or more likely ineffectiveness) of the measures proposed in the first period considered. But it's the third phase that really separates the Day After Game from other crisis exercises, because then participants are asked to step back to before the crisis event and consider what could have been done between the present day and the hypothetical future date of the crisis in the first period to improve the likelihood of mounting an effective response. By considering the severity of the day after, it becomes a mechanism for focusing attention on the "day before." In this respect, the Day After Game is a means of generating "prospective hindsight" in a similar fashion as the pre-mortem exercise.

This methodology was originally developed as a means of surveying the Washington policy community by running a series of games with many repetitions, in an attempt to identify the major schools of thought about policy alternatives that would likely dominate discussion in Washington in the near term. Over time it seems to have evolved into something more focused on generating "prospective hindsight." The RAND report on the first use of this technique (examining nuclear proliferation) is available for free in .pdf format in three volumes. One other Day After Game report (on cybersecurity) is publicly available as well. There is a good description of the methodology followed in each report. RAND has apparently used the technique in a number of other contexts, as well.

I'm a little surprised that this type of game hasn't made many inroads (to my knowledge) into academic or classroom settings. It's a simple enough concept. There isn't any strategic interaction, since Day After Games are run with all the participants responding as a group to the scenario, but that's part of what makes it easier to execute.

Nuclear War: Iran vs. Israel

This report from the Center for Strategic and International Studies is not strictly speaking a game report, but rather "the material used in a scenario analysis and interactive game that looks at some of the consequences of a future nuclear exchange between Israel and Iran, and the possible impact of its expansion to cover targets in Syria, Egypt, and the Gulf." Unfortunately, I can't find any details on the "interactive game," but the background materials are comprehensive and fascinating (if you're into nuclear weapons). The results from their analysis are presented as a quote from the movie Wargames: the only way to win is not to play. If anyone knows where I could find details about the "interactive game" they conducted, I would love to learn more about it.

Cyber Storm

In September 2006, the Department of Homeland Security ran an exercise named Cyber Storm to consider the U.S. response to a significant cyber attack. A wide variety of public and private sector agencies and entities were involved, leading to a better understanding of how the patchwork of responses by these groups could be better coordinated in the event of such an attack. The game report is available here.

Apparently, the game referees had to stop (overzealous?) participants from trying to hack the system the exercise was being run on:

In the middle of the war game, someone quietly attacked the very computers used to conduct the exercise. Perplexed organizers traced the incident to overzealous players and sent everyone an urgent e-mail marked "IMPORTANT!" reminding them not to probe or attack the game computers.

"Any time you get a group of (information technology) experts together, there's always a desire, 'Let's show them what we can do,'" said George Foresman, a former senior Homeland Security official who oversaw Cyber Storm. "Whether its intent was embarrassment or a prank, we had to temper the enthusiasm of the players."

The comments on this page have an interesting discussion of this incident, with some back and forth about whether this type of action should be allowable in the context of a game/exercise. A wargame called Millennium Challenge 2002 is referenced early in the discussion. Millennium Challenge '02 was a large-scale wargame conducted in 2002, pitting the U.S. against an unnamed Middle Eastern military. It achieved an unusual degree of notoriety for a wargame because the commander of the "Red" forces used several unconventional tactics to exploit weaknesses, which resulted in massive damage to the "Blue" fleet as it entered the Persian Gulf. The exercise was halted, and the Blue losses were "re-floated," causing some to cry foul. The Red commander himself said that the game was "fixed." The game has been in the news again lately after Iranian speedboats approaching U.S. Navy ships in the Persian Gulf recalled the tactics used in the game to devastate the Blue fleet (though the success of the tactic in the wargame was apparently predicated on a massive number of speedboats, cruise missiles, and other attack vectors making a simultaneous assault to overwhelm the capacities of the warships to track them and respond).

This comment (in the aforementioned discussion of the Cyber Storm exercise) in particular seems to get right at the issue:

My point is just that a particular wargame has a purpose. It's usually not run to find out who's the best or cleverest solider/commander/unit/force, even if that's what some of the participants want it to be. If the real purpose is damaged by people trying to figure out how to change the intended parameters of the game in order to "win", then players shouldn't be doing that.

In particular, I'd say that you shouldn't be trying to exploit the limits of the simulation. Hypothetically, suppose that you're supposed to be learning (among other things) how to deal with poor communications, so your radios have been jiggered to make them unreliable, or else the enemy can listen in, or something. I have no idea whether that's a plausible wargame, but just suppose.

Now, suppose you decide to adapt to your comms problems by using couriers. Fair enough, you'd think, but if the people designing the game didn't think of that, then their wargame might well not account for snipers either. Then all you've achieved, other than "winning", is to show that couriers are great if your opponent can't do anything about them.

That doesn't prepare you for a real war - obviously modern forces do have snipers, and your couriers would have a great deal more difficulty operating in a warzone than they did in the simulation. You've made the scenario be about couriers and snipers, when it was designed to be about something else (strategies that are robust against broken communications, maybe).

I agree that couriers should be considered in future planning, but if the consideration is, "they wouldn't last five minutes out there", then there's not much point allowing them in the simulation.

Of course for the Millennium Wargame, one accusation was that the envisaged scenario was a sweeping Blue victory no matter what Red did, with no intention to discover anything about real war. But such a "politically motivated" ruling, if that's what it was, doesn't detract from the fact that in general, wargames might have a reasonable purpose, and might need to use "unrealistic" restrictions to achieve that purpose.

I had initially intended for this post to go further into the whys and wherefores of things like "refloating" in the midst of a wargame, but so much has been written about Millennium Challenge 2002 over the years that I haven't finished going through it, much less finished thinking about the issues it presents. There have been a few blog posts this year that have been especially good at identifying the underlying issues. Yet another thing I'll have to come back to in a future post....

The Global War Game at the Naval War College

From 1979-2001, the Naval War College sponsored an annual exercise they called the Global War Game. I've already mentioned John Hanley's excellent summaries of the 1979-1990 games in his dissertation, but there are some other, more easily accessible descriptions of these games as well. Bud Hay and Bob Gile put together a review of the first five years of the game, and Gile returned to the subject later to write up the second five years. Both summaries were published in the Newport Papers series at the Naval War College, but only the second appears to be available online, here. The second of these provides more information about the way the games were structured, and includes a brief summary section on the first five years, so I would recommend starting with that one anyway. Both of these papers are available at some university libraries.

Beyond these summaries of the early games, few articles or other accounts have been published dealing with the Global War Games, particularly the later games. One of the few I have seen so far is Kenneth Watman's article on Global 2000. Watman was the head of war gaming at the Naval War College.

This post from an arms control blog last year is an indication that potentially useful insights might still be available from these games, which is why it is fascinating to me that no one has written a systematic review of the later years. The review of the second five years, mentioned above, was published in 2004, which suggested a new interest in studying the games, but nothing more has been forthcoming.

Hanley: On Wargaming

John Hanley's 1991 poli sci dissertation at Yale (On Wargaming: A Critique of Strategic Operational Gaming) is one of the best single sources I have found.

Unfortunately, this dissertation is not available online. It might be difficult to get a hold of via interlibrary loan, as well. I had to order it from the University of Michigan's dissertation express site, and have it printed from microfilm and shipped to me. But for me it was worth it, and I highly recommend this dissertation to anyone serious about studying pol-mil gaming.

Particularly helpful are Hanley's chapters analyzing the Naval War College's Global War Game from its debut in 1979 through 1990. I'll write some more about the Global War Game soon. Hanley provides a lot of information about how the games were organized and structured, what the critiques at the time were, and how the objectives of the games changed over time. This is in contrast to most publicly available game reports, which often focus on substantive results rather than the methods employed during the game. This can be a major frustration for those of us who would like to learn more about the way these tools are employed. There is much, much more to this dissertation, which will hopefully show up from time to time as I go forward with this blog.

The only journal article to cite Hanley's dissertation, as far as I can tell, was Robert Rubel's piece I discussed here. To date, it appears that no one has cited the Rubel article in a journal. In part, both Hanley's dissertation and Rubel's article aimed at starting a certain type of conversation about epistemology and methodology within the gaming community. That doesn't seem to have happened yet, at least not in magazines and journals. Why is that? I have some thoughts, but I'll save them for another time.

Lempert and Schwabe - Transition to Sustainable Waste Management

Another RAND monograph, but this one deals with an application of gaming techniques to a domestic public policy question (available in pdf form here). The goal was to investigate what factors might turn out to be important in the shift from a product packaging industry that primarily used "virgin" materials and generated large amounts of solid waste to a system based on recycling of packaging material. Participants from government and industry were assigned roles within the context of a model, and were asked to make decisions on investment or regulation related to packaging. Based on these decisions, the model then worked out the market clearing prices and quantities for each type of product or service, established profits and losses for the simulated activities, and presented the results to the participants for another round of regulation and investment.

This paper provides a detailed look at the way a model can provide the structure for a game, as well as showing the value added that gaming out a model can produce.

CIA "red team" exercise on Iraq

Here's an article about a (relatively) recent Red Team exercise conducted by the Counterterrorism Center (CTC) at the CIA. Red Teaming is when analysts/experts/whoever gather in a structured way to consider weaknesses and vulnerabilities of their own side that could be exploited by current or potential enemies. In this case, the exercise was meant to shed light on what the reactions of Al Qaeda and other terrorist actors would be to a U.S. withdrawal from Iraq. If there was a "Blue Team" at all, it was probably directly managed by the control cell, since the focus of the exercise was on considering "Red" actions. I'll write more about Red Teaming in the future, but this is an interesting application of the technique.

Desert Crossing

In 1999, Marine Gen. Anthony Zinni commissioned Booz Allen Hamilton (a government contractor with a large gaming shop) to produce a wargame (titled Desert Crossing) to examine the issues that would surround US intervention in Iraq. The after-action report and some related documents were declassified late last year, and much of it is available through GWU's National Security Archive website. Prior to declassification, Gen. Zinni mentioned the game in several public contexts, and Tom Ricks included a brief description of the game in his book Fiasco.

There were a number of mentions of Desert Crossing in the press when the documents became publicly available, as well as some attention from the blogosphere. Many of these comments were directed specifically at the number of troops the game participants assumed would be necessary to occupy Iraq, in contrast to the number that were actually used four years later. But the game is much more interesting than a simple question of troop numbers, and the troop number issue is a bit of a red herring compared to the broad insights that emerged from the game. The game was run to identify problem areas for future consideration, which it did. Despite the fact that the scenario was largely directed at internal Iraqi events that might require/precipitate an American intervention (rather than dealing with the sort of invasion action actually undertaken by the US), the interagency group of participants produced a list of challenges the US would face that looks fairly prescient, given current events. Unfortunately, the exercise itself never made much of an impact. In 2003, when the then-retired Gen. Zinni called CENTCOM to recommend that they look at Desert Crossing again, they didn't know what he was talking about.

The point is not that the game predicted anything. Games don't predict very well, and generally should not be used for that purpose. What Desert Crossing did was bring together the right group of people in the right setting with the right procedural setup to allow them to thoroughly examine the right questions. But Desert Crossing is also an example of the difficulty of assimilating the results of such a game into an organizational context.