Perception and misperception in gaming

Jon Alterman has written a short and thought-provoking piece for a CSIS newsletter about his experience on the U.S. team in a recent multi-cell policy game. Worth reading in full, but here's an excerpt:

The U.S. team swiftly leapt into a series of actions intended to direct the actions of its allies and blunt the efforts of its foes. In the second move, things got worse, and the U.S. side tried even harder to marshal its forces, artfully deploying its military and diplomatic assets. By the third move, the situation continued to worsen in many respects, but the U.S. team saw light at the end of the tunnel. We had a plan, and our allies were looking to us for leadership. Equally importantly, they were all acting precisely as we had hoped, abandoning the troublesome sorts of freelancing that had marked their earlier moves. We thought we had played the game well.

When we all gathered after the final move, however, it was clear how much we had misjudged the situation. Opponents talked about how easily U.S. moves were blunted or ignored. Allies were beside themselves that the United States had missed numerous opportunities to consult with them and raised tensions needlessly. But most importantly, they charged that the U.S. team had fundamentally misjudged the motivations of their actions. The U.S. team had congratulated itself on its ability to integrate all of the instruments of national power—in contrast to allies that could either convene summits or issue statements or host American military forces, but rarely more than one of those and almost certainly not in a sustained, purposive and coordinated way. Yet allies explained that perceptions of their own national interests drove their decisions, and that U.S. actions rarely shaped those decisions. It is true, the U.S. team had moved military aircraft and issued statements to and fro, but the other players did not find it very impressive. They had their own calculus. In their telling, it was as if the U.S. team was trying to take credit for the sun rising in the East. If anything, they said, the U.S. team’s actions had made it harder for them to comply with U.S. wishes.

There's a lot here. First, this is the sort of discussion that is missing in many public depictions of gaming events. Alterman was constrained by the event sponsor's desire to keep details of the scenario private, but the generalized insight he describes is probably of at least as much value to him and the other participants as any specifics of the game outcome.

My sense is that games are a largely unexploited avenue to bring participants to a better understanding of their own perceptions and misperceptions. In Alterman's example, the stark contrast between the U.S. team's sense of agency and the view of other actors will likely stick with him to a greater extent than hearing someone lecture about the pitfalls of psychological decision-making biases. Thomas Schelling viewed the RAND crisis games of the 50's and 60's as a tool uniquely suited to examining perceptual factors (discussed briefly here), but I'm not aware of much work since then on the subject, nor have I seen much application of social psychology to gaming, with a couple of exceptions that I should blog about sometime.

The role of the debrief session is key in all of this, as it was for Alterman's experience. Unfortunately, it's much easier to say how important debriefing is than it is to concisely define what it is about a debrief that makes it effective. How would you set up a game and the subsequent debrief to address perception and decision-making biases directly? Multiple cells, for starters, would be a key factor, as Schelling would have said, with closed information conditions. The debrief would have to be substantial enough to give each cell the opportunity to understand what was going on in each of the other cells, and the debriefer would need to be focused on drawing out the differing perceptions of the participants. Beyond that, I'm really not sure.

Schwabe: An Introduction to Analytic Gaming

William Schwabe wrote an entry on gaming for the Encyclopedia of Operations Research and Management Science in 1994 that was also released as a RAND paper. Unfortunately, the entry/paper is not available online at this time. It's only 2 or 3 pages in the encyclopedia, and the RAND paper is only 7 pages long (double spaced), but it does a good job of laying out the basics of gaming as a policy research tool. It's a bit short to quote at length, but here are several points:

Few methods have been so inadequately named, prompting ridicule from skeptics and attempts by adherents to call it something more serious sounding or descriptive, such as "operational gaming," "simulation gaming," "free-form gaming," and, in defense analysis, "war gaming" and "political-military gaming."
....

Unlike many other techniques of analysis, gaming is not a solution method. The output of a game is not a forecast, solution, or rigorous validation. The output of a good game is increased understanding.
....

Gaming has often not been as well integrated into studies using other methodologies as might be warranted. Gaming is but one form of analysis to inform policy, managerial, or operational decisions.

In addition, Schwabe quotes Clark Abt's 1970 book, Serious Games, in his definition of a game in his first paragraph. It's a book I'll have to pick up at some point (and apparently it was just reprinted in paperback in 2002), but for now:

Clark Abt (1970) defines a game as "an activity among two or more independent decision-makers seeking to achieve their objectives in some limiting context."

Note the importance of multiple decision centers, just as Schelling noted (as mentioned previously here). Schwabe quoted some of the same portions of Schelling's section in Crisis Games 27 Years Later that I did in his description of what games can produce that other research methods cannot.

For a very basic look at gaming as a research tool, this is a remarkably successful 7 pages. I have the RAND report, not the encyclopedia entry, but I believe they are about the same. It wouldn't be worth the $18 RAND is charging for the paper, though, and it shows the importance of checking the page count before purchasing thinktank reports online. I got mine through interlibrary loan, and I expect that the encyclopedia is widely available at university libraries.

More on Crisis Games

To continue the thoughts in the previous post, Crisis Games 27 Years Later contains a number of criticisms of the gaming form by Robert Levine, to which Thomas Schelling and William Jones respond.

Some of Levine's concerns:

Gaming could lend a sense that a plausible scenario is in fact probable. This could take place by virtue of the chosen, simulated scenario becoming more "real" in the minds of the participants than all of the other, unexplored possibilities. (More on cognitive biases like this one in another post soon, once I finish the paper I'm reading now)

A combination of "ersatz history" (in the form of scenarios that depart greatly from present day events) and "ersatz people" (since game players do not share the same characteristics as the individuals they are simulating) introduce biases to the gaming process that compromise its utility in generating worthwhile hypotheses.

The results of games may be used as evidence, even when they have no evidentiary value. Specifically: "If a game result appears in language which can be read as a confirmed policy conclusion, there is a substantial chance that it will be so read."


For a more recent view of some similar issues, Robert Rubel's "The Epistemology of War Gaming" (mentioned in this previous post) talks about a broader category of games, and also has steps in mind to try and address some of these methodological challenges, rather than simply advocating that gaming be abandoned as a research tool.


Rather than enumerate Schelling and Jones' replies to these and other arguments point by point, here are a few examples of their ideas (the paper is well worth reading in its entirety):

Schelling defines games as necessarily involving "at least two separate decision centers... neither of which is privy to the other's planning and arguing, neither of which has complete access to the other's intelligence or background information, neither of which has any direct way of knowing everything that the other is deciding on." This definition excludes many of the exercises we would categorize as games, and includes many that do not appear to be games at all, but this is because Schelling is focused on the strategic interaction between the two (or more) decision centers:

"What this mode of organization can do that can not otherwise be done is to generate the phenomena of understanding and misunderstanding, perception and misperception, bargaining, demonstrations, dares and challenges, accommodation, coercion and intimidation, conveyance of intent, and uncertainty about what each other has already done or decided on. There are some things that just cannot be done by a single person or by a team that works together."

"One is to judge how 'obvious' something is. An analogy is the 'hidden face' in the picture. If I draw a face with a hidden picture, there is no way for me to tell how hard it is to see the face except to show the picture to somebody. I can't not see the picture because I put it there, and the hidden face has the quality that once you've seen it it is awfully hard to recapture your innocence and not see it."

Jones responds to the charge of "ersatz history" thus (keep in mind that he was writing in 1964):

"[P]erfectly plausible, useful, and not improbable game scenarios of a situation three or four years in the future can be had by simply initiating a game using today's newspapers -- interjecting only tomorrow's equipment. Arab-Israeli controversy has existed for some time and it seems probable that it will be in existence in 1970. Turkey and Greece have not seen eye-to-eye on the Cyprus situation for a considerable period and, unfortunately, this controversy is likely to persist.... Notice here that I am not saying that games, played from scenarios which are extrapolations of today's problems, are perfect for determining tomorrow's force posture and weapons characteristics. I am saying that I can think of worse procedures (re-fighting the last war, for example) and I know of no other way in which I have even such a limited amount of confidence."

There is a good deal more in this paper than I can tackle in a blog post, but many of the ideas and concerns expressed resonate today, which is why RAND reprinted it in 1991, 27 years after it was written.

Crisis Games and RAND

In 1964, Robert Levine, Thomas Schelling, and William Jones circulated a series of drafts through the RAND Corporation internal mail system, discussing the strengths and weaknesses of crisis gaming. In 1991, feeling that the issues laid out remained relevant, Levine republished the entire exchange as a RAND paper: Crisis Games 27 Years Later: Plus C'est Deja Vu. RAND has released this paper (along with some others from their 50 years of research and policy memoranda) for free in pdf form. It's well worth reading in its entirety. Thomas Schelling won the Nobel Prize in economics in 2005, and his defense of crisis gaming as a technique is fascinating.

RAND did a lot of pioneering work in crisis gaming, which seems to have come on the scene during the Cold War as a way of examining the issues surrounding crises between the US and the USSR (I will confirm this at some point in the future, but the earliest references I have seen so far to crisis gaming date to the 1950's). The study of crisis was a priority, given the risks of nuclear war that attended a misstep. The various Berlin crises, the Cuban Missile Crisis, and the surprise attack that began the Korean War were all influential in shaping the view that crisis management was an important skill to develop. Cold War crises had some very appealing features when it comes to gaming: there were two main adversaries whose interactions (due to their superpower status) were much more important than the roles of any other states (whose actions could be devised by the control team), the stakes in any given confrontation were high and the time frames often short (due to the presence of large nuclear arsenals), which ensured that the highest level decision makers would be involved, superseding normal policymaking structures, and the novelty of the nuclear standoff, with unprecedented destructive power involved, which gave a sense to policy theorists that all bets were off and that this type of crisis event needed to be studied outside the context of previous conventional military confrontations. This last point was also a major argument made by the newly ascendant civilian policy analysts in favor of rethinking conflict in ways opposed by the military establishment.