The Simulation Heuristic

I finished another RAND monograph recently (Implications of Modern Decision Science for Military Decision-Support Systems), which is too long and too dense and too useful to sum up in a single post. It's available for free online, but I wound up buying a copy from Amazon after realizing how relevant it was for me (Amazon was cheaper than buying it through RAND this time).

The first section deals with the history and development of decision science. It's a great, concise summary of the psychological aspects of decision-making theory, including a description of the split between what the authors refer to as the Heuristics and Biases Paradigm and the Naturalistic School of decision science. More on this in the future, but something that struck me was the description of what some call the simulation heuristic, and which the monograph refers to as "imaginability" (as a subset of memory biases) or the "availability heuristic". The basic idea is that people see things as more likely if they can visualize them more easily. That echoes Robert Levine's criticism (referred to here) that a game could lead participants to believe that the events depicted were probable, as opposed to merely plausible. In the simulation heuristic formulation, it is the very act of making it easier to imagine an event that increases the perception of its probability.

In a sense, that might not be a bad thing, in that one of the benefits to simulation and gaming can be to make a given circumstance seem more real, for the purposes of spurring people to action, or taking a threat seriously, or provoking more thoughtful analysis. But Levine's point stands to a certain extent; what about the other, unsimulated, ungamed possibilities that are accorded less priority because the simulated case is assigned a higher probability than is warranted? The simplest answer I can see right now is that rather than doing less gaming, perhaps the simulation heuristic should in fact lead to doing more gaming, as a tool of exploratory analysis, with an eye towards keeping in mind the vast multitudes of possible futures (more on this sometime soon, I hope). Another answer might just be that gaming is not necessarily by itself a good indicator of the likelihood of a given set of events. That seems like a no-brainer, but perhaps it's difficult to keep that in mind when presented with a realistic game. How do the pros deal with this issue?

More on Crisis Games

To continue the thoughts in the previous post, Crisis Games 27 Years Later contains a number of criticisms of the gaming form by Robert Levine, to which Thomas Schelling and William Jones respond.

Some of Levine's concerns:

Gaming could lend a sense that a plausible scenario is in fact probable. This could take place by virtue of the chosen, simulated scenario becoming more "real" in the minds of the participants than all of the other, unexplored possibilities. (More on cognitive biases like this one in another post soon, once I finish the paper I'm reading now)

A combination of "ersatz history" (in the form of scenarios that depart greatly from present day events) and "ersatz people" (since game players do not share the same characteristics as the individuals they are simulating) introduce biases to the gaming process that compromise its utility in generating worthwhile hypotheses.

The results of games may be used as evidence, even when they have no evidentiary value. Specifically: "If a game result appears in language which can be read as a confirmed policy conclusion, there is a substantial chance that it will be so read."


For a more recent view of some similar issues, Robert Rubel's "The Epistemology of War Gaming" (mentioned in this previous post) talks about a broader category of games, and also has steps in mind to try and address some of these methodological challenges, rather than simply advocating that gaming be abandoned as a research tool.


Rather than enumerate Schelling and Jones' replies to these and other arguments point by point, here are a few examples of their ideas (the paper is well worth reading in its entirety):

Schelling defines games as necessarily involving "at least two separate decision centers... neither of which is privy to the other's planning and arguing, neither of which has complete access to the other's intelligence or background information, neither of which has any direct way of knowing everything that the other is deciding on." This definition excludes many of the exercises we would categorize as games, and includes many that do not appear to be games at all, but this is because Schelling is focused on the strategic interaction between the two (or more) decision centers:

"What this mode of organization can do that can not otherwise be done is to generate the phenomena of understanding and misunderstanding, perception and misperception, bargaining, demonstrations, dares and challenges, accommodation, coercion and intimidation, conveyance of intent, and uncertainty about what each other has already done or decided on. There are some things that just cannot be done by a single person or by a team that works together."

"One is to judge how 'obvious' something is. An analogy is the 'hidden face' in the picture. If I draw a face with a hidden picture, there is no way for me to tell how hard it is to see the face except to show the picture to somebody. I can't not see the picture because I put it there, and the hidden face has the quality that once you've seen it it is awfully hard to recapture your innocence and not see it."

Jones responds to the charge of "ersatz history" thus (keep in mind that he was writing in 1964):

"[P]erfectly plausible, useful, and not improbable game scenarios of a situation three or four years in the future can be had by simply initiating a game using today's newspapers -- interjecting only tomorrow's equipment. Arab-Israeli controversy has existed for some time and it seems probable that it will be in existence in 1970. Turkey and Greece have not seen eye-to-eye on the Cyprus situation for a considerable period and, unfortunately, this controversy is likely to persist.... Notice here that I am not saying that games, played from scenarios which are extrapolations of today's problems, are perfect for determining tomorrow's force posture and weapons characteristics. I am saying that I can think of worse procedures (re-fighting the last war, for example) and I know of no other way in which I have even such a limited amount of confidence."

There is a good deal more in this paper than I can tackle in a blog post, but many of the ideas and concerns expressed resonate today, which is why RAND reprinted it in 1991, 27 years after it was written.

Crisis Games and RAND

In 1964, Robert Levine, Thomas Schelling, and William Jones circulated a series of drafts through the RAND Corporation internal mail system, discussing the strengths and weaknesses of crisis gaming. In 1991, feeling that the issues laid out remained relevant, Levine republished the entire exchange as a RAND paper: Crisis Games 27 Years Later: Plus C'est Deja Vu. RAND has released this paper (along with some others from their 50 years of research and policy memoranda) for free in pdf form. It's well worth reading in its entirety. Thomas Schelling won the Nobel Prize in economics in 2005, and his defense of crisis gaming as a technique is fascinating.

RAND did a lot of pioneering work in crisis gaming, which seems to have come on the scene during the Cold War as a way of examining the issues surrounding crises between the US and the USSR (I will confirm this at some point in the future, but the earliest references I have seen so far to crisis gaming date to the 1950's). The study of crisis was a priority, given the risks of nuclear war that attended a misstep. The various Berlin crises, the Cuban Missile Crisis, and the surprise attack that began the Korean War were all influential in shaping the view that crisis management was an important skill to develop. Cold War crises had some very appealing features when it comes to gaming: there were two main adversaries whose interactions (due to their superpower status) were much more important than the roles of any other states (whose actions could be devised by the control team), the stakes in any given confrontation were high and the time frames often short (due to the presence of large nuclear arsenals), which ensured that the highest level decision makers would be involved, superseding normal policymaking structures, and the novelty of the nuclear standoff, with unprecedented destructive power involved, which gave a sense to policy theorists that all bets were off and that this type of crisis event needed to be studied outside the context of previous conventional military confrontations. This last point was also a major argument made by the newly ascendant civilian policy analysts in favor of rethinking conflict in ways opposed by the military establishment.