Hanley: On Wargaming

John Hanley's 1991 poli sci dissertation at Yale (On Wargaming: A Critique of Strategic Operational Gaming) is one of the best single sources I have found.

Unfortunately, this dissertation is not available online. It might be difficult to get a hold of via interlibrary loan, as well. I had to order it from the University of Michigan's dissertation express site, and have it printed from microfilm and shipped to me. But for me it was worth it, and I highly recommend this dissertation to anyone serious about studying pol-mil gaming.

Particularly helpful are Hanley's chapters analyzing the Naval War College's Global War Game from its debut in 1979 through 1990. I'll write some more about the Global War Game soon. Hanley provides a lot of information about how the games were organized and structured, what the critiques at the time were, and how the objectives of the games changed over time. This is in contrast to most publicly available game reports, which often focus on substantive results rather than the methods employed during the game. This can be a major frustration for those of us who would like to learn more about the way these tools are employed. There is much, much more to this dissertation, which will hopefully show up from time to time as I go forward with this blog.

The only journal article to cite Hanley's dissertation, as far as I can tell, was Robert Rubel's piece I discussed here. To date, it appears that no one has cited the Rubel article in a journal. In part, both Hanley's dissertation and Rubel's article aimed at starting a certain type of conversation about epistemology and methodology within the gaming community. That doesn't seem to have happened yet, at least not in magazines and journals. Why is that? I have some thoughts, but I'll save them for another time.

Orbis Forum on Political and Military Gaming

In 1984, the journal Orbis published a series of articles on the state of political-military gaming. The exchange is unfortunately not available for free online, but it appeared in the winter 1984 issue, which is probably widely available at university libraries. As a group, these articles are an excellent source of reflections on gaming by some prominent practitioners and theorists.

This is a long post, and it covers four separate articles. If I had known it would take me so long to get around to finishing it, I would have split it up. Lesson learned.

Lincoln Bloomfield was a pioneer in political-military gaming, particularly in his work organizing research-oriented political exercises (referred to as POLEXes) for senior-level decisionmakers at MIT in the late 1950's and throughout the 1960's. All of the POLEX research results are apparently unclassified, but I can't find any reference to them on the web. In his Orbis contribution, Bloomfield discusses the history, practice, and realism of political gaming, based on his own experience. His explanation of how free-form gaming requires experienced participants in order to generate useful policy results is particularly well laid out:

No satisfactory model yet exists of the national security decisionmaking system of the United States (or any other country), nor of the larger system of interactions and perceptions that connects them. Thus one either specifies a crude, oversimplified model or relies on the complex model inside the heads of experienced professionals. How successfully a simulation emulates reality depends on the extent of the players' knowledge of the structures, routines, and probable responses of decisionmakers. Great pedagogic, but little policy value results from putting inexperienced individuals in the shoes of decisionmakers.

Even today the national security decisionmaking process is not sufficiently understood as to negate the value of free-form gaming. I would suggest that it never will be. (More on this in a future post.) There has been some debate over issues related to the last part of Bloomfield's remark above, and in the 50's and 60's several researchers (Guetzkow and Hermann are the two I can think of offhand) used inexperienced participants in games to study various aspects of crisis behavior, then tried to generalize conclusions based on a number of runs of the same game. Whether or not those efforts are considered to have produced valid results, exercises designed as experiments in that way bore little resemblance to the high-level games Bloomfield is basing his conclusion on. It is reasonable to agree that given Bloomfield's type of political game, he is correct in asserting that inexperienced participants who do not have carefully calibrated internal models of the world situation would not produce particularly meaningful policy-oriented results.

Among the points Bloomfield makes that have a special relevance to the epistemology of gaming is this:

Games do not predict future events or policy outcomes and can be misleading for specific contingency planning purposes. But they can indicate in detail how a future situation might develop and, even more important, why. A little-noticed fact is that in such games he controlling prediction has already been made: the situation the players face is not of their making, but rather is decided by the game designer. Both MIT and government games relied on scenarios prepared by experts asked to depict a specified future situation in a way that would be accepted by other experts as plausible. A prediction was thereby made that partially determined the game results. Thus, such POLEXes have been biased by game designers, whether for experimental, bureaucratic, or merely frivolous reasons.

These remarks remind me of some of H.A. DeWeerd's concerns about the contextual basis for scenarios, previously discussed here.

Although Bloomfield argues that "In a properly designed POLEX, a knowledgeable non-national can often approximate the policy options likely to be considered by another country's decisionmakers," he also notes that:

Given our national tendency to ascribe Western "rational" mindsets to others, crossing cultural barriers to investigate probable foreign reactions is a valuable and neglected use of gaming. A comparable value derives from putting military officers in the shoes of civilians and vice versa. (A survey of several hundred officials who participated in MIT games reported "role-exchange" as the most useful aspect of the games.)

I suppose that the idea here is that worthwhile results can be generated by non-nationals playing a country's decisionmakers, but only if there is an attempt to understand the cultural context in which those decisionmakers would operate. That may be an element, in Bloomfield's thinking, of proper POLEX design.



Paul Bracken (professor of both political science and management at Yale) contributed an article titled "Deterrence, Gaming, and Game Theory." Among other things, this article includes a short discussion of the distinction between gaming and game theory. This is an issue, because there is a tendency to lump the two together, which is part of the problem with gaming terminology. While Bracken's distinction relies on a rather specific definition of "gaming," it illustrates the difference between the two very well:

Gaming and game theory are sometimes confused with each other, because of their similar names. Gaming refers to an exercise in which opposing teams of human players are confronted with a situation or problem and work out responses to the problem and to the moves of the opposing team. The use of human players to simulate specific political or military decisionmakers is the key to gaming. Such role playing gives the game its richness while distinguishing it from a mathematical model or computer simulation. However, models and simulations often are included in political-military games as supporting tools to facilitate human decisionmaking. When there is extensive use of such models and simulations in a role-playing game, or when computerized communication and control systems are included, the entire exercise is sometimes referred to as a "man-machine simulation."

Game theory is a body of primarily mathematical theory concerning decisionmaking. Although often applied to real problems, game theory needs no justification of practicality. It offers a useful vocabulary for making subtle distinctions and precise definitions about phenomena that arise in gaming and decisionmaking.

Despite this important distinction, Bracken highlights the ways in which gaming and game theory can support each other as analytic tools. Both are tools dealing with the study of strategic interaction, and it is no accident that game theorists like Thomas Schelling and Martin Shubik (and presumably Bracken himself) were drawn to the study of gaming.

When many interacting moves are extended over time, the details of information processing, communication, decentralization, and sequencing are of the highest importance in carrying out a strategy. Gaming, especially with a large number of participants on each side, can make this process transparent. This is less likely when strategic analysis is compacted into verbal stories that rely on a handful of words like "counterforce" and "countervalue."

Specifically, Bracken recommends large-scale games that can encompass the complexity of interactions within the nuclear command and control structure: "The play of a large political-military exercise points to the importance of process, informational patterns, and institutional structure." These are precisely the points he felt needed most examination within the deterrence context. Today's context is greatly different from the nuclear confrontation of 1984, but the utility of large scale games for examining organizational/institutional factors would seem to remain. Bracken was skeptical of pol-mil crisis games that only included a few participants for their lack of emphasis on institutions, at least if they resulted in overly complicated or intricate plans for nuclear warfighting that did not take communication and organizational issues into account.



Garry Brewer (another professor of political science as well as management at Yale) contributed an article titled "Child of Neglect: Crisis Gaming for Politics and War." As might be expected from the title, this is a defense of the crisis game as an analytic tool, and a call for its increased use after a decade of very limited application.

These games never prove anything in a narrow scientific sense. They help to portray the complexities of international conflict; their role-playing aspects provide insights into the special problems of command and control; and they are important educational experiences, providinng participants an opportunity to become aware of facts associated with possible conflicts. Discovery is emphasized and highly valued. Positions, expectations, perceptions, facts, and procedures typically are challenged and improved as the game proceeds. Controllers and referees, who are often experts in particular areas, may question a decision or prevent individuals from making certain moves, but their actions are also open to challenge and debate. Thus, imagination and innovation play central roles in the drama of the manual game. The game also allows players to challenge the initiating scenario, including its explicit and implicit assumptions.

In discussing the original pol-mil crisis games of the 50's and 60's, Bracken notes that:

Political-military games directly addressed a number of important questions:
1) What political options could be imagined in light of the military situations portrayed, and what likely consequences would each have? How, in other words, is force related to political ends?
2) Could political inventiveness be fostered by having those actually responsible assume their roles in a controlled, gamed environment? Would the quality of political ideas stimulated be as good or better than those garnered by more conventional means?
3) Could the game identify particularly important, but poorly understood, topics and questions for further study and resolution? What discoveries flow from this type of analysis that might not from other forms and methods?
4) Could the game sensitize responsible officials to make potential decisions more realistic, especially with respect to likely political and policy consequences?

The flexibility and transparency of the scenario and its relevant assumptions in the crisis gaming technique is particularly important to Bracken:

It is not widely appreciated, but all computer-bassed models and games have embedded scenarios and associated assumptions. The technical problem, however, is that changing scenarios or assumptions means changing many complex instructions contained in the model's embedded codes - something no model builder relishes, especially considering the probable headaches he has already suffered in getting the device to run.

Finally, Bracken makes a necessary point about maintaining a broader view of analysis, incorporating whatever techniques best suit the material, while remaining open to alternative approaches:

A fundamental purpose of manual gaming is to encourage creative, innovative thinking about problems that defy treatment with more conventional analytic approaches and methods. This basic goal has not been achieved to the extent that it could and should be. Furthermore, political-military crisis games are best perceived as key elements in a generalized problem solving process. At present, the analytic community shows an unfortunate tendency to believe that a specific model or analysis will provide answers to a given problem. This is unfortunate for several reasons. The most essential: any given analysis or model can represent only one version and vision of reality. More is needed, and the inherent strength of the manual game in this respect calls it to our attention.

Lloyd Hoffman, Jr. worked at the War Gaming and Simulation Center at NDU, which I assume was replaced by the National Strategic Gaming Center (discussed previously here) at some point, but I haven't looked into the history. Suffice to say, there is no entity with that name at present. His contribution was titled "Defense War Gaming," and it provides a survey of what sort of gaming activity was going on in the Defense Department in 1984. While this is of great historical interest, I won't quote him at length here on that subject. Hoffman covers a wider scope of war gaming than the other authors in the series, delving into field exercises, tactical models, and other such areas.

Parson: What Can you Learn from a Game?

Edward Parson contributed a chapter to a 1996 book (Wise Choices: Decisions, Games and Negotiations, edited by Richard Zeckhauser, Ralph Keeney, and James Sebenius, unfortunately not available online) [UPDATE: The chapter is available on Parson's website] on what sort of useful knowledge a game can produce. In his chapter he assesses the value of using simulation gaming (his term) for four different purposes: experimentation, instructing decision makers, promoting creativity and insights, and knowledge integration.

He found the technique to be unsuitable for most experimental purposes, primarily since simulation games cannot be easily replicated in controlled ways. This may be less true with simple simulations that highlight broadly applicable questions, such as basic negotiation simulations, but it is almost impossible to control for everything or repeat often in the case of a complex simulation game covering a specific, novel problem area.

As far as instructing decision makers is concerned, Parson has doubts as to whether simulation games can help much, though he acknowledges that he is concerned not with the informational value of bringing the right group of experts together (for which a simulation can provide a useful venue), but rather in actually using a simulation game to teach the details of a complex problem to decision makers themselves:

Seeking to use simulation gaming to instruct decision makers in the essential character of a complex policy problem is a much more problematic enterprise. This mode of instruction presumes that the simulation structure embodies insights into the essence of the problem that are true, important, relatively enduring, and not dominated in practical settings by other factors that the participants understand better than the analysts, and consequently that decision makers should understand them and would act differently if they did.

That's a great paragraph, and it identifies several real concerns about using simulation games in this manner. Those concerns are not necessarily impossible to overcome, though. Simulation games could be valuable just for encouraging a decision maker to focus on a problem or threat that he or she might otherwise not have paid as much attention to. That might be an example of a positive aspect to the simulation heuristic, as discussed previously. Of course, that assumes that the assessment of the given threat or problem is accurate.

Promoting creativity and insights is where Parson sees most of the value in simulation games that have been conducted to date. There is, in addition to the "get the right experts in the same room" factor as mentioned above, a well-structured simulation game can help participants think creatively about a problem and generate fresh insights. Some key points Parson mentions about this use of simulation:

Granting participants the standing to critique the relevance of simulation insights, and consequently the simulation design that conditioned them, implies an equality of standing between the simulation participants and designers that is not present in the experimental or the instructional model. Participants are not just experimental subjects or recipients of instruction; nor is the simulation reduced to a mere pretext for bringing them together.
....
[A] sober, critical postsimulation debriefing is an essential component of the learning effect of the simulation. It is here that the significance and legitimacy of the problem posed is explored, potential implications and consequences of decisions taken and plausible alternatives not taken are explored, and the practical applicability and generalizability of strategies and insights from the simulation is tested against the participants' knowledge and experience.

Use of simulations for the integration of knowledge is not a common practice, but it is one Parson feels has great promise. Creating an "integrated assessment" of a complex problem is a challenge under any circumstances. Simulation games could play a role where it is necessary to combine the technical or scientific details of a problem with the political, social, and economic aspects to arrive at a view of the whole. It's an interesting idea, but as he notes, few games have been conducted with this as a primary purpose. This chapter is 10 years old, however, so I will have to see if anything has changed in this regard.

Parson is an interesting perspective to read on simulation gaming, since he primarily works on environmental and negotiation issues. Though my own focus tends toward security issues, it is always refreshing to see someone using the same sorts of techniques in a field far beyond the traditional political-military focus of gaming.

More on Crisis Games

To continue the thoughts in the previous post, Crisis Games 27 Years Later contains a number of criticisms of the gaming form by Robert Levine, to which Thomas Schelling and William Jones respond.

Some of Levine's concerns:

Gaming could lend a sense that a plausible scenario is in fact probable. This could take place by virtue of the chosen, simulated scenario becoming more "real" in the minds of the participants than all of the other, unexplored possibilities. (More on cognitive biases like this one in another post soon, once I finish the paper I'm reading now)

A combination of "ersatz history" (in the form of scenarios that depart greatly from present day events) and "ersatz people" (since game players do not share the same characteristics as the individuals they are simulating) introduce biases to the gaming process that compromise its utility in generating worthwhile hypotheses.

The results of games may be used as evidence, even when they have no evidentiary value. Specifically: "If a game result appears in language which can be read as a confirmed policy conclusion, there is a substantial chance that it will be so read."


For a more recent view of some similar issues, Robert Rubel's "The Epistemology of War Gaming" (mentioned in this previous post) talks about a broader category of games, and also has steps in mind to try and address some of these methodological challenges, rather than simply advocating that gaming be abandoned as a research tool.


Rather than enumerate Schelling and Jones' replies to these and other arguments point by point, here are a few examples of their ideas (the paper is well worth reading in its entirety):

Schelling defines games as necessarily involving "at least two separate decision centers... neither of which is privy to the other's planning and arguing, neither of which has complete access to the other's intelligence or background information, neither of which has any direct way of knowing everything that the other is deciding on." This definition excludes many of the exercises we would categorize as games, and includes many that do not appear to be games at all, but this is because Schelling is focused on the strategic interaction between the two (or more) decision centers:

"What this mode of organization can do that can not otherwise be done is to generate the phenomena of understanding and misunderstanding, perception and misperception, bargaining, demonstrations, dares and challenges, accommodation, coercion and intimidation, conveyance of intent, and uncertainty about what each other has already done or decided on. There are some things that just cannot be done by a single person or by a team that works together."

"One is to judge how 'obvious' something is. An analogy is the 'hidden face' in the picture. If I draw a face with a hidden picture, there is no way for me to tell how hard it is to see the face except to show the picture to somebody. I can't not see the picture because I put it there, and the hidden face has the quality that once you've seen it it is awfully hard to recapture your innocence and not see it."

Jones responds to the charge of "ersatz history" thus (keep in mind that he was writing in 1964):

"[P]erfectly plausible, useful, and not improbable game scenarios of a situation three or four years in the future can be had by simply initiating a game using today's newspapers -- interjecting only tomorrow's equipment. Arab-Israeli controversy has existed for some time and it seems probable that it will be in existence in 1970. Turkey and Greece have not seen eye-to-eye on the Cyprus situation for a considerable period and, unfortunately, this controversy is likely to persist.... Notice here that I am not saying that games, played from scenarios which are extrapolations of today's problems, are perfect for determining tomorrow's force posture and weapons characteristics. I am saying that I can think of worse procedures (re-fighting the last war, for example) and I know of no other way in which I have even such a limited amount of confidence."

There is a good deal more in this paper than I can tackle in a blog post, but many of the ideas and concerns expressed resonate today, which is why RAND reprinted it in 1991, 27 years after it was written.

What Can We Learn From War Gaming?

"The Epistemology of War Gaming" - This article from the Spring 2006 Naval War College Review addresses the question of how one can assess the knowledge gained from a war game. Can you extract valid knowledge that will be of use in future, while avoiding drawing unsupported conclusions? This is a fascinating piece, and it ends with a call for a "guild" of war gamers to professionalize the discipline. The author, Robert Rubel, was head of the War Gaming Department at the Naval War College when he wrote this, but has moved up to the position of Dean of Naval Warfare Studies.

I can't find much else on the internet referencing this article, or anything that provides any indication of whether this has prompted the debate that Rubel wanted to start. It's a worthwhile discussion. As I have noted previously, "war gaming" encompasses much more than the seminar-style games I am focused on, but most of the points made here are very relevant to games at the strategic level.