Hot Models, Hard Questions
Editor’s Note: WOTR is pleased to welcome its newest columnist, Adam Elkus.
From computer models of climate change to intricate simulations of future conventional conflicts, abstract models, wargames, and simulations remain prized tools for analyzing security problems. Models and simulations offer the ability to rigorously visualize uncertain complex situations with a vast array of variables and moving parts. Syria is no exception—think-tankers have created crisis simulations while gamers play topical tactical Syria-themed games. But while ethical and philosophical questions surrounding the Cold War analysis of nuclear strategy loomed large, today’s games do not receive similar scrutiny. There is no cinematic equivalent of Dr. Strangelove for today’s computational “web of war.”
The utility of modeling, simulation, and gaming cannot be disputed. Why? Everyone, at heart, models when they forecast some complicated political or social thing or make a claim about its dynamics. Conceptually, such act models the thing of interest by abstracting useful features and reconstructing it inside the modeler’s thought process. Because our puny brains can’t visualize the world in its marvelous entirety, abstraction and simplification is inevitable. So voila! You’ve got a model—and all you did was drink some cheap beer and rant to your Hill staffer buddy about who you think the real winner of sequestration will be.
However, there is a difference between building a model and just thinking up (mostly unconsciously) a mental vision of how things are or will be: actual (or perhaps, explicit) models can be tested. Their logical consequences can be elaborated. They can be adjusted to fit the data we have. We can adjust all manners of interesting variables and see what happens.
To pick a humorous example, the famous El Farol Bar Problem models the quest of some enterprising Santa Fe residents looking to visit a popular bar. One problem: El Farol is pretty tiny and crowds quickly on Thursday nights when there is live music. Everyone wants in. But every person has an overcrowding threshold—too many people and patrons will start to stay home. Our New Mexican barflies predict bar attendance with a prediction strategy, and the modeler controls how many strategies each individual is allocated. The prediction strategy depends on how many past Thursday nights (memory size) each individual can use at each decision time.
All of the variables in italics are adjustable—try decreasing the amount of prediction strategies and memory size and see what happens. You can download the El Farol model and see if you can replicate its results. And if you like El Farol enough, if you can even extend it with new features, assumptions, and variables.
It would be great if all models did was create a nice, simple abstraction of the world to inform good decisions. However, it’s a bit more complicated than that. A model can also influence the reality it seeks to abstract. Both finance scholars and sociologists of science claim that financial theory has acted as an “engine” as well as a “camera,” building market practices around models and theories designed to explain and predict those very market practices. The simulation could very well alter the thing it simulates.
More common (and pernicious) is the way bad models dress up shoddy thought with walls of intimidating mathematical formalisms and computer code. While a simulation should be an exercise in exploration and experimentation, but it frequently also functions as a way to validate a narrow vision of a desired future. For example, the famous Millennium Challenge 2002 wargame’s purpose was to test a new suite of military concepts. But red-teamers playing opposing forces quickly found themselves stymied when they exposed the fragility of these operational concepts.
Finally, the real-world validation of models, simulations, and games is still an evolving question in both academic and applied settings. Financial modeler Emanuel Derman argues that financial models often style themselves as scientific in character; yet, physicists can repeatedly experiment in controlled conditions, while financial modelers are left with far cruder methods: “[m]arkets are made of people, who are influenced by events, by their ephemeral feelings about events and by their expectations of other people’s feelings.”
Derman’s solution to this problem lies in a renewed focus on modeling ethics. The “dirt” any conceptual model of the world “sweep[s] under the rug” should be highlighted. An exercise in simulation should not be a magic performance where razzle and dazzle is placed front and center. Those who deal in abstractions should be prepared to justify them to skeptical audiences, and if possible, allow outside replication of their experiments. This is not merely an academic problem—if we build a Syria simulation modeling the Air Force versus Syrian air defenses we should be prepared to explain and defend every important assumption we make, and experiment enough with it to find some assumptions we didn’t think we made.
However, simulation, modeling, and gaming do not possess some all-powerful guild that can coerce every modeler to adhere to modeling norms that remain fundamentally hazy and informal. And replicating many models and simulations in the security world is difficult for those without security clearances. To use economic modeling terminology, the information asymmetry such a situation creates can help dishonest modelers “strictly dominate” more scrupulous counterparts.
Thus, even those who dislike simulations, models, and games must adjust to a reality in which they will enjoy continued prominence. An effective way to become an informed consumer of models and simulations (and thus argue against bad ones) is to design/build or run/play one’s own. Powerful simulations were once only available to military organizations and research universities. Today, we live in an unprecedented age of computational abundance. Anyone with a rudimentary knowledge of programming can create powerful models and simulations on standard computing equipment.
Free tools, such as the modeling language NetLogo, exist for casual experimentation and analysis. Standard libraries in a variety of programming languages are available to do everything from solving complicated mathematical equations to designing videogames. Even if you never end up building the perfect model to present to the SecDef, you can still enjoy endless hours of fun blowing up the DMZ in your living room.
There is no optimal solution to the problem of ethics in simulation. Instead, we must satisfice by educating ourselves and others about the logic and dynamics of simulation—both to solidify our own abstractions and effectively critique the act of abstraction. If we want to stop dissimulation, we must all learn to simulate.
Adam Elkus is a PhD student in Computational Social Science at George Mason University. He has published articles on defense, international security, and technology at CTOVision, The Atlantic, the West Point Combating Terrorism Center’s Sentinel, and Foreign Policy.
Photo Credit: Imperial94, Flickr
