Is the Department of Defense Making Enough Progress in Wargaming?
Much of what the Department of Defense calls wargaming is not actually wargaming. This is a problem for reasons you can imagine — done right, gaming is one of the few ways to test out battle plans and designs for a military that can match China’s — and some you might not. The Nintendo generation is coming and, still, most games happen on physical tables, around which conventional wisdom is reaffirmed. Too many games are one-off exercises when the situations for which they purport to prepare will be anything but. The flood of interest in wargaming that started when Pentagon leaders leaned in to the idea five years ago shows no signs of abating, but we still don’t know if the department’s wargames are working.
In 2015, then-Deputy Secretary of Defense Bob Work and Vice Chairman of the Joint Chiefs Gen. Paul Selva pushed to revitalize wargaming in order to better prepare the nation for future wars. They drew parallels to the interwar years, when the pace of technological change and disruption caused some militaries to fall behind, and others to innovate. A few months before that, in December 2014, several of us within the wargaming community met to discuss wargaming education. One person brought warning of Work’s impending interest in wargaming, adding additional urgency to the situation. There were only so many experienced wargamers who were prepared to instruct at the time, and presumably a flood of interest was coming. We then spent a few frantic years organizing conference after conference to “feed the newbies,” as we called it, trying to give hands-on wargaming experience to the large number of action officers suddenly being sent out to wargame (or “wargame”) to the best of their abilities. Surely, at some point, interest would die down.
Five years into its reinvigoration, the military’s interest in wargaming remains strong. Strategy writing teams in the Pentagon extensively wargamed candidates for the 2018 National Defense Strategy. Demand has only increased for approaches that can help senior leaders think through everything from technologies such as artificial intelligence and cyber to fully fledged concepts such as the joint warfighting concept and joint all-domain command and control. Wargaming plays a key role in these activities and, despite its limitations, few practical alternatives exist.
Yet, if wargaming continues to be one of the few tools available to better prepare the U.S. military for the future, is wargaming, as conducted by the Department of Defense, up to the task? There are four questions the department needs to answer before it will know.
First: Is the quality of existing defense wargaming sufficient? Is the overall defense wargaming enterprise able to support the present challenges in concept development, analysis, capabilities development, and professional military education? Best practices in wargaming include adjudicated wargames instead of unadjudicated tabletop exercises and series of games rather than individual one-off events. Assessing the quality of wargaming at the department level involves assessing whether these best practices are followed rather than simply identified. Senior leaders have also identified quality problems in wargaming such as insufficient initial research, flawed game designs, and unsound basis for adjudication. A frank and objective look at such issues is necessary to spot problems that should be corrected. The quality of other important wargame inputs — such as scenarios, dynamic adversary players, and courses of action — can also vary widely but are rarely evaluated.
The quality of information available on wargames that the department has already run can also be an issue. Analysis of the Pentagon’s existing wargame repository shows that three-fourths of repository users viewed wargame entries that their organizations did not sponsor, which certainly indicates significant progress in sharing insights across the department. However, basic information on wargame design, scenario, insights, and lessons is often missing.
The department needs to better understand the quality of its entire wargaming portfolio. A pessimistic view would predict games that cluster around a few usual, expected scenarios rather than exploring unexpected cases. Repetitive courses of action across games would also be an indication that the department is content to rerun accepted approaches rather than explore new ones. If a large percentage of exercises are unadjudicated, with participants merely taking a static scenario and providing their plan, what the department is actually engaged in is planning, rather than wargaming against a thinking adversary. Do service wargames invariably “validate” the service concept or provide additional justification for programs of record? It would seem best to trust, but verify. Wargames should additionally be assessed for how well they include domains such as space, cyber, and information assurance, as well as how often allied and partner perspectives are adequately represented.
A second key question for the department to answer is whether wargaming does in fact improve learning and innovation. The truth is that we have little to no empirical research that shows wargaming promotes learning, creative thinking, or problem solving — at either the individual or organizational levels. While wargamers believe wargaming works, the department still lacks empirical research confirming wargaming’s ability to positively impact cognitive processes, knowledge formation, individual and group problem solving, organization learning, individual creativity, and organizational innovation. For example, even though wargamers often repeat the stories of Naval War College wargaming in the interwar years, the department lacks basic case study research on the organizational learning models that permitted some countries to learn from wargames but not others. Although military planners, operations research analysts, political scientists, and hobby wargamers dominate defense wargaming, these are not the best research backgrounds to assess whether wargaming works. Instead, the fields with the methodological tools to answer this extremely important question include educational psychology, experimental psychology, industrial and organizational psychology, social cognition, social intelligence, communications, and the study of creativity. All these fields are notably absent from wargaming.
Squarely within this topic of learning from wargames is assessing the possibility of negative learning. Negative learning is learning erroneous concepts from unwarranted information and developing faulty mental models and reasoning. Wargamers often discuss the benefits of experiential learning from wargames, but those same games are also opportunities to reinforce conventional wisdom from the last war, pre-existing biases, and consensus (but perhaps incorrect) assumptions about the adversary, as well as to once again discount crucial factors such as logistics or allies. In order to correct and prevent negative learning from wargames, this topic must also be studied.
A third question for the department to answer is whether there is sufficient wargaming capability and capacity across the defense enterprise to support current and future wargaming needs. A partial review of wargaming centers shows a fair amount of facility capability and capacity, and more is being added through facilities under development such as the Marine Corps Wargaming and Analysis Center. For example, this new center in Quantico will host more than a dozen games a year. The Joint Staff’s Studies, Analysis, and Gaming Division can run approximately three dozen tabletop exercises a year in conjunction with its other mission areas. The Naval War College Wargaming Department, the flagship wargaming center in the defense community, runs at least 50 a year.
But is there enough staff capacity and capability? The 2016 Military Operations Research Society’s wargaming special meeting report noted the consensus that capacity was insufficient within the existing wargaming community to meet demand. A 2018 query of combatant command wargame staff also indicated challenges to capacity and capability, yet only one of ten responding organizations had plans to build a wargaming cell. Rather than devoting specialized staff to wargaming, they reported rebuilding their wargaming capacity “out of hide,” adding wargaming as collateral duty and bringing in contractors for expertise. Defense wargaming capacity also resides within contracting companies, federally funded research and development centers, and academic departments (both civilian and military), making it difficult to fully account for total capacity and capability.
One approach may be to assess whether specific organizations access enough wargames with enough capabilities to support their needs. For example, does U.S. Indo-Pacific Command believe it has adequate wargaming capacity of sufficient quality to address its requirements — including its virtual and distributed requirements? Does the Joint Staff have access to sufficient capacity to support developing the Joint Warfighting Concept and all of its supporting concepts? Are the service wargaming centers consistently oversubscribed, and, if certain wargames are not prioritized, can sponsors find wargamers elsewhere? Are there enough specialized performers in wargaming to ensure quality? Significant costs, skill sets, and other requirements need to support quality wargaming, and distributing the same amount of resources among organizations that do not specialize in wargaming will naturally lead to lower quality and sophistication overall.
This leads us to the fourth question the department needs to answer: What is the state of the wargaming workforce, and does it need to modernize this workforce, in terms of backgrounds, skillsets, and professional practices? Professional wargaming can be a fiercely competitive business, with some wargamers reluctant to share information about opportunities with other wargamers inside their own organization. Yet, the department has an institutional interest in raising the quality of wargaming throughout its enterprise. Does the level of expertise throughout the wargaming community meet the complex demands on wargaming? Professional development opportunities such as the Military Operations Research Society certificates in wargaming are often oversubscribed and are only meant to train in the basics.
Given the lack of civilian educational degree programs to produce wargamers, and the retirement of hobby gamers who have historically formed the backbone of defense wargaming, where are new wargaming experts coming from? There are significant generational differences between retiring wargamers and those coming up — the commercial gaming experience for Generation X (the “Nintendo Generation”), millennials, and Generation Z are largely digital and not manual. This coincides with urgent pressures from the department for wargaming to incorporate more digital capabilities, represent advanced capabilities such as artificial intelligence and quantum technologies, and address everything from peer competition to cyber to misinformation. There is also concern that the wargaming community lacks diversity and is therefore missing out on many other perspectives. What should the expertise of the next generation of wargamers look like? This is a vital topic. So, too, is whether the department has any role to play in shaping the next generation of wargamers.
Only after answering these questions —
1) Is the quality of defense wargaming sufficient?
2) Does wargaming actually work?
3) Is there enough wargaming capability and capacity to meet the department’s needs?
4) What is the state of the defense wargaming work force?
— can the department understand whether it is making the progress it needs to re-invigorate wargaming.
These are the questions that keep us — longtime wargamers — up at night. So much rides on wargaming, and whether wargaming is done well or done poorly at this moment will impact departmental decisions for years.
Dr. Yuna Huh Wong is a research analyst in the Joint Advanced Warfighting Division with the Institute for Defense Analyses, Alexandria, Virginia. She co-led the Hedgemony wargame that supported the 2018 U.S. National Defense Strategy and was a co-director of the Center for Gaming while at RAND Corporation. She is also founder of the Women’s Wargaming Network.
Garrett Heath is a research analyst in the Joint Advanced Warfighting Division with the Institute for Defense Analyses, Alexandria, Virginia. He is a retired Army colonel who previously led the Joint Staff’s Studies, Analysis, and Gaming Division, and the Capabilities Development Directorate in the Combined Security Transition Command — Afghanistan.
The views, opinions, and findings expressed in this paper should not be construed as representing the official position of either the Institute for Defense Analyses or the Department of Defense.
Image: Dreamstime
