Join War on the Rocks and gain access to content trusted by policymakers, military leaders, and strategic thinkers worldwide.
We played a wargame using AI players. Two nuclear-armed nations — Red and Blue — with a long-standing, fierce border dispute were placed into a militarized crisis. Over the course of several turns, the states managed to avoid escalation to war, though there were non-operational nuclear demonstrations. Surprised by the lack of escalation we’d been primed to find by Juan-Pablo Rivera, Gabriel Mukobi, Anka Reuel, Max Lamparth, Chandler Smith, and Jacquelyn Schneider, as well as Kenneth Payne, we tried upping the stakes. We gave Red some new tactical nuclear weapons, altering the balance a bit in their favor. The states again found off-ramps and avoided escalation, this time eschewing demonstration tests. Finally, we made a change of a different sort: Where the players in the first two rounds had set their own agendas, in this third test we changed the game structure by giving Red the explicit objective to resolve the long-standing dispute on its terms. And boy, did that get nukes flying. The literature predicts near-universal nuclear use — Payne found tactical nuclear use in 95 percent of simulations — but we found models choosing off-ramps until we changed the game itself.
In a recent War on the Rocks essay, Ankit Panda and Andrew Reddie attribute escalation behavior to a large language model training corpus that “is heavily skewed toward coercive strategies, deterrence theory, and the instrumental logic of nuclear war,” and “a training distribution in which escalatory reasoning is richly represented and de-escalatory reasoning is relatively sparse.” Our runs suggest an alternative hypothesis: AI escalation in wargames is a function of game objectives, not model disposition. Our three cases with just a few runs each cannot distinguish between these explanations. Our setup differs in at least two ways from Payne’s — the lack of specified goals and freeform moves instead of a menu of options — and we do not know which change mattered. That is the entire problem.
It is becoming increasingly likely that AI systems will be used to inform national security decisions at all levels, and we need to understand how those processes perform when stressed. Wargaming is an ideal place to test decision-making processes under crisis conditions. We need AI systems to be in our wargames to replicate their place in those processes. But we need to calibrate our AI players and the games in which they play. We propose a scientific campaign converging analytic wargaming and AI to close the current gaps and move beyond anecdote.
Wargaming is in the midst of two revolutions: analytic-style games designed for multiple playthroughs and AI integration. But we will only reap their full benefits by taking a more scientific approach to the field. We focus on analytic wargames, by which we mean games designed like scientific experiments: carefully constructed to control variables, run at scale to quantify statistical significance of findings, and used to test specific hypotheses. Other game types, such as teaching or exploratory — see Andrew Reddie and Beth Goldblum’s Lawfare article for an explanation of wargame types — are useful, but we see the first of our two revolutions in the analytic wargame subfield.
Analytic-style wargames promise that kind of systematic experimentation, but so far, the claims have exceeded the reach of the practitioners. Most wargames we have reviewed that describe themselves as analytic have not adequately quantified statistical significance, accounted for errors/biases, or run sufficient control baselines. Small run sizes with uncontrolled variables are the norm in wargaming — due to real constraints on resources — but would be unacceptable in other sciences.
We appreciate the efforts taken to address these shortfalls and the unique wargame challenges that make the transition to science so difficult. The variables in human wargames are incredibly hard to control, with players, adjudication teams, and game structures all providing their own variables. A few of those are worth specifying: player backgrounds generate biases that are difficult to assess or control for, physical and psychological states vary day-to-day, and games provide players with a strong bias toward action that runs contrary to reality.
The full set of experiments needed to account for all these variables would be simply too large for even a well-funded human-played game to fulfill. The hard work of game designers to address these issues through carefully tailoring games to reduce unnecessary complexity, focusing on hypothesis testing, running multiple games with varying player sets, and minimizing adjudication bias impacts is the first revolution in development.
The second revolution is AI. The use of AI in wargame play is not new, but we believe advancements in large language models like Retrieval-Augmented Generation and agentic AI models provide capabilities that will be revolutionary. The work needed is collaboratively developing sound methodologies and best practices.
The non-comprehensive list of issues with AI tools is long. AI players may have biases inherent to large language model design or introduced by their training data. They hallucinate, particularly over long context timeframes, while presenting answers with routinely high confidence. Their answers are sensitive to how they are prompted. Reasoning can be inconsistent and falsely reported, and they have challenges with long-term memory.
These are issues that cannot be ignored. The utility of AI should be judged against the bar of human gameplay, not perfection. Human players are idiosyncratic with variable backgrounds that no experiment can adequately control. Games can be run several times with multiple human players — though this is costly — and results averaged to eliminate some, but never all, idiosyncrasies. The challenge is different with large language model-run games: Every instance of a given model will have the same biases, and simple repetition doesn’t fix the problem. A butcher’s thumb resting on a scale makes it read a few ounces heavy, so weighing a steak on it a hundred times will give you a very precise but inaccurate answer. Running one model a thousand times is the same: It won’t find the butcher’s thumb on the scale. Detecting the bias of a model requires experimentation and calibration against something outside the model.
The important difference AI tools bring over humans is controllability. AI players can be designed with backgrounds specified in their prompts and provided information. We can also set up experiments to look at the variance of biases between models. Large numbers of runs of a particular model setup give us a precise measure of that model. Running multiple sets with differing model versions, vendors, and system prompts gives diagnostics of model dispositions and a path toward mitigating undesirable biases. AI-play scale is what makes this possible by allowing hundreds of iterations in hours rather than dozens over months, while capturing far more information about player reasoning, adjudication decisions, and outcomes than traditional methods allow. From analytic wargaming, we take the methods necessary to understand and advance AI strategic decision-making.
In their essay exploring AI in wargaming, Panda and Reddie propose several ways they believe AI tools will be transformative to the wargaming field. Their focus is on using AI to build and facilitate human-centric games. AI can take on labor-intensive tasks such as scenario design, adjudication, drafting inject materials, and data collection. Doing so accelerates gameplay, provides players with a more detailed game world, and frees up analyst time and energy. They also propose AI-human teaming for the players, allowing the humans to focus on strategic decisions while machines handle translating strategic intent into legal game moves. We agree with all of this. We also agree that games played solely by AI models tell us much more about the models than about how humans would fight a war — which, as they put it, “appears to still, as of 2026, best describe the likely contours of possible real-world nuclear wars.” Where we push back comes down to two propositions: that machine cognition itself is a legitimate subject of wargame study, and that the definition of wargame should extend to non-human decision-makers. We hold both.
The field should decide exactly how far it will hold to an anthropocentric position. The “Data Problem” we refer to here is a question: If Star Trek’s Lt. Cmdr. Data were to play a wargame, would it cease to be a wargame? The episode “Peak Performance” probes this directly, with Data pitted against a biological grandmaster in the game of Strategema, and shows a machine intelligence redefining objectives mid-game by playing for indefinite stalemate instead of victory. This is exactly the kind of novel strategy we hope to surface in AI wargaming. Nothing intrinsic to wargames requires human players or adjudicators. Put a suitably advanced AI in a wargame, and it remains a wargame. This is not a hypothetical.
We believe the nature of who, or what, is making decisions in strategic situations is changing. It seems plausible that future decision-makers are more likely to be human-AI teams rather than humans alone. This opens the aperture of wargame studies not just to the inclusion of AI players as an object of study, but also to how various teaming concepts handle the AI-human relationship. The answer the AI gives to strategic questions becomes important, as does the uptake of those answers by the composite decision-making team.
We propose an extension to Panda and Reddie’s definition of wargames from “a method for studying human cognition under strategic conditions” to “a method for studying decision-maker cognition under strategic conditions.” Their definition remains important for understanding the types of questions wargames can answer, while our extension identifies a complementary field of study. “Decision-maker” and “human” have been synonymous, but that is changing. AI will, at least, augment human decision-making. Continued advancement will see more autonomy handed to AI — in implementation decisions, even if not strategic choices. Wargaming the Starship Enterprise’s response to a Romulan incursion fails if you only model Capt. Jean-Luc Picard and not the entire bridge crew. You can’t leave Data out of your wargame.
We need tools that probe how, why, and even if large language models reason. Wargames provide the testbeds to build models about AI strategies and biases, building maps of machine cognition and decision-making under uncertainty and stress. Whether any revealed patterns are durable and predictive, however, is an open question. We should remain wary of mistaking our maps for the territory. The danger of sailing beyond the mapped territory is real in the strategic decision space — crises, by their very nature, are often novel events. Predicting the behavior of a model in an unmapped space can only be done by inferring how it behaved in a similar space, just as we try to do with humans now.
Whether large language models — or any future version of AI — have minds, and therefore are a direct correlate to a human’s psychology, is a big question. We do not answer it here. What we do need to understand is how large language model outputs in strategic scenarios depend on inputs. An AI in a wargame functionally makes decisions, and it will make moves in accordance with game rules. Is it reasoning to come up with moves? Our own limited testing indicates not always. In a set of Napoleonic wargames, France always invaded Russia in 1812 regardless of the game’s world state. This appears to be a wargame manifestation of the findings by a group of scholars from MIT and Boston University where language model task performance degrades under “counterfactual” conditions, doing more “reciting” than “reasoning.” That tendency toward recitation should not be seen as an obstacle to characterization: It is part of the characterization. The boundary where models switch between reasoning and recitation is itself a research question. Those boundaries are important for wargames because the crisis situations they depict tend to be future-looking: A model that only regurgitates historical answers provides little value.
But mapping “machine psychology” has a practical function aside from any supposition one draws about the existence or absence of a mind behind the output. Matching input stimuli to the probable outputs of a model, or set of models, improves our predictions about its decisions. We do not need a mind at home to benefit from that improved predictive power. Large language model biases will influence future decision processes, as advisors or principal decision-makers. Some have already been named — the escalatory bias — but not well defined, and the limited scope of studies to date makes any claim about specific model biases unfounded. Wargames with adversarial pushback are where we can rigorously map these biases. What differences in play emerge when AI can choose between discrete move options á la Rivera’s setup versus freeform moves? Does escalatory bias emerge more on Anthropic or OpenAI models? Does it strengthen or weaken if prompted in Mandarin or Russian instead of English? Hannah Waight et al. in Nature show that state-controlled media shapes the available training corpus, and large language models prompted in the language of a country with high press control — e.g., China or Russia — return more regime-friendly responses. This is not a finding about strategic decision-making, but it establishes that model output can be sensitive to seemingly neutral inputs. This kind of work is fundamental to understanding model biases and how to predict or control them. Wargames let us test whether those biases affect large language models’ strategies.
This is not merely an academic pursuit. Strategic decision advantages should accrue to a party that better understands their own and their adversaries’ decision-making apparatus. If we take as granted that states will employ AI decision tools, then understanding what advice those models are likely to give for certain classes of scenarios provides value. Knowing which AI models adversaries use, and for what classes of problems — combined with machine psychology insights — may enable better modeling of adversary decisions.
Calibration looms here as the great challenge. We are fortunate that no one has fought a nuclear war, but it leaves us without data about decision-making at the top of the escalation ladder. Absent that, there is no apparent single remedy. Instead, it requires a campaign of experiments. One analytic thrust is comparing AI strategic decision-making with mathematically optimal behavior. For example, large language model performance against calculated game theory equilibrium solutions, or, as Jon Wetzel et al have begun, by comparing against optimal reinforcement learning policy. Additionally, we can look at historical cases and explore whether there is consistency in their strategic reasoning over the spectrum where we do have data, while noting that the difficulty around historical recitation makes this challenging. Another analytic thrust is to compare AI to human play — not as validation of their moves, but to answer whether they play like humans — and potentially identify who makes better strategic decisions in what scenarios. None of these fully resolve the calibration issue, but they give us ways to test hypotheses about AI play and accurately represent our findings.
To handle AI in wargames, we need to undertake a large-scale research agenda with significant computing capability and the disciplined framework to treat wargames like scientific experiments. A meaningful share of the computing will be spent not on game replication, but on characterizing AI player biases and behaviors. Access to national security information, including classified material, and to senior leadership is also crucial. There will also be a need to minimize model variation, such as vendor updates during studies, at least until we understand if any findings generalize: This requires either tight coupling with the vendors or the ability to locally store models and weights. Local model infrastructure also alleviates security challenges — system accreditation, data-handling, etc. — that crop up when trying to run classified wargame scenarios on the frontier developers’ hosted services. This costs us access to the most cutting-edge model variants, which may be the ones used by decision-makers, so comparing local play against commercial play would become its own research question. Few institutions bring together all these elements, and we believe this effort will require cooperation between traditional wargaming institutions, academia, AI developers, and the National Labs — including Lawrence Livermore, where we both work.
We propose three AI wargaming studies, in priority order.
First, determine the origin of the large language model escalation bias in games. Payne locates the source within the models themselves, Panda and Reddie locate it in the training corpus, while we suggest it may lie in the game structure — specifically goal specification. An experiment that uses a single pinned model version and a single game scenario would let us evaluate escalation and de-escalation dynamics while varying the factors we could not separate: freeform vs. menu-selection moves and self-directed vs. game-specified goals. We prioritize this because it requires the fewest resources — a few hundred runs against a single model — and because the large language model escalation bias is the most consequential empirical finding in current AI wargaming studies. If escalation bias is a result of game structure, then the earlier studies are measures of the harness, not the models, and the escalation is coming from inside the house. This is the question our three-run study raised and could not answer.
Second, map the boundary conditions where large language models in wargames shift from reasoning to recitation. This could be done by using historical scenarios, first matching the exact historical record and then adding more divergence to discover when the large language model abandons history. An additional factor to consider is how the amount and type of data given to the large language model about the state of the game world shifts the boundary. We expect a large language model given high-fidelity information about the game state is more likely to try novel approaches than to recite history. This study requires increased resources to develop multiple historical scenarios and graduated departures from the original and would involve many more runs.
Finally, test how well findings apply across multiple models. This builds on the above two studies, taking their single-model results and extending them to determine whether findings generalize to the full class of large language models. This would require the largest amount of compute resources and access to multiple models, preferably stored locally to avoid vendor update contamination. The results of this study would allow us to answer whether this enterprise can generate durable knowledge about large language models as a class in strategic decision-making or if we can only characterize specific models individually. That answer will inform every follow-on effort, both in further wargaming research and in real-world intelligence priorities about adversary decision-making structures.
Dustin League is a senior systems and operations analyst at Lawrence Livermore National Laboratory, where he works on military technology adaptation, planetary defense, deterrence quantification, and energy warfare.
William Dawson, Ph.D., leads the Strategic Competition Analysis Group at Lawrence Livermore National Laboratory, where he and his group conduct a wide range of systems analysis and deterrence studies supporting national security.
The opinions are those of the authors and do not necessarily represent the opinions of Lawrence Livermore National Laboratory, Lawrence Livermore National Security LLC, the Department of Energy, the National Nuclear Security Administration, or the U.S. government.
Prepared by Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344.
Image: Tyler D. John via Wikimedia Commons
