Lessons From the Long War: The Role of Information in Counter-Insurgency


Eli Berman, Joseph H. Felter, and Jacob N. Shapiro, Small Wars, Big Data (Princeton University Press, 2018).

Goodbye Syria. Goodbye Afghanistan. The Peacekeeping and Stability Operations Institute – axed. Publication of the Army’s history of the war on terror – indefinitely postponed. The U.S. Army is withdrawing from its longest wars and pivoting towards conflict against state opponents. Though some are calling to abandon all things counter-insurgency, the Army must preserve hard-won lessons from fighting against non-state opponents. With more leaders lacking combat experience, the military risks losing these lessons from its long wars.

At various times, the United States has successfully driven down insurgent violence in Iraq and Afghanistan. However, neither practitioners nor academics agree on the cause of this reduction. After 2006, the new counter-insurgency manual, FM 3-24, focused the U.S. Army on securing populations. Counter-insurgency doctrine may have reduced violence in Iraq, but we must also acknowledge the role of the 2008 troop surge and the truckloads of cash that accompanied it. Meanwhile, soldiers carved up cities with concrete and mounted massive MRAPs. Beyond tactics, new targeting techniques and novel technologies all contributed.

So what caused the drop in violence? With the last edition of FM 3-24 published in 2014, the Army should capture on paper what works before the knowledge is lost.

Enter Small Wars, Big Data by Eli Berman, Joseph H. Felter, and Jacob N. Shapiro. This team of practitioner-academics leads the Empirical Studies of Conflict program at Princeton University. Small Wars, Big Data argues that the U.S. military should focus its efforts not on winning hearts and minds, but rather on acquiring information from the people about insurgent activities. When satisfied with government security and services, civilians supply information. With information, counter-insurgent attacks can dismantle insurgent networks. Without it, insurgencies fester and violence increases.

Armed with this framework, I would have fought differently – and American forces in Iraq and Afghanistan might have as well. During my first deployment, we worked hard but aimlessly. Local leaders accepted development projects but failed to support the Afghan government in return. Our radio broadcasts reached as far as the local government: hardly off the base. The only way for local civilians to provide information was by arriving at the gate during the day. On my second deployment, we focused primarily on developing information and built that capacity among our Afghan partners. As tips flowed in, our targeted operations captured insurgent financiers, forgers, and destroyed insurgent material. In this tale of two deployments, focusing on actionable intelligence through tips made the difference.

Unlike other prominent works in the counter-insurgency field, Small Wars, Big Data brings a scientific approach. First, the authors compiled a remarkably broad picture of several conflicts. To understand Iraq, they combined conflict databases that recorded every significant activity by American forces in the country with development data from the Army Corps of Engineers and the U.S. Agency for International Development. Then, using the titular “big data,” the authors test their information theory to determine what works.

Contrast this data-driven approach with earlier counter-insurgency works. David Galula’s Counterinsurgency Warfare: Theory and Practice and Charles Gwynn’s Imperial Policing synthesized personal experience into recommendations without deliberately testing their validity. More recent works, like John Nagl’s Learning to Eat Soup with a Knife, compares cases to determine why some counter-insurgencies succeed. Small Wars, Big Data is a rare work that deliberately tests theories with econometrics techniques and conflict data from across the world.

Consider development. Small Wars, Big Data’s rigorous testing shows certain development can be unproductive, though FM 3-24 gives it a prominent role. FM 3-24 argues “funding developmental assistance … [will] win the support of an indigenous populace and erode support for the adversary.” To test this theory, the authors present academic research that reveals how poorly conceived development projects inflict significant harm. In one study, they compare violence in Philippine villages just above and below the poverty line. Those below qualified for a development program. Those above did not. The development program was associated with an 85 to 110 percent increase in annual casualties. However, results of other studies offer a clear prescription for policymakers: Small, secure projects conditional on local cooperation with security forces do reduce violence. In the same way, Small Wars, Big Data considers dozens of studies to determine how violence levels are affected by aid, suppression, violence against civilians, and economic conditions.

These studies point to six conclusions counter-insurgency doctrine should incorporate:

First, information is key. Information – and especially tips from local residents – is associated with short-term reductions in violence. In Basra, Iraq, in 2006, each additional tip reduced mortar attacks by 50 percent and roadside bombs by 15 percent. This is a big impact for a small action. Counter-insurgents need information.

Second, getting more information is easier in counter-insurgency than in interstate conflict. While intrastate war requires mass mobilization, precise counter-insurgent interventions increase information flow. Based on data from Iraq in 2007 and 2008, the authors conclude that each coalition-caused civilian casualty was associated with almost one fewer tip each week. Tighter rules of engagement, such as those enacted by Gens. Stanley McChrystal and David Petraeus, protect civilians and keep tips flowing.

Third, hearts and minds are important, but counter-insurgents just need one tip. FM 3-24 emphasizes winning the loyalty of locals through expensive population protection and development. Small Wars, Big Data indicates that popular attitudes are less important than those of citizens on the margin. Rather than target all citizens, my Afghan partners collected tips from local civilians. From these tips, we captured insurgent leaders, destroyed opium factories, and disrupted insurgent finance. These effects did not require winning hearts and minds – just tips.

Forth, information mechanics matter. After restoring cell phone coverage in Anbar province, attacks dropped because the population felt safe providing information over the phone to the coalition. Citizens share more information when the channels for doing so feel secure.

Fifth, keep aid modest, secure, informed, and conditional. In Iraq, projects valued at less than $50,000, guided by coalition experts who worked closely with the local community and conditional on the community’s cooperation, reduced violence the most. The best projects focus on small, deliberate projects coordinated with military forces.

Finally, counter-insurgents should collect data, then weigh the benefits of sharing that data against the risks. Data collected by military and civilian bureaucracies on tips, casualties, development, and public opinion made the authors’ research possible. Though data cannot win a war – see Vietnam – timely analysis can inform tactical and operational approaches to curbing violence.

Despite these six valuable conclusions, the book struggles in two areas. First, it focuses overwhelmingly on the Western interventions in Iraq and Afghanistan. This makes sense given the author’s support from the U.S. Department of Defense and their backgrounds, but may limit the validity of their recommendations. Though the authors did include lessons from other conflicts like the Philippines and Mexico, most intrastate conflicts do not involve Western intervention. Readers should hesitant to extend these lessons to all conflicts. Still, the focus on Western counter-insurgencies makes this book a good starting point for rewriting FM 3-24.

Second, Small Wars, Big Data has a “Goldilocks problem” in terms of accessibility. As an informed practitioner with two combat deployments and recent study of econometrics, I felt the book was written for the small number of officers like me with graduate degrees in policy. The descriptions of methods are too technical for military generalists but lack sufficient detail for academics. Consider this tortured description of fixed effects in a discussion of how they help clarify the relationship between aid and violence:

We had to isolate the effect of changes in aid. We did so by looking at the statistical relationship between changes in aid spending and changes in violence. This removed elements that can affect both aid spending and violence but do not change over time… Geography for instance, doesn’t change… [and] we need to account for trends in local attitudes.

Though the authors make significant efforts to explain their process, their explanation is insufficient for graduates of the professional military education system. (While commissioned officers must earn a college degree, neither professional military schools for mid-grade nor senior officers include statistics instruction.) Without more detailed instruction on the methods employed, most practitioners would gloss over this section and skip to the findings below. For this audience, the book would benefit from tables or a “one-slider” of takeaways and recommendations for action.

Conversely, this description’s lack of detail will disappoint scholars concerned with reproducibility. The authors describe the inclusion of geographical fixed effects and attitude controls, but imply there may be other variables like ethnicity or race they control for but do not mention in the description. Without knowing the type of model or variables used, fellow social scientists cannot determine whether these findings rest on solid science without referencing studies in the footnotes. While the authors back their work with references to peer-reviewed research, academics may desire greater methods discussion to allow independent judgement of the work without having to reach for references.

Despite these shortfalls, Small Wars, Big Data’s important contribution deserves recognition. The authors’ six principles should be enshrined in a revision of FM 3-24. Further, the techniques that the book validates, like improving cell phone access, warrant inclusion in doctrine focused on counter-insurgency tactics. Even if those manuals don’t get updated, Small Wars, Big Data should definitely make counter-insurgency reading lists.

As the United States reorients towards peer competition, focus on counter-insurgency has waned. Despite this shift in focus, American troops remain engaged in low-intensity conflicts from more than 800 bases around the world. Today’s troops deserve scientifically validated doctrine. Future troops need today’s best practices for the next time the United States engages in counter-insurgency. Though I’m not the first to call for updates to counter-insurgency doctrine, now is the right time and Small Wars, Big Data is the right book to guide that effort.


Zachary Griffiths is an Instructor in the Department of Social Sciences at West Point. He is also a U.S. Army Special Forces Officer and Resident Fellow at West Point’s Modern War Institute. He earned his MPP from the Harvard Kennedy School in 2017. He tweets at @z_e_griffiths.