A password will be e-mailed to you.
Hide from Public

Are We Winning Yet? Led Astray By Metrics

October 2, 2013

How do you know if you’re winning a war? It turns out it is much more complicated than you’d expect.

I recently returned from the Naval Postgraduate School in California where I met with other scholars to put together a book on “assessing war” – the efforts that militaries or other fighting forces make to determine whether they are winning or losing a war.  I left California seeing the problem in quite a different way from what I’d expected or from what the organizers had intended.

Early discussions about a theoretical framework suggested a common way of assessing our historical case studies: identify the political goals that were to be obtained by the war, then identify the benchmarks that the military in question used to measure its progress toward that goal.  Then we were to look at the information that the military collected to determine if those benchmarks had been reached.  Finally, we were to consider the incentives that the desire to collect this information created for the military.  Often those incentives were perverse, as in the case of the body counts in Vietnam.

Each author gave a brief summary of his or her topic and as I listened to historians talk about assessment within most American wars from the civil war through today, it struck me that the historians were describing very quantitative data-heavy methods of assessment for the wars since Vietnam.

We are past the time when progress could be measured by looking at the movement of a front line on a map.  However, at some point the United States military—like the rest of the country—fell in love with data.  Lots of data.  Data in the form of numbers: munitions expended, body counts, percentage of the country with electricity, poll numbers, number of dollars spent on development projects, etc.  The U.S. military loves the word “metrics.” Metrics, of course, allow us to measure things and we believe that anything can be measured.  So now the military gathers a whole slew of data that it can use to measure things so we can understand what is going on.  The problem is, we still don’t understand what is going on.  One of the organizers of the conference mentioned that in 2009 he had been offered access to a drive containing “metrics” data on the Afghanistan war… 47 terabytes of data.

As I sat in that room in Monterey, I got to thinking about the distinction in the intelligence business between secrets and mysteries.  Secrets are questions to which there is a factual answer.  An example is “Where is Ayman al-Zawahiri?”  There is an answer to that question, we just don’t know what it is yet.  By contrast, mysteries are questions to which there is no factual answer.  An example might be “What will Ayman al-Zawahiri do next week?”  (Note that this is quite different from “What does Ayman al-Zawahiri intend to do next week?”)  There is no factual answer to this question because it depends on future events, including interaction with other human beings, and the future is always in motion.

The way to find secrets is to collect more data and somewhere in the mass of data will be the secret or pieces of a secret which can be assembled like a puzzle.  In the case of mysteries, however, collecting more data is typically the wrong thing to do.  More data often makes it impossible to see the forest for the trees.  Instead the answer is to bring in experts, people who have great experience and insight.  Of course, the United States Government has managed to badly debase that term.  However, with a great deal of luck experts can be found who have coup d’oeil.  This is a characteristic that Clausewitz described in On War as “the rapid discovery of a truth which to the ordinary mind is either not visible at all or only becomes so after long examination and reflection.”  Not everybody has this ability.  Napoleon certainly had it and it was that fact that made him the transcendent military commander in Clausewitz’ eyes.  Clausewitz explained:

When all is said and done, it really is the commander’s coup d’œil, his ability to see things simply, to identify the whole business of war completely with himself, that is the essence of good generalship. Only if the mind works in this comprehensive fashion can it achieve the freedom it needs to dominate events and not be dominated by them.

I believe that the United States military, indeed, the entire Government, has allowed itself to be dominated not by events so much, perhaps, but by hubris, the belief that its legions of staff officers can crunch the numbers, connect the dots and come up with a scientific answer to any question that it desires to answer.  In short, it has has treated the question of assessing wars like Vietnam or Afghanistan, the question of “Are we winning?” as a secret.

Instead, perhaps the question of whether we are winning is a mystery.  In short, perhaps there is no factual answer to the question.  This corresponds with one of the most important things that we know about war: it is interactive.  It is a duel.  Or, to use a different metaphor, the enemy gets a vote.  Indeed, in the kinds of wars we have fought recently the enemy not only gets a vote but multiple enemies get votes as do various factions of the civilian population.

If I am correct that the assessment of the kind of wars that the United States fights these days is a mystery, then how can that assessment be done?  If the commander does not have coup d’oeil—and most do not, including many of the most capable—then he will need assistance.  In any event, even a commander with this rare skill probably should have assistance.  Force commanders, like all leaders, are charged with getting complicated and difficult things done and they will inevitably see the situation through the lens of their own optimistic intentions.  This is normally a functional behavior.  After all, taxpayers pay them to be Pattons not Hamlets.  However, it also inclines them toward motivated bias—seeing what they want to see.  (Note that motivated bias is not intellectual dishonesty.  It is simply an analytic pathology.)

There is an important role for outside experts—genuine experts—who are not in a command position and who do not have the force commander in their chain of command.  These might be commanders from previous wars who know what winning and losing feel like, they might be respected strategists, experienced intelligence analysts, historians, sociologists, perhaps even investigative journalists who have a sixth sense for BS or a good story.  Note that these experts cannot be exclusively intelligence experts because this is a problem of net assessment.  This type of assessment entails understanding the interactions of three things: the enemy (normally under the purview of intelligence analysts), the friendlies (normally under the purview of the commander), and all the other actors in the battlespace (nominally under the purview of intelligence analysts but frequently ignored).

It’s a complicated job, to put it mildly.  This is why the experts should not be force fed a diet of PowerPoint slides, or buried under 47 terabytes of data.  They could have access to all the data they wanted, but it shouldn’t be inflicted upon them.  Rather, they should be left alone to, as has been written of Ulysses S. Grant, comprehend problems “in all their simplicity.”

What does this process look like?  Who knows?  As a matter of psychological reality and of the structure of these problems (they are mysteries without demonstrably correct answers), the outputs can’t be fully justified in a logical or mathematical sense.  Is this a weakness?  We Americans are probably tempted to say that it is.  Yet, that is us looking through our 21st century cultural blinders.  Clausewitz would scoff at us.

With 47 terabytes of data, any drone can produce a convincing case for pretty much anything.  A wise commander would prefer the judgment of genuine experts with insight even if they can’t enunciate why they know what they think they know, even if they can’t show their work.

 

Mark Stout is a Senior Editor at War on the Rocks. He is the Director of the MA Program in Global Security Studies at Johns Hopkins University’s School of Arts and Sciences in Washington, DC.

Leave a Reply

Your email address will not be published. Required fields are marked *

15 thoughts on “Are We Winning Yet? Led Astray By Metrics

  1. Mark,

    Good article. “Big Data” is great if you are a credit card company and planning your next marketing campaign. Not so much for some other applications.

    As for knowing whether you are winning or losing, we were taught this:
    1. If you are making decisions and the opposition has to react, you are winning.
    2. If the other side acts and you are forced to react, you are losing.
    3. If you are heavily dependent on intelligence to determine your next actions, you are likely on the defensive and therefore are losing.
    4. If the other side is depending on its intelligence to find a way to react, they are losing.
    5. Momentum – if you have it, you are winning. If not, you are losing.

    I do not have access to that 47 terabytes of data on Afghanistan, so I am forced to fall back on simple (but not simplistic) analysis. The mission in Afghanistan was winning from October 2001 until Jan/Feb 2002. American and allied forces were on the move and acting. AQ and the Taliban were reacting and fleeing. Since then, it has been a static war with no objectives capable of being met by military force. The Taliban (et al) acts and the coalition has to react. Therefore: losing.

    Iraq: Americana and allied forces were acting from 19 March to May of 2003 and were by definition winning. After that, the coalition forces had no assets or clear goals on how to stabilize Iraq and they slowly shifted from ‘acting’ to ‘reacting’ and therefore by definition were losing. After that, it was just finding a way out for the most part.

    Sorry this is not more complex or sophisticated, but I do not have a computer science degree and I refuse to believe that collecting more data can tell you if you are winning or losing. You should be able to tell whether you are winning or not by looking out the window at the battlefield around you.

    1. Tom: I think you make an excellent point about initiative. I rather suspect that that is one of those non-quantifiable (ultimately quite elusive) things that every real commander feels and knows that would, indeed, say a great deal about who is winning.

      1. I would caution against mistaking action for initiative. Just because we are doing things and the enemy is reacting to them, does not mean we are doing the “right” things. So you could have all the initiative in the world but if the enemy is mostly just sitting back and allowing you to exhaust yourself, it has not brought you any closer to victory.

        It may, in fact, be better to wait. To take time to react to the enemy, in order to get a better sense of what is going on and how their actions and ours shape the battlespace/human terrain. It will not be very satisfying and in the press it will look like “losing” but it may be far more effective than simply “seizing the initiative” and “forcing” the enemy to react.

  2. Isn’t there a difference between winning a battle and winning a war? Large amounts of data might tell you who won the battle. Knowing whether you’ve won a war seems far more straightforward. What war has ever been won without the leader of the nation or group surrendering or being killed for all to see? The war is being waged by people after all. General Lee surrendered. Hitler killed himself and his next in line surrendered. Lincoln could’ve lost the war if he had changed his mind and agreed to defeat. Mullah Omar has not surrendered and has not been defeated. Zawaharry has taken over leadership of AQ and has not surrendered. If Obama came on national television and announced we were giving up the fight and going home everyone in the world would know the USA lost the war. If Zawaharry announced AQ was abandoning the fight and asked his fighters to lay down arms everyone would know we won. It is certainly possible low ranking individuals could step in and take on the fight but they would by definition become the leader from that point though they would probably have less resources and less of a chance at victory. If the US could identify what Zawaharry and Mullah Omar consider intolerable losses than we would have a chance at defeating AQ in this war. We really need to grab them by the balls and squeeze until they cry for mercy. Leadership whom decide whether to fight in the first place are solely responsible for choosing to continue the war effort. Leadership is decisive.

  3. Your line about Secrets and Mysteries sounds like what Professor McLaughlin at SAIS taught and which is a very real issue as we over utilize technical means to collect and analyze data without paying sufficient time to ask the “So What” of that data and how does it drive ours or our enemies’ motives.

    1. John McLaughlin is a bit of a legend and you are fortunate to have had him as a professor. I used to work for him some echelons down the food chain and I and my colleagues all thought very highly of him.

      However, I believe it was Gregory Treverton of RAND (and various other places) who came up with the secrets vs. mysteries dichotomy. It’s a great insight and one that I use in my own intelligence courses that I teach for Johns Hopkins across the street from the SAIS building.

  4. Great post, Mark. Agree on all points. If we aren’t able to look past our reductive, analytical mindset we’ll know everything about every damn tree but never see the forest, let alone understand the ecosystem. I believe Christopher Paul at RAND is working on a report on metrics – will be interested to see what they come up with.

  5. Great post on what is very much a live topic among those of us involved in teaching and training officers (in my case, Marine ground ops and intelligence officers) to plan and execute operations. Assessment is an absolutely integral part of this, and we suck at it. Anyone interested in the subject should read Ben Connable’s “Embracing the Fog of War,” a RAND monograph on assessment in COIN. (http://www.rand.org/content/dam/rand/pubs/monographs/2012/RAND_MG1086.pdf)
    I would argue that poor assessment methodology (seen these days most obviously in the obsession with quantitative data mentioned in Dr. Stout’s post, but also noticeable in the proliferation of assessment frameworks advanced and used in parallel by a bewildering variety of agencies and sub-elements of agencies) is almost always a direct reflection of strategic and operational confusion. If you cannot truly articulate your ends and how the ways and means you propose to employ will get you to them, you are going to have a hell of a time thinking through what you might see or not see that would indicate progress, or its opposite, much less how you might explain your logic up the chain to more senior military and political leadership.

  6. I very much enjoyed the secret versus mystery framing device in your discussion whether we are winning or losing along the frontier. I proposed a while back that we should just change our success criteria (Proceedings: Metrics Impossible, July 2011, Vol. 137/7/1.301) instead of wrestling with incompatible metrics in our dealings in Afghanistan.

    A number of thoughts regarding measures of effort and measures of effectiveness in Afghanistan. First, it appears to me that we are attempting to create a balance-of-power system under the guise of a representational republic in Afghanistan. The challenge we face is that a Pashtun Empire has been the typical mode of governance in Afghanistan. I define empire in this case as an extensive assembly of territories and ethnic groups under a single supreme authority, esp. a Khan, or an Amir al Mumineen, etc.

    Second, Clausewitz stressed the notion that strategic planning and strategy in general rests on tactical success alone because only tactical success can produce favorable strategic outcomes. Strategy grows silent without tactical victory. Our COIN experiences in Vietnam, Iraq and Afghanistan surely turns this notion on its head. We revel in tactical successes but are unable to achieve strategic victory. Maybe in COIN, you have to lose to win. What is the Taliban defeat mechanism? General Allen explained that it was local government forces. Maybe it is more than just local security forces. I propose it is local forces and the local deal that defeats the local insurgency. When we study Afghan military history a discernible pattern of behavior emerges, which by the way, is very rarely, if ever included in our metrics for determining victory or defeat. A pattern emerges in which networked forms of governance, based on local deals, merge over time into Empire. The Pashtun are expert, if history is any indication, at consolidating power over disparate groups and territories. What can we learn from Pashtun military history to improve our metrics?

  7. I greatly appreciate both Mark Stout’s article and the opinions expressed above. Whereas Mark Stout clearly implies it however, I believe we should go further and state explicitly that the use of Clausewitz’s concept of coup d’oeil is inappropriate; an impossible construct, when applied to complex adaptive systems, to modern, multidimensional, interconnected conflict. Neither is another of Clausewitz’s hallowed concepts – the “Center of Gravity” …but that is another rant.

    A commander’s coup d’oeil has been long viewed as the Holy Grail, the paragon of military operations. As such, it doesn’t just influence, it DRIVES the military’s view toward operations and its assumptions. Indeed it is a fundamental element of Western military culture.

    Modern conflict – this “spectrum of conflict” we face, is NOT the set piece, terrain dominated battles of the Napoleonic age. Napoleon could see and understand – “at a glance,” that is until battles became increasingly protracted and complex affairs. Indeed, near the end of the era of Napoleonic warfare, history reveals Clausewitz’s “God of War” straining increasingly to comprehend the scope of war and control the pace of events as they unfolded.

    Moreover, it isn’t just an issue of “data,” or “secrets” VS “mysteries,” these are representative of deeper issues associated with our worldview and our culture that more fundamentally DRIVE our thinking… Thanks. KAS

    1. I was going to say this exact same thing. Maybe not as well but the same thoughts. Comparing military commanders who dealt with conventional enemies on conventional battlefields with modern commanders who are dealing with political situations on unconventional terrain is comparing apples to tractors. It does not help to identify failures in training or new techniques to be employed.

  8. I would argue that the issue isn’t the metrics so much as the understanding of “victory.” As you point out, the assumption is that victory will appear to be the establishment of a Republic Democracy in Afghanistan. But that’s not really victory for two reasons.

    1) You could create this modern, westernized democracy and still end up with a government that hates the US and is willing to support terrorists groups against us. Or at least turn a blind eye to their existence in favor of not having to deal with the terrorists themselves.

    2) It may not be possible at all to achieve this form of government but that may not matter. Do we *need* a democracy in place to win or do we *need* a government that will prevent terrorists from attacking the US. This comes down to an issue of clarity regarding the ultimate desired endstate.

    If you can accept that your endstate is a more of an idea than a fixed point, then you will be willing to accept a much larger range of options. It may in fact be possible to have a Theocracy in power so long as it secures its denied territory. Or Afghanistan may disolve into several smaller states with no strong central government but each area is strictly controlled and there is no refuge for terrorists. Or you could install an iron-handed dictator who will (at least for the next 30 or so years) keep the population in check and therefore Americans safe. This last case particularly highlights the idea that victory may not ever be a final “we’re done” so much as a “we have won…for now” and you may in fact have to readdress a situation later.

    So if you can accept all of that, then you find that metrics are pretty much meaningless with regards to the overall victory. You may simply “back into” a victory by letting things happen without directly causing them to happen. But if it meets your overall *true* goal (Americans safe) instead of some “assumed” goal (a Westernized democracy), then it doesn’t really matter.

    Metrics, however, can still be used to identify smaller victories. Reducing the number of insurgent attacks gives time for a solution to form, even if it is not the solution you envisioned. So if all you count are the reduction in attacks, even if this doesn’t tell you how close you are to a solution, it is still an important metric.

  9. Good article. In our quest to continue to maintain funding for wars, technology programs, etc we have developed ways to justify our costs and actions that can be briefed to our elected officials who ultimately, and rightly make the decision to fund the DOD.

    I agree with the author’s premise that often times we lose the forest for the trees. To help the commander achieve the “coup d’oeil” is a combination of factors, no one singular panacea. Using data to identify patterns that both enemy and friendly are utilizing, his feeling about how the situation feels on the deck, his and his staff’s knowledge about the history of the AO in which he is operating.