Is the Terrorist Threat Declining? The Use and Abuse of Statistics

December 16, 2013

For special access to experts and other members of the national security community, check out the new War on the Rocks membership.

Earlier this month, terrorism analyst Peter Bergen wrote at CNN that the declining number of jihadists indicted in the United States demonstrates that the domestic terrorist threat has “markedly declined over the past couple of years.” His view is a counterpoint to the proclamations of Senate and House intelligence committee heads Sen. Dianne Feinstein (D.-Calif.) and Rep. Mike Rogers (R.-Mich.), who have claimed that the U.S. is no safer than it was in 2011. Who is right?

Bergen, the head of the National Security Program at the New America Foundation (NAF), contends that though Feinstein and Rogers might be on firm ground in arguing that al-Qaeda is resurgent in the Middle East, a NAF study of jihadist militants shows a substantial decline in the number of indicted extremists since 2010. Bergen contends that this establishes a declining domestic threat:

The total number of such indicted extremists has declined substantially from 33 in 2010 to nine in 2013. And the number of individuals indicted for plotting attacks within the United States, as opposed to being indicted for traveling to join a terrorist group overseas or for sending money to a foreign terrorist group, also declined from 12 in 2011 to only three in 2013.

Of course, a declining number of indictments doesn’t mean that the militant threat has disappeared. One of the militants indicted in 2013 was Dzhokhar Tsarnaev, who is one of the brothers alleged to be responsible for the Boston Marathon bombings in April. But a sharply declining number of indictments does suggest that fewer and fewer militants are targeting the United States….

In short, the data on al-Qaeda-linked or -influenced militants indicted in the United States suggests that the threat of terrorism has actually markedly declined over the past couple of years.

Here’s the interesting thing, to me, about Bergen’s analysis: it depends almost entirely on how one reads a sudden spike in homegrown terrorist cases that occurred in 2009-10. Exactly four years ago, in December 2009, Bergen’s view of the homegrown terrorist threat, based on the sudden rise in cases we were then experiencing, was that “there is no denying it is increasing.” He explained that a trend toward more homegrown jihadism “is just a fact,” since the phenomenon had “sort of grown exponentially in the last two years.” In other words, Bergen assessed at the time that the rising cases weren’t aberrant, but rather part of a trend of increased homegrown jihadist violence that would continue.

At the time, I disagreed in print with Bergen’s confidence that we were seeing a definite trend toward a persistently higher number of homegrown terrorism cases. Now that the 2009-10 spike in cases has receded, Bergen argues just as confidently that we are safer. I disagree with this conclusion, too. Much of our disagreement boils down to differences between my methodology of interpreting statistics and Bergen’s. In turn, this discussion has implications for broader efforts to assess U.S. counterterrorism policies: are we drawing the right lessons from the statistics and evidence that we gather, or are we being fooled by our own numbers?

The 2009-10 Spike in Homegrown Terrorist Cases

2009 saw almost twice as many people in the U.S. indicted for illegally supporting the jihadist cause as any previous year. According to NAF’s database of homegrown terrorism cases, there were 43 such cases in 2009, when the highest number in any other year since the 9/11 attacks had been 23, in 2003. The following year, in 2010, the number of homegrown jihadist terrorism cases declined to 34, but that still represented more such cases than any year but 2009.

As previously noted, Bergen viewed this sudden spike in homegrown terrorist cases at the time as an undeniable increase in the threat. I wasn’t so sure. In an article I published in the summer of 2010, I concluded that it wasn’t “clear that homegrown terrorism is increasing,” for two reasons. The first was that the perceived spike could be based on changes in policing strategies and tactics. If authorities started making arrests at a different point relative to a suspect undertaking illegal activities, that could artificially trigger perceptions of a major increase in homegrown terrorists; and so too could an increase in the number of sting operations. Second, I raised the possibility that this could be a statistical aberration:

Another possibility is that the current rash of homegrown terror cases is an aberration. In a statistical sequence measured over the course of years (such as weather patterns or a baseball player’s career), aberrant sequences will frequently arise. A spike or precipitous decline in numbers does not mean the numerical trajectory will extend indefinitely. For example, an unusually cold May does not mean that July will also be unusually cold. While in the middle of an unusual statistical sequence, it can be hard to have perspective; and in five years, 2009-2010 may seem exceptional in terms of the level of homegrown terrorist activity, rather than the beginning of a new trend.

Now that five years have passed since the onset of that spike in cases, it appears to have been just that, an aberrational sequence. And we can pinpoint the precise development that drove the 2009-10 rise in cases: the Somalia war.

In December 2008, the U.S. media first reported authorities’ discovery that more than a dozen young Somali men from Minnesota’s Twin Cities area (which has the U.S.’s largest Somali community) had disappeared, going abroad to join jihadist groups in Somalia. They decided to fight there after Ethiopia’s U.S.-backed invasion of Somalia in 2006, which was designed to shore up the country’s U.N.-recognized transitional federal government and push back its main adversary, the Islamic Courts Union. In addition to the young men being driven by nationalist sentiments, jihadist recruiters focused their efforts on the Twin Cities area—a somewhat unique dynamic for domestic terrorist cases, the vast majority of which do not feature recruiters from any established militant organization.

Thereafter, domestic law enforcement made apprehension of the young men who went to fight abroad, and the networks encouraging and supporting them, a top priority. Terrorism-related indictments increased as a result: the NAF dataset suggests that 14 indictments in 2009 and 16 in 2010 were related to the Somalia conflict. If you subtract these figures from the number of total indictments for both years, the numerical spike becomes less extreme, with only 29 indictments in 2009 and 18 in 2010 that were unrelated to the Somalia war. Though 2009 still would have a higher number of indictments than any year preceding it even with the adjusted figures, 2010 would be more in line with the numbers from previous years, featuring fewer terrorism indictments than either 2003 or 2006.

As recruiting for the Somalia conflict has declined, the number of indictments has also gone down. So the question remains: how do we interpret the lower numbers we are seeing now? Has the threat of homegrown terrorism “markedly declined,” as Bergen insists, or is there a better way to understand the number of homegrown jihadist cases that we have seen in 2012-13?

The Longer View

Fortunately, there has always been a relatively small number of homegrown jihadist terrorism cases in the United States. The fact that these numbers are small should make us hesitant to infer too much from numerical fluctuations. Take a look at the number of homegrown jihadists who have been indicted or killed by year, per the NAF database:

2002:            16

2003:            23

2004:            8

2005:            12

2006:            19

2007:            16

2008:            6

2009:            43

2010:            34

2011:            22

2012:            7

2013:            10

Bergen asserted that there had been nine indictments this year, but his article came out before Terry Loewen’s arrest, which pushes the number to ten. Looking at the full data by year, it’s not clear that there is a declining threat. Ten indictments in 2013 is three more than there were last year; it’s also a higher number than we saw in either 2004 or 2008. Indeed, there were only six such cases in 2008—the year before we saw the sudden jump to forty-three indictments.

In fact, rather than basing our assessment on indictments, there’s an entirely different statistic for measuring whether we face a declining threat: the number of people killed or injured by homegrown terrorists in the U.S. in any given year. That number was zero in 2011, while in 2013 three people were killed and 264 injured (in the Boston bombings).

The bottom line is that it’s perilous to infer too much from the data when the numbers in question are rather small, because small numbers makes it extraordinarily difficult to measure trends reliably. All it takes is one unusual development—such as the outbreak of war in Somalia, and its resulting impact on Minneapolis-St. Paul—to make it appear that everything has changed from a numerical perspective.

There is thus little proof that “the threat of terrorism has actually markedly declined over the past couple of years.” The core problem with Bergen’s use of statistics is evident when you compare his analysis in 2009 with his assessment today: his methodology is prone to perceiving a significant change in the level of threat based upon the direction that the numerical trend line is pointing at any given time. If the number of indictments doubled to twenty next year, by Bergen’s established methodology the threat would seem to be increasing again—even though the absolute numbers would still be lower than 2003, 2009, 2010, or 2011.

In attempting to determine whether we are grappling with an increasing or declining threat, it’s important to view the most recent data in as broad a context as possible. We should be wary of any method of statistical interpretation wherein temporary fluctuations in one direction or another can be mistaken for massive shifts in the threat we confront.

 

Daveed Gartenstein-Ross is a senior fellow at the Foundation for Defense of Democracies, an adjunct assistant professor in Georgetown University’s security studies program, and the author of Bin Laden’s Legacy (Wiley, 2011). He worked as a statistics tutor while an undergraduate at Wake Forest University.

 

Photo credit: thierry ehrmann

We have retired our comments section, but if you want to talk to other members of the natsec community about War on the Rocks articles, the War Hall is the place for you. Check out our membership at warontherocks.com/subscribe!

6 thoughts on “Is the Terrorist Threat Declining? The Use and Abuse of Statistics

  1. The other problem when looking at this question is a methodological one. These analysis rely on indictments which are probably not the best way to track terrorist plots. There have been numerous cases where terrorism related charges have been brought against a suspect only to be dropped because of lack of evidence, plea bargaining or tactical prosecutorial decisions. Likewise, there have been cases that (I suspect) most reasonable persons would classify as terrorism that don’t get terrorism related charges attached to them. Last years Fort Stewart-centered F.E.A.R. arrests appear to fall into that category.

    The number of plots also doesn’t appear to be a great way to determine the extent of the threat. The vast majority of plots we’ve seen over the past five (?), eight (?), ten (?) years have been amateurish affairs intercepted earlier and earlier in the operational cycle. Determinations of threat need to be based on both intent (# of plots) and capability (how likely those intents can reasonably be expected to translate to successful action).

  2. Here’s another thought: how many people in our country are killed by terrorist attacks each year compared to deaths from car accidents or cancer? Do deaths from terrorism even reach the double digits? While it’s understandable that terrorist attacks are more dramatic and provoke our outrage far more than “mundane” threats to life such as car accidents, perhaps it’s worth asking whether we’re throwing a disproportionate amount of our nation’s resources at a problem that isn’t that big once the numbers are stacked up?

  3. Peter Bergen is a journalist and has no business at all blathering his useless opinions about something he doesn’t understand. Peter probably personally helped Osama arrive at the conclusion that Americans will hear what they want to hear. Why does anyone take a journalist’s opinion seriously? Would you ask Peter how to conduct heart surgery? Warfare is as serious as the medical profession. I watched Peter’s debate with the men from the Long War Journal… his arguments boiled down to “do you know who I am”…”I interviewed Usama after all”. Malarkey. I’m sure he’s made lots of money from his snake oil sales though. How many of those indictments were issued in the 12 months before 9/11/01… and what did that say about the threat?

    1. Matt: Please avoid making ad hominem attacks in the comments section. Peter Bergen is a journalist as well as being a valued and incisive analyst of al Qaeda and South Asia. Peter is also one of the most kind and humble men in Washington.

  4. Daveed, the question should be when will terrorist attacks become more numerous. With Islam invading the country and applying “stealth jihad” tactics until they gain strength, terrorism will gain in number as Islam’s strength grows in country. Just look at Europe. Terrorism is a way of life there and the liberal “leadership” does not know how to handle the problem.

  5. Yes I called him out on a professional level for being wrong. Good journalists don’t make themselves part of the story. One of the lessons of 9/11/01 was to challenge the temptation to downplay the threat as was done before 9/11. Facing the facts saves lives even if it hurts a few feelings. I watched Peter make his argument and I honestly believe he had not done his homework or was making it up because what he was saying was so far off the mark. If a doctor was consistently way off the mark in his profession you better believe he would be called out eventually. This is a deadly serious issue also.