Is the Terrorist Threat Declining? The Use and Abuse of Statistics
Earlier this month, terrorism analyst Peter Bergen wrote at CNN that the declining number of jihadists indicted in the United States demonstrates that the domestic terrorist threat has “markedly declined over the past couple of years.” His view is a counterpoint to the proclamations of Senate and House intelligence committee heads Sen. Dianne Feinstein (D.-Calif.) and Rep. Mike Rogers (R.-Mich.), who have claimed that the U.S. is no safer than it was in 2011. Who is right?
Bergen, the head of the National Security Program at the New America Foundation (NAF), contends that though Feinstein and Rogers might be on firm ground in arguing that al-Qaeda is resurgent in the Middle East, a NAF study of jihadist militants shows a substantial decline in the number of indicted extremists since 2010. Bergen contends that this establishes a declining domestic threat:
The total number of such indicted extremists has declined substantially from 33 in 2010 to nine in 2013. And the number of individuals indicted for plotting attacks within the United States, as opposed to being indicted for traveling to join a terrorist group overseas or for sending money to a foreign terrorist group, also declined from 12 in 2011 to only three in 2013.
Of course, a declining number of indictments doesn’t mean that the militant threat has disappeared. One of the militants indicted in 2013 was Dzhokhar Tsarnaev, who is one of the brothers alleged to be responsible for the Boston Marathon bombings in April. But a sharply declining number of indictments does suggest that fewer and fewer militants are targeting the United States….
In short, the data on al-Qaeda-linked or -influenced militants indicted in the United States suggests that the threat of terrorism has actually markedly declined over the past couple of years.
Here’s the interesting thing, to me, about Bergen’s analysis: it depends almost entirely on how one reads a sudden spike in homegrown terrorist cases that occurred in 2009-10. Exactly four years ago, in December 2009, Bergen’s view of the homegrown terrorist threat, based on the sudden rise in cases we were then experiencing, was that “there is no denying it is increasing.” He explained that a trend toward more homegrown jihadism “is just a fact,” since the phenomenon had “sort of grown exponentially in the last two years.” In other words, Bergen assessed at the time that the rising cases weren’t aberrant, but rather part of a trend of increased homegrown jihadist violence that would continue.
At the time, I disagreed in print with Bergen’s confidence that we were seeing a definite trend toward a persistently higher number of homegrown terrorism cases. Now that the 2009-10 spike in cases has receded, Bergen argues just as confidently that we are safer. I disagree with this conclusion, too. Much of our disagreement boils down to differences between my methodology of interpreting statistics and Bergen’s. In turn, this discussion has implications for broader efforts to assess U.S. counterterrorism policies: are we drawing the right lessons from the statistics and evidence that we gather, or are we being fooled by our own numbers?
The 2009-10 Spike in Homegrown Terrorist Cases
2009 saw almost twice as many people in the U.S. indicted for illegally supporting the jihadist cause as any previous year. According to NAF’s database of homegrown terrorism cases, there were 43 such cases in 2009, when the highest number in any other year since the 9/11 attacks had been 23, in 2003. The following year, in 2010, the number of homegrown jihadist terrorism cases declined to 34, but that still represented more such cases than any year but 2009.
As previously noted, Bergen viewed this sudden spike in homegrown terrorist cases at the time as an undeniable increase in the threat. I wasn’t so sure. In an article I published in the summer of 2010, I concluded that it wasn’t “clear that homegrown terrorism is increasing,” for two reasons. The first was that the perceived spike could be based on changes in policing strategies and tactics. If authorities started making arrests at a different point relative to a suspect undertaking illegal activities, that could artificially trigger perceptions of a major increase in homegrown terrorists; and so too could an increase in the number of sting operations. Second, I raised the possibility that this could be a statistical aberration:
Another possibility is that the current rash of homegrown terror cases is an aberration. In a statistical sequence measured over the course of years (such as weather patterns or a baseball player’s career), aberrant sequences will frequently arise. A spike or precipitous decline in numbers does not mean the numerical trajectory will extend indefinitely. For example, an unusually cold May does not mean that July will also be unusually cold. While in the middle of an unusual statistical sequence, it can be hard to have perspective; and in five years, 2009-2010 may seem exceptional in terms of the level of homegrown terrorist activity, rather than the beginning of a new trend.
Now that five years have passed since the onset of that spike in cases, it appears to have been just that, an aberrational sequence. And we can pinpoint the precise development that drove the 2009-10 rise in cases: the Somalia war.
In December 2008, the U.S. media first reported authorities’ discovery that more than a dozen young Somali men from Minnesota’s Twin Cities area (which has the U.S.’s largest Somali community) had disappeared, going abroad to join jihadist groups in Somalia. They decided to fight there after Ethiopia’s U.S.-backed invasion of Somalia in 2006, which was designed to shore up the country’s U.N.-recognized transitional federal government and push back its main adversary, the Islamic Courts Union. In addition to the young men being driven by nationalist sentiments, jihadist recruiters focused their efforts on the Twin Cities area—a somewhat unique dynamic for domestic terrorist cases, the vast majority of which do not feature recruiters from any established militant organization.
Thereafter, domestic law enforcement made apprehension of the young men who went to fight abroad, and the networks encouraging and supporting them, a top priority. Terrorism-related indictments increased as a result: the NAF dataset suggests that 14 indictments in 2009 and 16 in 2010 were related to the Somalia conflict. If you subtract these figures from the number of total indictments for both years, the numerical spike becomes less extreme, with only 29 indictments in 2009 and 18 in 2010 that were unrelated to the Somalia war. Though 2009 still would have a higher number of indictments than any year preceding it even with the adjusted figures, 2010 would be more in line with the numbers from previous years, featuring fewer terrorism indictments than either 2003 or 2006.
As recruiting for the Somalia conflict has declined, the number of indictments has also gone down. So the question remains: how do we interpret the lower numbers we are seeing now? Has the threat of homegrown terrorism “markedly declined,” as Bergen insists, or is there a better way to understand the number of homegrown jihadist cases that we have seen in 2012-13?
The Longer View
Fortunately, there has always been a relatively small number of homegrown jihadist terrorism cases in the United States. The fact that these numbers are small should make us hesitant to infer too much from numerical fluctuations. Take a look at the number of homegrown jihadists who have been indicted or killed by year, per the NAF database:
2002: 16
2003: 23
2004: 8
2005: 12
2006: 19
2007: 16
2008: 6
2009: 43
2010: 34
2011: 22
2012: 7
2013: 10
Bergen asserted that there had been nine indictments this year, but his article came out before Terry Loewen’s arrest, which pushes the number to ten. Looking at the full data by year, it’s not clear that there is a declining threat. Ten indictments in 2013 is three more than there were last year; it’s also a higher number than we saw in either 2004 or 2008. Indeed, there were only six such cases in 2008—the year before we saw the sudden jump to forty-three indictments.
In fact, rather than basing our assessment on indictments, there’s an entirely different statistic for measuring whether we face a declining threat: the number of people killed or injured by homegrown terrorists in the U.S. in any given year. That number was zero in 2011, while in 2013 three people were killed and 264 injured (in the Boston bombings).
The bottom line is that it’s perilous to infer too much from the data when the numbers in question are rather small, because small numbers makes it extraordinarily difficult to measure trends reliably. All it takes is one unusual development—such as the outbreak of war in Somalia, and its resulting impact on Minneapolis-St. Paul—to make it appear that everything has changed from a numerical perspective.
There is thus little proof that “the threat of terrorism has actually markedly declined over the past couple of years.” The core problem with Bergen’s use of statistics is evident when you compare his analysis in 2009 with his assessment today: his methodology is prone to perceiving a significant change in the level of threat based upon the direction that the numerical trend line is pointing at any given time. If the number of indictments doubled to twenty next year, by Bergen’s established methodology the threat would seem to be increasing again—even though the absolute numbers would still be lower than 2003, 2009, 2010, or 2011.
In attempting to determine whether we are grappling with an increasing or declining threat, it’s important to view the most recent data in as broad a context as possible. We should be wary of any method of statistical interpretation wherein temporary fluctuations in one direction or another can be mistaken for massive shifts in the threat we confront.
Daveed Gartenstein-Ross is a senior fellow at the Foundation for Defense of Democracies, an adjunct assistant professor in Georgetown University’s security studies program, and the author of Bin Laden’s Legacy (Wiley, 2011). He worked as a statistics tutor while an undergraduate at Wake Forest University.
Photo credit: thierry ehrmann
