What is Policy Relevance?
Editor’s Note: This is the latest article in our special series, “The Schoolhouse.” The aim of this series is to explore and debate the state of advanced graduate education in international affairs. We aim to move beyond the often-repetitive and tiresome debates about the usefulness of scholarship to policy. We believe there are deeper issues at stake.
Attempts to bridge the gap between academia and the real world, especially in the area of international relations, have accelerated over the last few years. There are several initiatives ongoing, including the Bridging the Gap project, the Tobin Project, the Institute for the Theory and Practice of International Relations (TRIP), and a variety of others. The Bridging the Gap project, for example, trains academics how to ask policy relevant questions and write for public policy audiences. To complement these efforts, our understanding of what policy relevance “is” needs improving. In particular, distinguishing between research with significance for policy and research that is policy actionable — promoting realistic policy actions — can bring analytical clarity to the concept of policy relevance and help enhance efforts to bridge the gap.
A fear that academia is becoming more irrelevant drives these efforts to bridge the gap. From former MacArthur President Robert Gallucci’s call for more relevance to New York Times columnist Nicholas Kristof’s fear that academic insights are not useful for the real world to writing by Peter Campbell and Michael Desch, Steve Van Evera, and Jim Steinberg and Frank Gavin, along with others, there is a sense that academics are not doing enough. On the other hand, the information age has been a game changer in the ability of academics to communicate directly with the policy world, and increasingly in their willingness as well. Just in the area of international relations, from blogs run by academics (e.g. Washington Post’s Monkey Cage blog, Duck of Minerva, Political Violence at a Glance, Lawfare, and others) to social media avenues such as Twitter to media outlets that consistently publish academics with policy insights (e.g. War on the Rocks, Foreign Policy, and Defense One), academics have a greater number of venues than ever to publish policy-relevant insights based on their scholarship. Anecdotally, it seems easier to take advantage of the opportunity than ever.
Yet, if improving policy relevance in academia, as well as the ability of the policy world to consume academic research (if presented correctly), is an important goal, it makes the question of what exactly policy relevance “is” a vital one. Policy relevance often seems to be in the eyes of the beholder. A lack of precision in what policy relevance is, however, can encourage thinking about the issue that does not accurately capture reality, and leads to confusion when different people use the term in different ways.
One way forward is to distinguish between four key concepts that seem at the core of discussions of policy relevance. What follows separates the components of policy relevance from the quality of the work itself, e.g. whether it holds together intellectually. While that is a crucial consideration for evaluating the overall utility of any piece, this is an attempt to drill down on the concept of policy relevance itself.
Policy significance: Policy significance, most simply, is research that has implications for the policy world. A good deal of international relations research, even that involving advanced statistical methods and lots of academic jargon, arguably passes this bar. For example, Paul Staniland points out, in a response to Thomas Ricks, the utility of many recent political science articles and books for understanding topics ranging from how rockets launched from the Gaza Strip influence voting patterns in Israel to what we can learn about the emerging balance of power in East Asia from studying the period before the outbreak of World War I.
A critic might respond, however, that many of these pieces are written in ways that make them inaccessible to mainstream policymakers, or that they are too long for policymakers to read. That is often true. As Paul Avey and Michael Desch’s research on how policymakers view academic journals shows, policymakers doubt the utility of academic journal articles featuring quantitative work and game theory (whether any senior policymakers read any journal articles, regardless of method, is a different question).
Thus, the potential significance of research and writing for understanding a public policy issue alone is only part of the issue, though it is a useful sub-category.
Policy accessibility: Policy accessibility reflects the essential readability of research — whether presented as a book, journal article, or blog post — for the policy world. A journal article and blog post by the same person on the same issue could be just as policy relevant as each other, but have very different levels of policy accessibility. The journal article might feature many theoretical references and discussions of academic debates that make it policy relevant, but not policy accessible. The blog post, in contrast, on the exact same research, might be policy accessible.
One might even further refine this concept by distinguishing between writing that is policy accessible at the working level in the government, which is often staffed by younger people with more recent engagement with academic research, and writing for senior policymakers. Senior policymakers have even less time than their busy staffers to consume outside writing, and, unless they come from academia, potentially less familiarity with academic concepts.
Another way to refine the concept of policy accessibility is to think about different venues as having different levels of policy accessibility for different audiences. Even within the policy world, venues more targeted at the Defense Department (DoD) versus the public conversation on an issue might require writing in somewhat different ways. Demonstrating fluency with DoD jargon and policy levers can be important for being accessible to busy staffers in a DoD office. In contrast, writing a mass media op-ed requires using more public language designed for the broadest audience.
Policy actionability: Policy actionability refers to a recommendation that is possible to implement for the target of the recommendation. Most academic work is not policy actionable, fundamentally. For example, implications from international relations research are things such as whether countries with high male-to-female ratios are more likely to start military conflicts or that countries that acquire nuclear weapons become harder to coerce. Understanding the motivations for countries to have nuclear weapons has broad policy implications, but it is not a specific thing that is going to drive U.S. policy on Iran or North Korea, except in the broadest way.
Academic research of this type shows what happens, on average. For research that is only policy significant, however, policymakers then figure out whether the particular case they are dealing with, e.g. Russia’s invasion of Eastern Ukraine, fits the baseline and gives them useful information. That is to say, the policymaker is taking policy significant research and making it policy actionable. For example, understanding that democracies generally do not fight each other, the phenomenon known as the democratic peace, could shape policymaker expectations about whether a given dispute between two democracies will escalate into violence, all other things being equal, and then influence a policy response.
For example, Peter Feaver and Eric Lorber’s piece on Obama’s Iran strategy and the role of economic sanctions is clearly policy significant and written in an accessible way. It points to challenges involved in a sanctions “snapback” against Iran if nuclear talks between Iran and the P5+1 go south, but does not recommend a specific course of action for the Obama administration. Its larger contribution is to the public debate on the topic.
In a different way, some policy relevant writing is actionable in theory, but not in practice. Arguing that the United States should withdraw from North Atlantic Treaty Organization (NATO), for example, is a specific policy recommendation, but it is not a realistic recommendation given the strong commitment of the United States to remaining in NATO. It can be useful for agenda setting or dialogue generating, however.
Policy actionable writing is different. Policy actionable writing, taking political and bureaucratic realities into account, generally does one of two things. First, policy actionable writing does the translation-function itself. It explains, for example, investment decisions the U.S. military should make based on lessons learned from research on military innovation. Second, policy actionable writing engages directly in a debate such as whether the United States should deploy ground troops to confront the Islamic State in Syria and the Levant or arm Ukraine, drawing on academic research or concepts to make the argument in favor or against.
Public debate: Finally, academic writing in the public sphere can also help set the agenda for discussion, even though it is not necessarily designed to influence short-term public policy activities. Books and articles by scholars such as Paul Kennedy or Samuel Huntington exemplify this type of agenda-setting research. This type of writing can affect how policymakers think about various essential issues and evaluate the world when they make decisions, but does not explicitly guide any particular choices.
For a recent example of an academic engaging public debate in a shorter format than a book, take John Mearsheimer’s article in Foreign Affairs on Western responsibility for ongoing Russian aggression in Ukraine. His provocative suggestion that the United States should work with Russia to make Ukraine a neutral buffer state stimulated public dialogue, even though it was unlikely to be implemented, given the commitment of the Obama administration (and any other likely U.S. administration) to pushing back against Russian aggression.
Gaining a better sense of what exactly constitutes policy relevance is a useful and arguably vital task. Too often, the debate about policy relevance devolves into rhetorical bomb throwing where it is entirely possible that both sides are right, depending on the metrics used and the definition of policy relevance used. No one wants academia to whither into obscurity. In reality, some writing might score highly in some of these areas, but lower in others, while other research might score highly across the board, and so on. Distinguishing between the significance, accessibility, actionability, and public debate functions of writing can provide a clearer understanding of what constitutes policy relevance. This framework can aid the field of international relations, as well as others, in moving the discussion forward and promoting policy relevance that we can all get behind.
Michael C. Horowitz is an Associate Professor of Political Science at the University of Pennsylvania. You can follow him on Twitter @mchorowitz.
Photo credit: DrKenneth
