In Defense of Best Practices
At the height of Operations Iraqi Freedom and Enduring Freedom, I worked as part of a contract team providing intelligence support to U.S. Central Command. Our team’s job was to help document how exactly the U.S. Central Command Intelligence Directorate provided key intelligence support to operations in Iraq and Afghanistan. We examined how Central Command performed its functions — how data was received, integrated, analyzed, disseminated, and used from Tampa to Baghdad to Kabul.
To share these findings, my team created several documents that we titled the “USCENTCOM Intelligence Directorate Best Practices.” Other commands looking to model Central Command’s intelligence support processes read and shared these documents. Unfortunately, these other commands did not have teams of dedicated process analysts, so cross-command process comparisons were rare.
This level of understanding, documentation, and sharing of process should be encouraged across the entire intelligence community. Every intelligence organization should follow the first recommendation a House of Representatives joint task force gave to Central Command following the accusation of distorted intelligence about Islamic State: “Formalize and Document its Analytic Processes and Procedures.” Encouraging community-wide documentation and process awareness will help the best practices of the intelligence community rise to the top and increase efficiency and return on investment.
Each organization and mission in the intelligence community should understand what it does, how it operates, and how it compares to similar organizations and missions. And, when processes are similar, the intelligence community should look to establish community-wide best practices.
Last year, in a War on the Rocks article entitled “Boxed In: The Bad Side of Best Practices in Intelligence,” National Intelligence University Provost Dr. Debora Pfaff discouraged the use of best practices in the intelligence community. Pfaff argued that best practices limit creativity and bind intelligence practitioners to checklists, citing the speed and volume of global information as a reason intelligence personnel should not be bound to predetermined processes.
Pfaff makes a number of important arguments about the downsides of dictating processes via best practices. But my experience writing and sharing intelligence processes has suggested one important advantage that her article discounts: using best practices for learning and awareness, both within and across organizations. A well-developed program of best practice documentation would provide operational guidance to early and mid-career intelligence professionals and help make senior leaders more aware of how other organizations in the intelligence community best conduct operations. Moreover, Pfaff’s suggestion that the intelligence community must be able to improvise to respond to complexity applies only to a small subset of upper-echelon professionals. For most in the intelligence enterprise, best practices are an effective way to build a foundation of knowledge and to document procedures for the benefit of those in peer organizations.
Lessons from the Private Sector
While there are differences between national intelligence efforts and business and tech organizations, these communities all deal with the same challenges: how to make risky decisions in the shortest amount of time using large amounts of data. When it comes to best practices, the intelligence community has much it can learn from the corporate and technology worlds.
In the business world, one definition of best practices is “a set of guidelines, ethics or ideas that represent the most efficient or prudent course of action.” Best practices may be set forth by an authority, such as a regulator or governing body, or internally by management. The generally accepted accounting principles (GAAP), for example, regulate and control the world of accounting. Not every accountant needs to operate in exactly the same way, but these principles sets the ground rules so business and corporate accountants can understand each other. Within accounting, the GAAP are best practices that operate not so much as a predetermined set of processes dictating how people do their jobs, but rather as a common language or reference for people doing the same work in different organizations that may otherwise fail to share information with each other.
In the technology community, an accepted definition of best practices is a “technique, method, or process” agreed to standardize “the most efficient and effective way to accomplish a desired outcome.” The tech world has best practices for product integration, education, cyber security, customer support, and project management. Developers and other professionals often discuss methodologies and processes on websites, bulletin boards, and other knowledge-sharing platforms. Even coding languages such as Python have shared best practices.
Best Practices in Intelligence
Understanding methodologies and encouraging the sharing of processes is beneficial for intelligence work just as it is in the private sector.
There are many best practices already in use by the intelligence community. Many of the profession’s earliest practices were solidified by best practices published in intelligence journals of the 1950s. The intelligence cycle, for example, is a tried and true process by which to generate intelligence. Analysts who create intelligence in a different order (analysis before collection, for example) may not achieve the same quality of results. The same sequence national security analysts use to create intelligence is also used by market analysts and cyber threat analysts. Although the environment and subject of analysis might be different, the intelligence cycle is a checklist that can be taught to any type of analyst in any educational setting.
Using industry standards and best practices, beginning analysts develop understanding of their work and target environments and minimize their learning curves. Eventually practitioners understand how to bring in new thoughts, ideas, and methodologies. But this evolution takes time, experience, and leadership that encourages growth.
Process documentation is a hallmark of high-reliability organizations. Defined by organizational behavior expert Karl Weick, a high-reliability organization is an organization defined by adherence to risk-reducing details. Examples of high reliability organizations include nuclear power plants, aircraft carriers, air-traffic-control teams, firefighting units, and hospital emergency departments. According to Weick, “[High reliability organizations] operate under very trying conditions all the time and still manage to have fewer than their fair share of accidents.”
With the need for split-second decisions and billions of dollars in investment in life, property, and reputation, many intelligence organizations, especially those supporting tactics and operations close to the warfighter, should be considered high-reliability organizations. Like emergency room personnel or nuclear engineers, entry-level analysts or mid-level analysts, handling an overwhelming amount of incoming information, are best facilitated by checklists or widely-recognized tactics, techniques, and procedures.
My first experience developing intelligence process documents was as a member of the 1st Cavalry Division in Operation Joint Forge in Bosnia-Herzegovina. Our section documented how the combat battalion conducted intelligence operations, and the document was eventually adopted by higher commands as the standard for the brigade. Our method of business for our high-reliability organization became the approved best practice.
Improvisation vs. Following Best Practices: The Case of Flight 1549
Pfaff’s skepticism of best practices and advocacy for more creativity and flexibility in intelligence decision-making is better placed for professionals at later stages of their career making higher-level decisions. As an intelligence professional progresses in their career, perhaps moving further from the tactical and operational toward the strategic, the purpose of process documentation and best practices changes. Intelligence professionals become, on balance, less concerned with basic tactics, techniques, and procedures, and instead require a higher level of awareness, both within their organization and of the data environment they are responsible for. They need to understand where the unexpected could arise and how to improvise, adapt, and overcome.
Pfaff referred to the need for upper-level professionals to improvise when she discussed the 2009 landing of U.S. Airways Flight 1549 in the Hudson River. While Capt. Chesley “Sully” Sullenberger found an answer that did not exist on a checklist, it is essential to note that he had been flying for over 30 years when he made the unconventional decision to land in a body of water. A less-experienced pilot may not have made the same decision. As Pfaff notes, “Sully and Skiles accomplished only a fraction of the items on the checklist before the plane hit the water, instead relying on experience, intuition, teamwork, and critical and creative thinking to land the plane as safely as possible.”
Still, I am leery to use Sully’s case as a model for improvisation in the intelligence community for a few reasons. The first reason is that while some experts may have the cognition to adapt, others might not. It is difficult to identify individuals — even experts — who will succeed in an environment that requires quick thinking and improvisation. Risk-taking is not predictable. Organizations can only support risk-taking by providing a steady foundation of situational knowledge via best practices.
The second issue is the distance between the decision-maker and the situation in Flight 1949. Sullenberger was in the cockpit, meaning that if the plane crashed, he would suffer the same fate as his passengers. This short distance between actor and situation encouraged quick, innovative thinking. Most intelligence professionals, on the other hand, have a greater distance between themselves and the situations they monitor.
The third problem with comparing Sullenberger’s actions to intelligence improvisation is the size of and structure of the decision-making organizations. There were only two decision-making personnel in the cockpit of Flight 1549: Sullenberger and his co-pilot, whom he outranked. Most intelligence analysts have to work on more diverse teams with more views and, most importantly, more authorities. Disagreements with authorities were at the crux of the controversy over the Islamic State analysis situation at Central Command in 2014, when analysts claimed they were overruled by their supervisors and their intelligence reporting was edited to portray the Islamic State as less dangerous than it really was .
A better model for improvisation might be how a team of expert jazz musicians interact in a performance. According to world-class jazz violinist and educator Christian Howes, “the extent to which any improviser suffers limitations or barriers to improvising is the extent to which he or she has not mastered these fundamentals of harmony.” Harvard neuroscientist Aaron Berkowitz compared improv to learning a second language: the basics come first, then articulation, then improvisation. The same is true when creating intelligence products.
I also disagree with Pfaff that best practices are not helpful for getting ahead of the global environment. She argues that the global environment has progressed too far along Dave Snowden’s model of “simple, complicated, complex, chaotic, and disordered” systems for adherence to strict process documentation. I would argue that this is not a shortcoming of best practices, but rather of leadership without enough experience and vision to turn over every rock.
Best practices, according to Pfaff, offer a “90 percent solution” in terms of situational awareness. I would argue that for entry and mid-level intelligence professionals, achieving 90 percent is an excellent foundation. This offers a base onto which experts and senior leaders can add additional knowledge and direction. Organizations can further raise the 90 percent by increasing personnel and employing new tools that can either automate collection or look in new directions. In a chaotic environment, we can standardize elements of data collection and focus more resources on our unknowns.
Lastly, the 90 percent threshold can be improved upon by senior leaders exploring the best practices of similar organizations throughout the intelligence community. A Central Command office working counter-terrorism should be able to learn how other organizations with the same scope handle their intelligence mission. By reading the best practices of other organizations, and innovating in their own organizations, senior leaders may be able to increase the 90 percent threshold established by their lower-downs and reduce risk to mission, assets, and reputation.
Conclusion
In nearly every organization I have worked with over the last 12 years, documenting and sharing processes has been essential. From the U.S. Central Command Intelligence Directorate to supporting disclosure of intelligence to allies in Afghanistan and, most recently, supporting information operations in Qatar, formalizing how we did business and sharing those processes with other organizations always benefited the greater good. In Afghanistan, for example, the disclosure best practices I documented were pushed to every U.S. military organization in eastern Afghanistan, giving warfighters clear point-by-point instruction on intelligence sharing that benefitted international cooperation while avoiding unauthorized disclosure. Having an approved best practice checklist instilled confidence in personnel new to intelligence-sharing.
Of course, in intelligence as well as in business and technology, best practices are not foolproof. Best practices should always be considered living documents, just as all organizations are living organizations, constantly evolving due to the environment, risk mitigation, technology, and customer demand. When better processes are determined, best practices should evolve. In fact, modifications of best practices are not only recommended, they are expected. But the benefit of best practices is clear. They assist in bolstering process understanding in early and mid-level career professionals and provide a basis of comparison and sensemaking for senior leaders.
Intelligence agencies should establish teams to document best practices. These teams would have awareness of processes throughout the intelligence community. They could assist new or inexperienced offices or teams with insights into methodologies and processes. In addition, designating a documentation team, as Central Command did, ensures that the burden of writing does not fall on busy analysts.
Organizations should then be encouraged to share their processes. Through discussion, the best processes should rise to the top. The intelligence community is a large, constantly evolving organization, and one of its biggest advantages its steady flow of new talent. Determining community-wide best practices can get new talent up to speed as quickly as possible. Best practices in the form of process documentation have their limitations, but they remain the most efficient, effective way to make use of human potential across the intelligence community.
Michael Lortz has provided support to intelligence and special operations for over 20 years. He has worked with military, government, and private clients at national, state, and local levels. He holds an MBA in Process Development and Innovation and an MA in International Affairs. He can be followed at @JordiScrubbings.
Image: Israel Defense Forces
