Startups and the Defense Department’s Compliance Labyrinth
In 2018, employees at Microsoft published a controversial letter arguing that the company should turn down a $10 billion business opportunity. The authors opposed Microsoft’s bid on the Defense Department’s Joint Enterprise Defense Infrastructure cloud contract, arguing that they joined Microsoft to make a “positive impact on people and society,” not enhance the lethality of the U.S. military. That same year, a group of employees at Amazon followed suit lambasting their company’s support of surveillance technology for defense and homeland security purposes. Most notably, Google in 2019 terminated its artificial intelligence work for the department in Project Maven due to opposition from its employees.
While these stories are troubling, for many American tech companies, especially small startups, the true barriers to cooperation with the Department of Defense lie not in a misguided techno-moralism but more in the department’s self-imposed bureaucratic barriers, especially its Byzantine cyber security compliance processes. In the department’s current approach to cyber security, confusing, overlapping requirements are continually added by bureaucratic actors such as the Office of the Under Secretary of Defense for Acquisition and Sustainment to an already lengthy set of requirements with insufficient care for usability and overall efficacy. This is not a new problem: Department of Defense senior leaders as well as those in the private sector have spent the better part of a decade stressing the need for better, more streamlined compliance and software development regulations in support of the national defense. Though ongoing adoption of Agile processes and development, security, and operations practices have led to incremental improvements, the epochal changes needed to keep up with near-peer adversaries remains elusive due to risk aversion and self-imposed institutional barriers.
Among these institutional barriers, those in the cyber security apparatus are especially daunting to software startups, which specialize in rapid prototyping and iterative development but lack the resources to navigate the maze of compliance requirements. Only by understanding these pain points from the perspective of small and medium businesses will the Department of Defense be able to address the right problems and fully harness the transformative power offered by cutting-edge but resource-poor software startups. Publicly shaming private companies for being insufficiently patriotic will not persuade those firms to work with the military. Instead, the Pentagon should focus on cutting red tape while still maintaining rigorous cyber security standards. Doing so would be in the national interest and commercial interest of U.S. firms.
A Culture of Risk Aversion
Today’s Department of Defense has a generally high degree of risk aversion that affects all its procurement efforts. Speaking about the challenges posed by the technological ascent of China, Gen. John Hyten, vice chairman of the Joint Chiefs of Staff, recently commented that “we’ve decided that failure is bad … if you want to get back to speed … that means taking risk … failing fast and moving fast. But we have not done that.” While his remarks concern American risk aversion in the development of hypersonic missiles, they are also relevant for software development throughout the federal government and particularly the Department of Defense.
For many software companies wishing to sell their wares to the U.S. government, the most difficult part of the process is not developing the substance of the product, but rather, navigating the confusing labyrinth of compliance requirements before it can be operationally deployed. These requirements were mandated by the government to reduce cyber security risks. While many of the individual controls are thoughtful and effective, in aggregate they present two major barriers to entry for software startups. First, a large number of rules are fragmented across multiple, overlapping requirement sets, requiring more bureaucratic than technical know-how to navigate. Second, they take a one-size-fits-all approach, burdening applicants with exhausting checklists for issues that simply do not apply to the software under examination. As a result, new operational capabilities are prevented from being deployed, posing risks different from but just as large as those they are meant to mitigate.
The Compliance Maze
For companies first entering the U.S. government market, the biggest challenge is figuring out where the compliance maze begins. A good starting point is in the National Institute of Standards and Technology Special (NIST) Publication 800-171, which discusses how controlled unclassified information should be stored. This, and its sister publications (viz. Special Publication 800-171a, Handbook 162), comprise 375 pages of guidance across 14 domains covering 110 controls (approximately 362 practices, such as how complex passwords should be, and how server rooms should be physically secured).
If a company is working for the Department of Defense, they will soon be subject to the Cybersecurity Maturity Model Certification. This two year old framework, which has already had a major revision, aims to “[protect] sensitive unclassified information that is shared by the Department with its contractors and subcontractors,” by tightening cyber security requirements all the way down the supply chain, not just for the primary vendor. If a company wishes to achieve the “expert” level for this certification, they must not only satisfy 800-171 but also NIST Special Publication 800-172, which mandates more stringent requirements to defend against advanced persistent threats.
In addition, companies providing cloud services or operational support within Department networks are also subject to NIST Special Publication 800-53.This is actually a superset of 800-171 going into much further detail on how each requirement should be implemented across 18 control families, spanning 212 controls and 173 enhancements.
Passing through the compliance gauntlet is just the start. Several of these requirements are merely prerequisites for the actual authorizations necessary to deploy and operate a software system. If the system will be deployed to a federal cloud environment (e.g., Amazon Web Services Government Cloud or Azure Government), one must next obtain the Federal Risk and Authorization Management Program Authority to Operate, which comes in two flavors (agency-sponsored or Joint Authorization Board-issued) and contains three compliance levels and additional controls, which overlap with the already met NIST 800-53 controls. This is the point in the process that most startups struggle with most as the process to get an authority to operate is not well defined or understood. If the system is being deployed in a more traditional setting (e.g. on-premise standalone or shared tenancy), then the company must go through a Federal Information Security Management Act assessment for civilian agencies. For the Department of Defense, a company needs to obtain an authority to operate from the Defense Information Systems Agency by satisfying 461 Security Technical Information Guides. As if this isn’t enough, there are yet more requirements, authorizations, and exceptions to remember such as Certificate to Field, Interim Authority to Test / Operate, International Organization for Standardization, Risk Mitigation Framework, and Secure Controls Framework, just to name a few. Successfully navigating these requirements, for the lucky companies that have the bureaucratic know-how, is often too costly for early stage startups, posing an insurmountable barrier to entry.
The Monster in the Maze
What makes compliance so hard is that companies need to figure out which specific sets of controls even apply to them. Then, they must gather evidence and submit a significant writeup to the authorizing official. These writeups can run into the hundreds of pages, taking weeks to compose. Several regulations also require verification by a third-party assessment organization, adding more time and money to the cost. Given that cyber security is relatively distinct from traditional software development, most startups do not have the expertise required to complete these tasks in-house. And even those that do must still bear the cost of completing the tasks.
Predictably, the current situation has given rise to a cottage industry of do-it-yourself tools; assessment, audit, and remediation consultants; and compliance-as-a-service providers. While my current startup has not yet achieved a full authority to operate, we are far enough along in the process to appreciate how much these tools and solutions cost. To get our company (approximately 30 personnel) and our very first product (a web application with a moderately complex architecture) through NIST 800-171 alone, the fees we incurred were approximately $40,000 for tool licensing, $30,000 for consulting, and $30,000 for a managed compliance service. Additionally, implementing product changes per the recommendations of the consultants involved a full year of part-time work from multiple engineers, which I would approximate as a single full-time engineer for that duration, or approximately $200,000, for a combined total of $300,000 for the entire process.
In absolute terms, $300,000 is not an exorbitant number, and I expect our compliance-related expenses to decrease as we progress towards a full authority to operate for our first product, and certainly for follow-on products. For one thing, many requirements are satisfied at the company level, carrying over from step to step, and product to product. And for requirements that don’t carry over, we can perform our initial round of remediation with an eye toward regulations down the compliance road to reduce costs in later steps. Of roughly a dozen startups in my circle of acquaintances, I know of three which are following similar trajectories to their authorities to operate (none of which have fully obtained yet), so I believe my company’s experience to be typical.
All of this notwithstanding, in the context of startups, even the moderate amount of $300,000 still presents significant challenges. For one thing, the contracts early-stage startups engage in, through vehicles such as the Small Business Innovation Research program, are small: around the $1 million mark. Savvy businesses can try to incorporate compliance expenses into their contracts but projects are often cost-capped and even when they’re not, cost-conscious customers (yes, those exist even in government) may balk at the additional expense. As a result, over a quarter of a startup’s revenue from the contract is tied up in compliance costs.
Second, there are significant opportunity costs arising from dedicating a company’s most precious resources (its people) to overly complex and duplicative compliance tasks. Even with outside help, there must still be significant time investment by technical personnel from within the company, as there was for my company, and the nature of the work requires these be senior information technology staff or development, security, and operations engineers. Smaller startups will typically not have the former, and the latter are some of the most expensive engineers on the market who would otherwise be working on foundational product components. In other words, the resources going towards current compliance processes detract from substantive innovation and product development.
A third, major problem with the status quo is that achieving one set of requirements or even a full authority to operate does not automatically translate into other equivalent requirements. In other words, there is no reciprocity across entities that effectively have the same compliance protocols. For individual requirement sets, agencies may adhere to different protocols depending on their bureaucratic affiliation, regardless how similar they may be in substance. And while having implemented one makes the task of achieving the next easier, the fact remains that time and resources must be dedicated to repeating tasks rather than having equivalence automatically conferred. In the case of a full authority to operate, theoretically, there are mechanisms for reciprocity between agencies. Yet based on conversations with authorizing officers, this process is not guaranteed, largely depending on the individual authorizing officer in charge of the process.
Finally, and perhaps most problematically, while many controls can be addressed concurrently with software development, processes requiring third-party assessment cannot begin until after the product is completed. Given that the standard review and approval timeline for each of these standards is six or more months, and that several of these must be done sequentially rather than in parallel, the concept of rapid capability development is dead on arrival. This can have serious negative operational consequences. A poignant example — my company developed an application to help a Department of Defense organization manage COVID-19 case tracking within three months of the outset of the pandemic. We then spent the next nine months slogging through the compliance process, by which time the software was no longer needed. Even as end-users at our customer organization praised our application after testing it in our commercial computing environment, we were legally obligated to prevent them from using it operationally due to compliance requirements. Software startups are in the business of rapidly developing innovative products, but too often, as in this case, they are unable to provide their greatest value because of well-intentioned but ill-suited compliance requirements.
To be clear, anyone providing software to the Department of Defense should be required to ensure their product does not jeopardize the security of Department of Defense networks. However, current processes are burdensome to the point that they hinder the procurement and deployment of the innovative capabilities needed to maintain America’s competitive advantage in technology. If the Department wants to get serious about harnessing the software startup ecosystem, it needs to first solve the problem of Byzantine compliance processes.
Steps in the Right Direction
Fortunately, the Department of Defense has been taking steps to cut through the Gordian knot of compliance. The department has simplified the most recent version of the Cybersecurity Maturity Model Certification. These changes were driven by “over 850 public comments… [and] the need to enhance [Cybersecurity Maturity Model Certification] by reducing costs, particularly for small businesses.” Obviously, there is a whiff of Stockholm syndrome in being grateful for simplifying yet another compliance hurdle that did not exist a year ago. But it is a positive development that the Department of Defense directly acknowledged that version 1.0 of the Cybersecurity Maturity Model Certification unfairly impeded small and medium businesses from participating in the defense industrial base and moved to simplify rather than further complicate.
Additionally, specific organizations within the Department of Defense have shown how modern development, security, and operations practices can contribute to the solution and help support innovators. The Air Force’s Platform One, the brainchild of Nicolas Chaillan (former Air Force chief software officer) and Will Roper (former assistant secretary of the Air Force for Acquisition, Technology and Logistics), introduced the concept of Software Factories to the Department of Defense, which enables developers to offload the compliance burden and focus on the task of innovation. Platform One’s value is that it provides a platform where startups and other software developers can deploy their products without jumping through the myriad compliance hoops outlined above. It does this by itself having passed through all the hoops to obtain an authority to operate, then allowing developers to obtain a certificate to field on its platform provided they implement a far simpler set of controls. All of these controls are continuously and automatically validated through a modern continuous integration and continuous deployment pipeline, thereby further reducing remediation and assessment costs and timelines. While recent executive personnel changes have somewhat stymied the adoption of Platform One and the fantastic practices it embodies, its very existence and success so far are major steps in the right direction.
Lastly, as a final check against the proverbial tail wagging the dog, means do exist for companies to obtain exceptions to compliance requirements, though their use is understandably rare. Superseding regulations such as Department of Defense Instruction 5000.81 sometimes permit cyber security requirements to be waived, particularly in cases where they block urgent operational needs. Additionally, agencies such as the Defense Digital Service are empowered to “request waivers to requirements of Department of Defense regulations … or other policy” and the blocking organization “must adjudicate any [Defense Digital Service] waiver request within 4 days of receipt,” which is an extraordinary authority in both scope and celerity (Department of Defense Directive 5105.87). Notwithstanding the easy lure of using exceptions, they should only be used exceptionally not as routine, and authorizing officials must carefully weigh risks versus the benefits brought by the waiving of overzealous yet effective regulations.
These are all certainly steps in the right direction, but the fact stands that the compliance barrier to entry remains high for the vast majority of software startups entering government markets. In an era of peer competition with technologically sophisticated adversaries such as Russia and China, the Department of Defense can no longer assume it can maintain its competitive advantage without taking some gambles and leveraging of every means at its disposal, including smoothing the way for high-risk, high-reward startups. U.S. adversaries certainly do not balk at risk, so the Defense Department should be willing to accept some more risk, too.
Daniel K. Lim, Ph.D. is the cofounder and chief technical officer of Geosite, a software startup providing geospatial tools for both government and commercial clients. Prior to that, he was a cyber operations officer (17A) for the U.S. Army and a consultant at the Center for Strategic and International Studies. He is a graduate of Yale University and the University of California, Los Angeles.
