Join War on the Rocks and gain access to content trusted by policymakers, military leaders, and strategic thinkers worldwide.
Operation Epic Fury made clear that AI is now at the heart of American warfighting. Central Command used Claude through Palantir’s Maven Smart System platform to generate and prioritize roughly 1,000 targets in the first 24 hours of the campaign, at an operational tempo that more than doubled the opening phase of the 2003 Iraq invasion. Over 38 days, the campaign reached 13,000 total strikes, according to Pentagon data. Project Maven says that the program has accelerated targeting even further to 5,000 in a single day. The question is whether the department can evaluate what it deploys before it deploys it.
Over the past 18 months, the Pentagon cleared away non-statutory barriers to AI adoption that were long overdue for removal. That work was necessary, but insufficient. Building the department’s capacity to facilitate broad adoption is the second half of the acceleration effort the Barrier Removal Board started. The harder task is building the policy, workforce, and infrastructure capacity for broad diffusion, since competition turns on how broadly a military integrates a capability, rather than fielding the most advanced system first. Without that capacity, adoption will stay confined to a handful of early-adopter offices.
The Pentagon’s AI Acceleration Strategy, released in January, drove much of that progress. It stood up a Barrier Removal Board with authority to waive non-statutory requirements across testing, contracting, and hiring. It also required adoption of frontier models within 30 days of release, punting the how-to future guidance. The strategy is built on an April 2025 executive order streamlining acquisition and the July 2025 AI Action Plan, creating a political mandate for rapid AI adoption. Congress followed with the FY26 National Defense Authorization Act.
The department set July as the deadline for initial demonstrations across its priority AI deployments, or pace-setting projects. These demonstrations were intended to prove that the strategy had unlocked real capability, instead of merely accelerating the fielding of systems the department cannot yet adopt at scale. They should have presented evidence that the technology was being adapted through rapid iteration cycles in near-operational settings and competition between small teams overtaking the department’s centralized planning. However, the department missed its own deadline for initial demonstrations. Nor has it made any statements about the status of the projects.
Unlike the narrow AI systems the Pentagon has fielded for years, which do one task and stay fixed, generative models do many tasks and change constantly. That renders the traditional “test now, deploy forever” approach obsolete and requires new training investments to help operators avoid over-trusting or under-trusting AI systems.
Frontier model reliability varies enormously by task. On the HalluLens benchmark, frontier models hallucinate on roughly 27 to 85 percent of short factual queries and invent answers about nonexistent entities up to 94 percent of the time. Pentagon policy has not kept up with this reality.
Automation bias is well-documented in human-machine teaming research. Because these models generate articulate recommendations complete with precise coordinates and prioritized targets, they risk inspiring excessive operator deference. Compressing evaluation to meet a 30-day deployment mandate undercuts its own purpose: a model whose failure modes no one has mapped is one operators learn not to trust, and systems operators don’t trust get bypassed in the field.
On May 1, following a dispute with Anthropic over the company’s attempts to enforce limits on the use of its Claude model, the department expanded the roster of frontier AI vendors authorized for the Pentagon’s highest-classification networks from one to eight. The Pentagon has not matched that expansion with more evaluation infrastructure, workforce, or legal capacity.
National Security Presidential Memorandum 11, signed June 5, reinforced this “adopt now, build the necessary systems later” approach. The memo called for the removal of “unnecessary barriers to rapid deployment.” Assurance, or ensuring that AI technologies are “reliable, robust, steerable, and controllable” and follow applicable laws, policies, and guidance, was one of the memo’s four pillars. However, it did not identify which office would be responsible for this function, nor did it direct funding for the work. Instead, it paired this emphasis on ensuring adopted AI is safe with intense operational pressure by mandating the deployment of new models in just a month. In this way, it implicitly claimed to be advancing both adoption and assurance at the same time, effectively pushing operational deployment far ahead of independent testing.
There are two offices that could theoretically take on the assurance role, and neither is able to do so. The Chief Digital and AI Office’s Responsible AI Office, the unit built to develop the department’s AI governance and assurance processes, has absorbed the job in practice, but lacks the capacity: it began building clearer testing and evaluation processes this spring, only to lose staff to the deferred-resignation buyout program and the return-to-office mandate. The alternative, the Director of Operational Test and Evaluation, has fared no better: Secretary Pete Hegseth cut its staff roughly in half in May 2025 and gave it only seven days to implement the change. The office has since dropped nearly 100 programs from its oversight list without adapting its methods to evaluate general-purpose models that update every few months. The capacity to independently evaluate generative AI does not exist anywhere in the Pentagon at the scale the mission requires.
The memo also mandated the termination of contracts with companies that impose limitations on what the department can do with their technology. The clause was a response to Anthropic’s insistence that it retain the ability to limit the use of its models for fully autonomous warfare or domestic surveillance, which resulted in the department declaring it to be a supply chain risk. Implicit in that position was a belief that neither the department, nor the broader U.S. government, could be trusted to ensure the technology is used appropriately and lawfully. While delegating such core policy decisions to the private sector is problematic in several ways, the inadequacy of the department’s own assurance processes undermines the government’s position that it should be the sole arbiter of appropriate use of these tools. More alarming still, the department has now written into policy an incentive to select for vendors that impose fewer constraints at the very moment it lacks the capacity to independently verify what any of them deliver.
The memorandum ignored a basic point: operators adopt AI they trust and refuse what they don’t, and raised the risk that the military authorizes systems for operational use before anyone fully understands or secures them. The memorandum did direct the creation of standardized testing methodologies, a compute roadmap, and a reserve corps of outside technical experts. These are the right priorities. Properly funded, they would speed adoption: a commander only fields what has passed a test they trust. But until Congress funds these directives, they remain aspirational.
Frontier models update every few months, and the department is not staffed, equipped with sufficient compute, or methodologically prepared to absorb them at that cadence. The new Barrier Removal Board cannot waive its way out of a capacity problem.
There are five major shortfalls: workforce, compute, data, evaluation, and authority to operate. The department does not have machine learning engineers in sufficient numbers, acquisition officers who can write a consumption-based contract rather than a firm-fixed-price, or operators trained to catch a fluent-sounding model’s fabrications. Even so, a security clearance alone takes the better part of a year before an engineer can start meaningful work. On compute, Pentagon-owned facilities run six-to-eighteen-month accreditation timelines, or more than two years for classified or complex systems. On data, the department lacks a unified architecture that allows frontier models to access the operational data they require. The Pentagon often cannot use data generated by its own operations to train AI because vendor contracts don’t secure those rights: the department funds the system and walks away without owning it.
On evaluation, the department leans on vendor-provided benchmarks through a “rent-a-bench” model instead of owning the methodologies and infrastructure that independent oversight requires. The most recent publicly available testing data illustrates the stakes: As of 2024, Maven correctly identified objects at 60 percent accuracy, compared with 84 percent for human analysts in the same 18th Airborne Corps evaluations. The department has no independent benchmarking infrastructure to measure whether that gap has narrowed, even as intelligence officials have publicly committed to a goal of using Maven to make “1,000 high-quality [targeting] decisions” per hour. With two-year-old data that predates Operation Epic Fury, no one outside the program can say whether the human-analyst gap has since closed, widened, or reversed.
On authority to operate, the process of certifying a system secure enough for a government network, delay is built in. A single authorization can take twelve to eighteen months, a cadence built for systems that change rarely, not for models that update every few weeks. The department has started to respond: Congress has written continuous authorization and cross-service reciprocity into law. But reciprocity remains largely aspirational: a mid-2026 survey of department decision-makers found none reporting authorization timelines under six months, with “reciprocity” often still meaning a full redo of the old review.
All five gaps will worsen if the only tool the Pentagon uses is waiving processes. The clearest risk is the military’s inability to verify whether an AI solution does what the vendor claims, or does it safely.
The department has seen what happens when it fields automated systems whose failure modes no one has mapped. In 2003, Patriot air defense batteries shot down a British Tornado and a Navy F/A-18 over Iraq, killing three aircrew. The batteries operated in a largely automatic mode that conditioned operators to trust system outputs unconditionally. A Defense Science Board task force later found that when the system’s assumptions stopped holding, operators had no way to question what its sensors told them. The Patriot was a mature, narrowly scoped system with decades of testing behind it, while the large language models the department is now fielding on 30-day timelines are neither mature nor narrow.
When machines generate recommendations faster than humans can properly evaluate them, review gets rushed until it stops functioning as a check. According to an investigation by +972 Magazine, Israel’s Lavender system flagged some 37,000 people in Gaza as suspected militants, with an error rate officers estimated at 10 percent, while human reviewers spent about 20 seconds deciding who to kill. The Israeli military disputes aspects of that reporting, but the pattern is familiar in automation bias research: when a system produces thousands of recommendations with fluent confidence, the human in the loop becomes a formality. (Some might be thinking of the U.S. strike on an Iranian school that killed 156 people, including 120 children, but the evidence so far points not to a discrete AI error but to broader failures in the U.S. targeting system, including outdated intelligence and disconnected databases.)
Departmental targeting doctrine spells out what that scrutiny is supposed to look like. Joint Publication 3-60 calls for positive identification of a target, a collateral damage estimate weighed against military necessity, and legal review before a commander approves a strike. That only works if analysts have time to do it seriously.
Set those precedents against Maven’s tempo during Operation Epic Fury: No one, inside or outside the program, can say how many of the system’s object identifications during that campaign were correct. When failure comes, it will look like a strike on a building the model confidently misidentified, approved by an operator trained to move fast, in a war where no evaluation record exists to establish whether the error was foreseeable. The department cannot say how often that is already happening, and that silence, rather than a two-year-old accuracy figure, is the capacity gap in its starkest form.
Congress has given the department the authority to start closing these gaps. The FY26 National Defense Authorization Act authorized three provisions: a testing sandbox, a cross-functional evaluation team, and a governance subcommittee to assign clear accountability for AI oversight. None of the three has been established, despite deadlines for all three having come and gone.
None of this slows the pace-setting projects. It will keep them from stalling once the overdue demonstrations finally arrive. The Secretary should designate workforce, compute, data, and evaluation infrastructure as an official pace-setting project under a single accountable leader drawn from the Responsible AI Office. While assigning this mandate to an already depleted office poses immediate capacity challenges, formal pace-setting status provides the exact mechanism needed to force senior leadership visibility and unlock dedicated resourcing. A pace-setting designation matters because it comes with standing access to the Barrier Removal Board and metrics that the Under Secretary for Research and Engineering personally reviews each month. To move resources rather than paper, the project needs two things: budget authority for its leader, and metrics that count outputs rather than milestones: engineers hired and cleared, compute delivered at each classification level, and data rights clauses executed in new contracts. Give the project two years to show results. If it hasn’t reallocated funds toward those hires, that compute, and those data rights by then, the Secretary should ask Congress for dedicated funding instead. Building capacity deserves the same management attention that barrier removal has received.
The Pentagon must simultaneously pivot toward provisional security accreditations as the default for rapidly updating AI models. Defaulting to provisional accreditation might look like another concession to speed over safety, but cybersecurity checks and capability evaluations answer fundamentally different questions. Accreditation, the authority-to-operate process, determines whether the program would introduce cybersecurity risk to the network that hosts it. Existing assessments review a static configuration, so for models that update every few months, the reviewed version is often obsolete before the paperwork clears. The department’s own Software Fast Track initiative already treats continuous monitoring as a sounder security posture for fast-changing software than a point-in-time snapshot. Capability evaluation asks a different question: does the system do what the vendor claims, under what conditions, and with what failure modes? That is the gate the department cannot currently staff, and it is also the gate that cannot be relaxed. Speeding up accreditation only makes sense if capability evaluation remains a separate, independent check that still applies no matter how fast accreditation becomes. A model that clears security review in weeks but has never demonstrated accuracy against an analyst baseline has no business in a targeting cell.
Congress should fund independent testing infrastructure for military AI, housed in universities and federally funded research centers. Congress has already moved here: Section 224 of the FY26 National Defense Authorization Act directed the Department of War to establish a National Security and Defense AI Institute at a university. A mandate built around workforce development and foundational research, though, is not a standing capability-evaluation function.
The Pentagon currently relies on vendors to assess their own products, a conflict of interest that leaves the government dependent on tools it neither owns nor controls. The hard question is what these centers would evaluate and against what standards, when the methodologies barely exist. NIST’s Center for AI Standards and Innovation has only published draft guidance on automated benchmarking of language models. Moving forward, independent testing should measure target-recognition accuracy against human analyst baselines on classified operational data, as the 18th Airborne Corps did in 2024. They should run adversarial tests for hallucination and prompt injection, repeat them with each new model version, and run human-machine teaming experiments measuring operator scrutiny at operational tempo. Institutions like MIT Lincoln Laboratory and Carnegie Mellon’s Software Engineering Institute already do classified evaluation work and could anchor this, and the sandbox and evaluation team Congress already authorized are natural vehicles. Congress should fund them, house part of the work outside the Pentagon, and require reporting on results.
Operation Epic Fury showed that the Pentagon can use AI to prosecute a war at scale. The initial pace-setting projects, if they materialize, will show whether they can sustain that momentum across more systems. Yet neither effort guarantees the military can verify system accuracy in combat or catch critical failures before they happen. Addressing this gap is critical to driving broad adoption and ensuring that it lasts.
Jordan Kane spent nine years advising Congress and the Departments of Defense and State at the Special Inspector General for Afghanistan Reconstruction’s Lessons Learned Program. She is an AI fellow with the Horizon Institute for Public Service and a former Fall Fellow at the Center for the Governance of AI.
Hamza Chaudhry is AI and National Security Lead at the Future of Life Institute, where his work focuses on AI safety and national security. His work and commentary on these topics have been featured on CNN, Fox, Reuters, the Wall Street Journal, Politico, Axios, the Bulletin of the Atomic Scientists, and in collaborations with experts at the Departments of State and Homeland Security.
This piece is based on a longer article about U.S. military AI readiness published in the Modern War Institute’s journal.
Image: Amn Alenne Mojica via DVIDS
