Posted in

Pentagon AI Weapons Projects Often Fall Short of Internal Safety Rules

A congressional watchdog report released in October found that some artificial intelligence weapons and tools under development at the Department of Defense are not consistently following the Pentagon's own guidelines for responsible AI use. The findings, from the U.S. Government Accountability Office, offer a rare, detailed look at how AI-enabled military technology is actually built, and where oversight is falling behind the pace of development.

What the GAO found

The GAO surveyed officials overseeing 10 AI-enabled weapons and tools - a small sample out of more than 300 such projects underway across the Pentagon. These systems are designed for tasks such as identifying battlefield targets, processing surveillance data and supporting tactical decisions. Six of the 10 officials surveyed said their teams had identified risks associated with deploying these tools in real operational environments. Four said the project requirements were not aligned with the Pentagon's "Responsible AI" tenets, and three said they did not know whether their systems would be continuously monitored once deployed, despite that monitoring being one of the department's own stated priorities.

Those tenets stem from a 2021 Pentagon memo that lays out six basic principles: departmentwide oversight, verification processes, real-time monitoring of AI systems, and cooperation with allied nations to build shared global norms. Alissa Czyz, who led the GAO review, noted that the guidance is not a binding department instruction but a memo the Pentagon has signaled should be followed. Most teams reviewed had taken at least initial steps toward compliance, and some projects were still in early development, which may partly explain the gaps.

A short-staffed oversight office

Part of the problem, according to the report, is capacity. The Department of the Responsible AI Division - the office tasked with helping teams understand and apply the guidelines - was authorized to hire 10 staff members in 2023. As of February, only three of those positions were filled, and the GAO report states the Pentagon is unlikely to fill the remaining vacancies or expand staffing. Czyz described this as "a very small number of folks tasked with a large responsibility of responsible AI throughout the department."

The GAO issued two recommendations: clarify the responsibilities and meeting schedule of the oversight body charged with AI governance, and address the understaffing. The Pentagon agreed with both recommendations but, as of Oct. 22, had not implemented them, according to a GAO representative who spoke with NPR. The Pentagon did not respond to NPR's question about a timeline for changes.

Policy tension: speed versus caution

This gap between stated principles and practice comes as Pentagon leadership pushes to accelerate AI adoption. A January memo described a goal of making the department an "AI-first" warfighting force "across all components, from front to back." Defense Secretary Pete Hegseth has also announced a new command focused on autonomous weapons, including drones, and has publicly criticized slow-moving bureaucratic processes. Speaking at SpaceX in January, Hegseth argued that "military AI is going to be a race for the foreseeable future where the risks to U.S. national security of moving too slowly outweigh the impacts of imperfect alignment."

That stance contrasts with longstanding warnings from AI researchers and international bodies, who have urged strict controls on autonomous weapons systems, citing risks of unpredictable behavior, reduced human control and the potential for rapid, unintended escalation in conflicts. The GAO report does not resolve this debate but documents, with specific survey data, that internal compliance mechanisms are not yet keeping pace with the department's stated ambitions. It remains unclear how or when the Pentagon will close that gap.