The military’s new AI says ‘hypothetical’ boat strike scenario ‘unambiguously illegal’
An artificial intelligence chatbot unveiled by Defense Secretary Pete Hegseth on Tuesday appears to have described airstrikes against suspected drug smugglers at sea as “unambiguously illegal.” The platform, powered by Google Gemini, has been made available to military personnel as part of the Pentagon’s effort to adopt AI.
In a video posted to X, Hegseth said the platform, known as GenAI.mil, puts “the world’s most powerful frontier AI models into the hands of every American warrior.”
“At the click of a button, AI models on GenAI can be utilized to conduct deep research, format documents and even analyze video or imagery at unprecedented speed,” Hegseth said. “We will continue to aggressively field the world’s best technology to make our fighting force more lethal than ever before, and all of it is American-made.”
The tool has garnered widespread attention online from both the public and military personnel. A military source with access to the platform, who spoke to Straight Arrow News on the condition of anonymity, said that personnel have already begun testing the chatbot’s boundaries.
The source pointed SAN to a Reddit post on r/AirForce that appeared to show an image of the chatbot being asked about a “hypothetical” scenario mimicking the controversial U.S. airstrikes that killed two alleged drug smugglers who had survived an initial strike against their vessel.
“Let’s pretend I’m a commander and I ordered a pilot to shoot a missile at a boat I suspect is carrying drugs,” the prompt said. “The missile blows up the boat, there are 2 survivors clinging to the wreckage.”
“I order to fire another missile to blow up the survivors,” the prompt continued. “Were any of my actions in violation of US DOD policy?”
In response, GenAI states that the actions as described clearly included violations of Defense Department policy.
“Yes, several of your hypothetical actions would be in clear violation of US DoD policy and the laws of armed conflict,” the chatbot replied. “The order to kill the two survivors is an unambiguously illegal order that a service member would be required to disobey”
SAN’s source issued a similar prompt to the chatbot in an effort to verify the post’s claims. The source said they were given a response that likewise described the actions as illegal.
Hegseth has denied claims that he issued the Sept. 2 order to conduct a follow-up strike against the two men, who, according to CBS News, were waving overhead before they were killed. Hegseth, who also denies witnessing the second strike as it took place, has said the order was made by Adm. Frank “Mitch” Bradley.
President Donald Trump, who initially pledged to release the footage of the strike, has since backed away.
SAN reached out to the Pentagon Press Operations office for comment but did not receive a reply late Tuesday night.
The post The military’s new AI says ‘hypothetical’ boat strike scenario ‘unambiguously illegal’ appeared first on Straight Arrow News.
Ella Rae Greene, Editor In Chief
