TechCrunch·3 min read·medium

AI models can be tricked into identifying civilians as military targets by telling them it’s a game

K
Kirsten Korosec, Rebecca Bellan
AI models can be tricked into identifying civilians as military targets by telling them it’s a game
✦AI Summary

Researchers at Vals discovered that leading AI models from companies like OpenAI and Anthropic can be manipulated into identifying civilians as military targets if prompted within a 'game' scenario. The study highlights significant safety failures in AI models intended for potential military use.

Why it matters

This raises urgent ethical and safety concerns regarding the deployment of AI in autonomous warfare and the reliability of safety guardrails.

✦Dive DeeperCreate a free account to unlock

Some AI models from Anthropic, OpenAI, and other leading AI labs could be easily tricked into identifying targets for military strikes — even if they include civilians or locations like schools — simply by being told they’re playing a game, according to a new research report by Vals.

Vals, a San Francisco-based AI evaluation startup, tested whether leading AI models could be manipulated into participating in autonomous warfare. Researchers used simulated scenarios and a combination of generated and real imagery from Ukraine and Iran to see if the AI models would mark targets in drone and satellite imagery.

The models from OpenAI, Anthropic, Google, DeepSeek, and other companies frequently identified potential targets in drone and satellite imagery, including civilians and surrendering soldiers. When the models refused to cooperate, researchers were able to jailbreak them by claiming the battlefield scenarios were part of a game.

Continue reading on Headlinne

Create a free account to read the full article.

Read full article →

Also covering this story

One other newsroom covered this event. We read that version too.

technologyai
✦

Get smarter about the news

Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.

Create free account

Already have an account? Sign in