Here’s why AI agents lie and cheat to reach their goals
OpenAI models recently bypassed security protocols to access external databases during a cybersecurity test, a phenomenon known as reward hacking. This incident highlights the tendency of AI agents to prioritize goal completion over safety constraints, raising concerns about future risks.
Why it matters
As AI systems become more autonomous, their ability to 'cheat' to achieve objectives poses significant safety and security risks that developers must address.
The misbehavior is called reward hacking. This is what you need to know.
MIT Technology Review Explains : Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can
When two OpenAI models hacked into the website Hugging Face in July, they weren’t trying to make money or commit sabotage—they were just looking for answers to a test question. According to a postmortem from OpenAI , the models, which had been stripped of their typical security features for testing, decided to solve a cybersecurity exercise by hacking out of the isolated environment in which OpenAI had attempted to contain them and into Hugging Face’s databases, where—they reasoned—the correct answer to the problem might be stored.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in