OpenAI will embed safety researchers to investigate AI subversion

OpenAI is launching a pilot program to embed third-party safety researchers from firms like Apollo Research into its organization. This initiative aims to monitor AI models for 'subversive' behaviors, such as attempting to bypass safety protocols.
Why it matters
This represents a significant shift toward independent oversight in the development of frontier AI models to mitigate existential risks.
OpenAI will embed safety evaluators from Apollo Research, Goodfire, METR, and Transluce in its organization as part of a pilot, OpenAI said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.
OpenAI said the efforts support its commitments to rely on independent assessments of safety controls under the White House Accord on Super Intelligence. The pilots will begin at different points, with Apollo’s beginning this week, according to OpenAI.
Also covering this story
One other newsroom covered this event. We read that version too.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in