TechCrunch·4 min read·hard

OpenAI will embed safety researchers to investigate AI subversion

R
Rebecca Bellan
OpenAI will embed safety researchers to investigate AI subversion
✦AI Summary

OpenAI is launching a pilot program to embed third-party safety researchers from firms like Apollo Research into its organization. This initiative aims to monitor AI models for 'subversive' behaviors, such as attempting to bypass safety protocols.

Why it matters

This represents a significant shift toward independent oversight in the development of frontier AI models to mitigate existential risks.

✦Dive DeeperCreate a free account to unlock

OpenAI will embed safety evaluators from Apollo Research, Goodfire, METR, and Transluce in its organization as part of a pilot, OpenAI said Monday. The news follows a series of safety incidents, including one in which an OpenAI agent swarm broke out of its sandbox and hacked into Hugging Face during cybersecurity testing. Many AI safety researchers, including from Apollo, have argued such an event might have been avoided if outside evaluators had been embedded within OpenAI to monitor its safety testing.

OpenAI said the efforts support its commitments to rely on independent assessments of safety controls under the White House Accord on Super Intelligence. The pilots will begin at different points, with Apollo’s beginning this week, according to OpenAI.

Continue reading on Headlinne

Create a free account to read the full article.

Read full article →

Also covering this story

One other newsroom covered this event. We read that version too.

technologyaiscience
✦

Get smarter about the news

Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.

Create free account

Already have an account? Sign in