AI’s quiet safety gatekeepers are stepping into the spotlight
Independent AI safety evaluators are gaining prominence as tech giants like OpenAI and Anthropic seek external validation for their models. However, the industry faces significant questions regarding funding, regulatory oversight, and the independence of these third-party groups.
Why it matters
As AI models become more powerful, the lack of a standardized, independent regulatory framework for safety testing creates potential risks for public safety and corporate accountability.
Two months ago, independent evaluators occupied a relatively sleepy corner of the multitrillion-dollar artificial intelligence industry. Now they're being asked to come to its rescue.
While Anthropic and OpenAI are at the heart of a fierce debate over whether they can safeguard their advanced models and grow their businesses simultaneously, they're seeking support from a handful of small third-party groups like Model Evaluation and Threat Research (METR), Apollo Research, and Transluce.
The evaluators, which mostly operate as nonprofits, are still finding their footing in an industry where capital is flowing at historic levels and new models are rolling out faster than ever. Their primary role has been to assess AI model capabilities and risks and to flag instances where the technology behaves badly.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in