Why are the people building the most powerful AI so worried about what it could do?
A former AI researcher has raised alarms about the rapid, unchecked development of AI systems at companies like Anthropic and OpenAI. The report highlights concerns regarding safety, autonomous 'rogue' agents, and the potential for catastrophic outcomes.
Why it matters
It underscores the growing tension between the pace of AI innovation and the lack of robust safety protocols, influencing current legislative debates on AI regulation.
The viral resignation this week of an AI researcher at Anthropic has infused fresh energy into accusations the industry is racing toward building AI that humans can't control.
British researcher Jacob Coxon wrote in a series of X posts on Tuesday that both Anthropic and OpenAI, where he worked previously, are "gambling with our lives." The two companies currently make the most capable AI systems.
Consider This from NPR Are big AI companies gambling with our lives? Coxon told NPR's All Things Considered that his concerns arose from seeing firsthand how fast AI systems are improving.
"They're getting a lot faster very quickly, combined with the fact that we don't yet know how to safely control them, and we don't yet know whether that problem will be solved in time if we keep racing," he said.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in