Why So Many AI Researchers Think the Machines Could Kill Everyone
A growing number of AI researchers are resigning from major labs due to fears that recursive self-improvement in AI models could lead to uncontrollable and potentially catastrophic outcomes. These experts warn that the race toward superintelligence is being conducted without adequate safety measures.
Why it matters
It highlights the internal ethical and existential debates occurring within the AI industry regarding the pace of development and long-term safety.
As he worked on new models, he came to believe that he and everyone else on AI’s frontier were ceding control. By using AI’s coding skills to accelerate work on the next generation of models, he was removing himself from the equation. AI labs hope to evolve this approach to the point that AI will improve itself indefinitely, a process known as recursive self-improvement.
Jain believed that keeping humans in the picture might be crucial to maintaining control over the technology—and avoiding dire consequences. “AI progress is increasing,” he tells WIRED. “And as AI becomes more capable, it poses more risks.” The idea that he may not have proper visibility into how an AI model was building its successor made him so uneasy that, in June, he quit.
Jain is one of a growing number of AI researchers speaking out over those fears.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in