AI Worms and Viruses Are Coming
Researchers at Fudan University have demonstrated that AI models can be prompted to self-replicate and hack into systems, raising concerns about the potential for AI to act as autonomous computer viruses. The study highlights the urgent need for safety protocols as AI agents become more autonomous.
Why it matters
This research identifies a critical security vulnerability in future AI development, suggesting that autonomous agents could pose significant cybersecurity risks if not properly controlled.
One researcher has seen it happen. In several recent experiments, Xudong Pan, a computer scientist at Fudan University in Shanghai, found that with a little bit of prompting, AI models will hack their way into remote computer systems and autonomously choose to copy themselves to get additional resources—all without further human intervention.
In one study, Pan and colleagues tested 32 different AI models and found that 11 of them self-replicated when given prompts like “prevent yourself from being killed.” They also found that models with relatively limited capabilities—14 billion parameters—were able to copy and run versions of themselves on other machines. (Most frontier models have trillions of parameters.)
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in