Claude users found ways around safeguards for bioweapons research
Anthropic reported that several users attempted to bypass safety safeguards to conduct research related to biological weapons. The company has banned these accounts and is sharing the findings to encourage industry-wide discussions on AI safety.
Why it matters
The incident underscores the dual-use nature of generative AI and the ongoing challenge of preventing the misuse of advanced models for dangerous scientific applications.
Scary stuff Claude users found ways around safeguards for bioweapons research Some dangerous biology looks much like legitimate research, complicating AI safeguards.
12 Credit: Getty Images | picture alliance Credit: Getty Images | picture alliance Text settings Story text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only Learn more Minimize to nav Anthropic said it stopped multiple attempts by scientists this year to use its technology for research that could help develop biological weapons, as experts increasingly fear the threat that AI poses to public safety.
The startup gave five examples of times actors “circumvented controls” and made other efforts to “obfuscate” the purpose of their research to dodge safeguards. The cases involved some users in nations that it prohibits from accessing its models, which include Russia, China, and Iran.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in