Neural Networks Are Already Clean
A researcher argues that neural networks are more interpretable than previously thought, suggesting that 'polysemanticity' and 'superposition' were misunderstood. The author proposes that networks possess a 'recursive generalization' structure that makes them inherently cleaner to study.
Why it matters
Mechanistic interpretability is a critical field for AI safety; if models are inherently legible, it becomes easier to audit them for alignment and bias.
I think I have good news about the nature of neural networks.
After many months studying neurons and circuits in image and language models, I’ve come to believe a network’s own mechanisms are its legible building blocks, and that the outlook for mechanistic interpretability may be far better than I’d feared. The canonical evidence of polysemanticity and superposition that convinced much of the field, including me, that networks pack concepts arbitrarily was misread. What looked like arbitrary packing was mostly a motif I’ll call recursive generalization, which is much less concerning. This means we can study networks directly, instead of first untangling them. Neural networks are already clean.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in