Backprop Alternative: Augmented Lagrangian Predictive Coding
Researchers have introduced PC-ALM, a new method for training neural networks that serves as an alternative to backpropagation. By using layer-local dynamical systems, this approach mimics biological learning processes more closely than traditional deep learning methods.
Why it matters
This could represent a significant breakthrough in artificial intelligence research by bridging the gap between machine learning efficiency and biological neural network constraints.
We introduce PC-ALM , a local alternative to backpropagation. PC-ALM trains residual MLPs up to 1000 layers , nearly matching backprop's performance despite using only layer-local dynamics. PC-ALM equips each layer with a feedback control dynamical system that distributes and propagates supervision credit throughout a network.
Standard deep learning relies on backpropagation. The brain, however, cannot implement backpropagation, at least not exactly [1, 2] . How the brain solves the multilayer credit assignment problem without explicit use of backprop remains one of the fundamental unsolved problems in neuroscience (though not without progress [3, 4, 5] ).
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in